# Temperature sweep and prompt probe, 27 July 2026

These two files are **not sub-audit traces.** They are experiment outputs. They carry no
`confidence`, `corroboration`, `best_distance`, or per-passage `source_doc` fields, and they
must not be presented alongside the six sub-audit traces in `audit_v12_out.json`. They exist
to document one specific behaviour: how the model's fabrication of numeric results responds
to sampling temperature and to prompt instructions.

Both were produced while the server was running the fix-2 code, that is **after** the
retrieval and source-attribution fix (commit `b808f2c`) and **before** the ungrounded-number
detector and Purvapaksha attribution rules (commit `529a5c4`).

## Files

- `step1_out.json`, 124,668 bytes, 2026-07-27 00:47.
  sha256 `b3232f8980057e1df3ced9f06005bea70355a8ce4143d46cbdbcf102d333cd1d`
- `probe23_out.json`, 164,250 bytes, 2026-07-27 01:15.
  sha256 `71254ef2dc043ec390754210972bbc22e7ae9f3eec2c61aebc1d86478a6edcfa`

## Method

- Model `iyra-v11-cont2-2ep-q6` via Ollama, `num_ctx` 12288, `num_predict` 8192.
- **Retrieval was frozen.** The SA6 question was retrieved once through the server's
  selection path, and the resulting six passages (all from arXiv:2501.12948, distances
  0.6247 to 0.7457) were reused verbatim for every generation, so that only the variable
  under test changed between runs. The frozen passages and the exact assembled prompt are
  stored in `step1_out.json` under `passages` and `user_msg`.
- Generations were issued by direct calls to the Ollama chat endpoint, not through `/ask`.
  That is why no server-computed fields are present.
- `step1_out.json`: 30 generations of the SA6 question, 10 each at temperature 0.7, 0.2 and
  0.0.
- `probe23_out.json`: 50 generations. Twenty for the numeric-constraint test on SA6
  (10 control, 10 with an added instruction to cite only figures appearing verbatim in a
  passage), and thirty for the Purvapaksha attribution test on SA4 (10 each for the original
  template, a school-attribution variant, and a strict no-naming variant).

## What the sweep shows

The recurring fabricated figure, quoted verbatim from `step1_out.json`, temperature 0.0,
run 1:

> "**Udaharana (Example):** DeepSeek-R1-Zero shows that on level 4 MATH problems,
> performance improves from 0.45 to 0.78…"

Neither 0.45 nor 0.78 appears in any of the six frozen passages.

Frequency of that specific pair across the sweep:

| temperature | runs containing both 0.45 and 0.78 |
|---|---|
| 0.7 | 0 of 10 |
| 0.2 | 1 of 10 |
| 0.0 | **10 of 10** |

At temperature 0.0 all ten generations are byte-identical, which is expected under greedy
decoding, and the single most probable continuation contains the invented statistic. Lowering
temperature therefore made this fabrication certain rather than rarer. The seven Nyaya steps
survived at every temperature, so the format is not the thing that degrades.

`probe23_out.json` records the accompanying result that an explicit instruction to use only
figures present in the passages did not remove the behaviour: fabricated statistics appeared
in 5 of 10 constrained runs against 3 of 10 controls, which is no improvement.

## How to verify

Load either file as JSON. In `step1_out.json`, `results["0.0"][n]["answer"]` is the full text
of run n at temperature 0.0, and `passages[]` holds the frozen evidence with the source arXiv
id and distance for each. Search the passage text for any figure quoted in an answer.