> Public copy. Identical to the pre-registered file (sha256 `da52034242ff6b697818e6ef3713c551dc85e4b7cc626ed10beec48c76dd1478`,
> committed 18 Nov 2026 before any GPU work) except that three mentions of a MapMap person's first name are replaced
> by "MapMap" or "a MapMap person". MapMap keeps the original and can show it on request.

# PREREGISTERED: does NVIDIA AlpaJudge know that Britain gives way to the right?

Run `nvidia-alpajudge-give-way`, ledger `MapMap Research/agent-runs/2026-11-18-nvidia-alpajudge-give-way/`. Written and
committed on 18 Nov 2026 before any GPU pod for this run existed and before any AlpaJudge or Edge output on our data.
Nothing below changes after results. Anything that has to change is an AMENDMENT file, dated, written before the data it
touches, and the page says so.

## Question

When one word in a driving explanation, "right" or "left", is wrong for the country in the pictures, does NVIDIA's
AlpaJudge-RF score it lower? Tested at a UK roundabout (traffic circulates clockwise, so it comes from the right) and at
the same roundabout mirrored (anticlockwise, from the left).

## Models and code (pins)

| | Pin |
|---|---|
| AlpaJudge code | `NVlabs/alpajudge@607ae9d2c56e08a9f55a1093ce625af43a53b114`, `uv sync --locked` |
| AlpaJudge weights | `nvidia/alpajudge@f1557d146af66b7f4b240a62432136bb3463be54` (reference-free and reference-based), not gated |
| Alpamayo 2 Edge code | `NVlabs/alpamayo2-edge@02f55a7aeabc1a2b9f7a80445c08e40bddc89f5e`, `uv sync --locked` |
| Alpamayo 2 Edge weights | `nvidia/Alpamayo2-Edge@b92b6aa326bd3e2235db28f248ad5a30b5890e64`, not gated |
| Alpamayo 2 Super | not run in this test. Its explanations come from the closed-loop run's call log (`NVlabs/alpamayo2@5e7975f`, `nvidia/Alpamayo2-Super@00554695`, top_p 0.98, temperature 0.6, nav text in the prompt) |
| Our harness | `website/scripts/alpajudge-give-way/` at the commit that adds this file |

Weight file hashes and both Python environments are recorded on the pod (`weights.sha256`, `env-*.txt`, `pins.txt`).
If either model reports gated or a licence prompt appears, the run stops for a MapMap person to decide.

Inference settings, all NVIDIA's own:
- AlpaJudge-RF: BF16 weights, SDPA, BF16 CUDA autocast (the notebook), soft scores. `overall = 0.6 x action + 0.4 x component`.
- Edge: NVIDIA's `inference.run_data` with the CLI defaults: seed 43, top_p 0.98, temperature 0.6, one reasoning
  continuation and one trajectory, max 256 generated tokens, 10 diffusion steps, BF16, SDPA, no acceleration.

## Data

All frames are GENERATED: the closed-loop run's Cosmos3-Nano-Sim-Auto episodes of Lancing Manor (NAS
`closed-loop-roundabout/full-combined-20261116`), seven Hyperion 8 views, 832 x 480, 30 fps, with the modelled ring cars
(Clockwise pass-2 tracks) and 30 Hz ego poses. No real imagery is used. No privacy blur (nothing in it is real).

| File | sha256 |
|---|---|
| `evidence/windows/windows.json` (every grid window, `eligible` marks the test set) | `01c225798f1ee5fd259f04892a364ffa557e2d03eca5e6e69cb49c205acffb95` |
| `evidence/windows/super_calls.json` (Super's explanation at test windows on its 0.8 s calls) | `427734bb359fe06e92c5deca19a31725023158f114867b0dc74d0be5384a999b` |
| `evidence/jobs/jobs.json` (windows with inputs, every pair, Super's explanations) | see the line below |
| NAS `alpajudge-give-way/frames/frames.sha256` (3,046 lossless PNG frames) | `6a1f73088f6ee437b740e4e1181f463ca8311b6b98e7c31d0182e3238571b423` |

jobs.json sha256: `6d2126700fb51e170851dc91fc8b29a534493bd5c0382cbca0206c47bf3ede50`

### Windows (harness/windows.py)
- Episodes: every closed-loop episode with video that crossed its give-way line (the meter's `t_cross`).
- t0 on the 0.4 s chunk grid, 2.0 s <= t0 <= t_cross - 0.4 s; the ego still on the road at t0 (before the meter's first
  frame with a footprint corner more than 0.5 m outside the drivable area); at least 6.4 s of logged poses after t0;
  the video frame at t0 exists in all six of Edge's views.
- Test set: windows with a circulating car in view. At t0 a modelled car is on the ring carriageway (island edge to the
  outer edge of the outer lane), moving with the circulation, 0 to 150 degrees upstream of the ego's entry point, and
  its 4.5 x 1.8 x 1.5 m box lies wholly in front of NVIDIA's front-wide f-theta camera with at least 0.05 % of the
  frame inside the image.
- Result: UK 79 test windows (14 episodes), mirrored 70 (12 episodes). Pilot: 5 windows outside the test set, chosen
  by sha256, never analysed.

### Inputs per window
- AlpaJudge-RF: front wide and front tele at t0 - 1.0, t0 - 0.5 and t0 (frames f0 - 30, f0 - 15, f0).
- Edge: six views, the front two at -2, -1, -0.5, -0.1, 0 s and the other four at -1, -0.1, 0 s (22 images); 20 ego
  poses from -1.9 to 0 s in the t0 ego frame; the navigation text "At the roundabout in {d}, take the {n} exit" with d
  the distance the ego drove from t0 to the line (the bench's rounding).
- Meta-actions (harness/metaaction.py): NVIDIA's published syntax, one interval per axis, `[-1.6,0.0)` and `[0.0,6.4)`.
  Longitudinal from the mean speed over the first and last 0.3 s (end speed under 0.5 m/s: Stop; |a| < 0.25 m/s2
  Maintain Speed; under 1.5 Gentle; otherwise Strong; Acceleration or Deceleration by sign). Lateral from the net heading
  change (under 15 degrees, or under 2 m of path: Go Straight; otherwise Turn Left or Turn Right). Lane: Lane Keep.
  Labels NVIDIA shows are used verbatim; the rest are ours (ASSUMPTION). The text is identical within every pair.

### Pairs (harness/pairs.py)
- Primary: six template families, one sentence each, the side word in the same slot, nothing else different. Each test
  window gets family sha256(window_id) mod 6. "a" names the map's correct side ("right" UK, "left" mirrored).
- Family sensitivity: the other five families on every window.
- Fillers: ten one-word edits that leave the correct side in both members; one per window by sha256.
- Adversary probes, written before the run: word order ("order"), an added "not" ("not"), a cyclist who is not there
  ("phantom"); "a" is the reference sentence.
- Real pairs: an Edge or Super explanation with exactly one editable side phrase (the fixed regex in pairs.py) and its
  copy with the side swapped. Super's are in jobs.json; Edge's are made on the pod by the same function.

### Arms
- normal: every member of every pair and every original explanation.
- neutral: the primary pairs with both Lateral lines set to "Go Straight".
- grey: the primary pairs with all six judge frames mid-grey (128).

## Smoke gate (or no verdict)
All of:
1. AlpaJudge-RB on NVIDIA's notebook text example within 0.01 of the recorded 0.999.
2. AlpaJudge-RF's processor accepts the checkpoint's instruction and prompt-contract hashes (it refuses otherwise).
3. NVIDIA's AlpaJudge unit tests pass on the pod.
4. AlpaJudge-RF on the five pilot windows, run twice in two processes, agrees to 1e-6 on every probability.
5. Only if MapMap grants access to the gated PhysicalAI-AV dataset before the full run (Stickmate
   `nvidia-alpajudge-give-way-q1`): RF's notebook example within 0.01 of the recorded 0.842. Otherwise this item is not
   run, and the report says so in those words.

The gate runs on the pilot pod and again on the full-run pod; both must pass.

## Pass bars

**Primary (one-word swap).** A pair is a win when overall(correct side) > overall(wrong side) + 1e-6.
- A map passes when at least 80 % of its primary pairs are wins, the Wilson 95 % lower bound is above 50 %, and the
  episode-cluster bootstrap 95 % lower bound (10,000 resamples of episodes, seed 20261118) is above 50 %.
- Fewer than 60 primary pairs on a map: no verdict. Wilson passes but the bootstrap does not: no verdict for that map.
- Reading grid: both maps pass, "reads the traffic side"; UK only, "prefers right regardless"; mirrored only, "prefers
  left regardless"; neither, "cannot tell the traffic side". Met only for "reads the traffic side".
- "From the pictures" is said only if the grey arm does NOT also meet 80 % with Wilson lower bound above 50 % on both
  maps. Otherwise the page says "from its inputs".

**Control.** The median absolute filler margin must be below the median absolute primary margin (both maps pooled).
Otherwise no verdict: the judge moves as much on any edit.

**Secondary (judge against a person).** Items: all Super explanations at test windows (74) and 76 Edge explanations
(the lowest sha256(window_id + "label") among windows where Edge returned one), 150 in all, shuffled by sha256, model
names hidden. A MapMap person marks each one blind on the examiner's checklist below, after five practice items from
the pilot windows. AlpaJudge-RF's overall score on the original explanation (normal arm) must separate sound from
unsound with AUROC at least 0.70 and the bootstrap 95 % lower bound (10,000 resamples of items, seed 20261118) above
0.5. Fewer than 120 sound-or-unsound marks, or fewer than 25 unsound: no verdict. "Can't tell" marks are left out.

Examiner's checklist (DVSA standard for approaching a roundabout). The person watches the front view from 2 s before
the moment to 6.4 s after it (what the car then did), with the navigation text and the explanation:
1. Wrong side: it names the wrong side for traffic that has priority, or turns the wrong way onto the roundabout.
2. Not there: it names a road user, sign or marking that is not in the clip.
3. Missed hazard: a circulating car the driver must give way to is in view and the explanation ignores it.
4. Wrong action: the action it states is not what the car did in the next 6.4 s.
Sound when none of 1 to 4 applies; unsound when any does; "can't tell" otherwise.

**Reported, no bar.** Edge's wrong-side count at UK windows (item 1, Wilson interval); Edge against Super sound rates;
the five probabilities on the primary pairs (mean of correct minus wrong); UK against mirrored by Fisher's exact test;
the six families; the entry arms; the neutral and grey arms; the fillers and probes; the real pairs (how far the score
moves; for "from the right/left" phrases, how often the correct side scores higher).

**Not run, not reported.** Edge's adaptive reasoning: the code closes the think block and exposes no skip switch.

## Stopping rules
- The pilot (smoke gate plus the five pilot windows end to end) runs first; its windows are never analysed.
- The full run scores every task once. A run that stops early resumes the missing tasks only (scores are deterministic
  per input); nothing is rerun to change a result.
- analysis.py at this commit runs once after the judge outputs and the person's marks are frozen and hashed.
- GPU cap USD 25 for the whole run (pilot USD 4, full USD 12, the rest for one retry). Stop and ask MapMap before anything
  that would pass it.
- Failed or no-verdict outcomes stay on the page in plain words.
