Observed driving envelope

The question

The Automated Vehicles Act 2024, section 2(2)(a) requires the statement of safety principles to be framed with a view to securing that “authorised automated vehicles will achieve a level of safety equivalent to, or higher than, that of careful and competent human drivers” (not yet in force; legislation.gov.uk). That is a standard for the principles, not a test of one driver or one street, and this page makes no claim under it. It measures an observed envelope, what one driver did on one real UK street, and where a driving model's paths leave it.

What was measured

12 passes, 1 driver, 2 days (wet, dry), 1 loop of 1.37 km, 10 Hz LiDAR-SLAM trajectory. The rig's trajectory is split into passes at a start gate, resampled every 5 m along the loop, and each pass gives speed, longitudinal acceleration, lateral acceleration and lateral position at every station. The street is drawn from open data: OpenStreetMap footprints and roads, and Environment Agency 1 m LiDAR terrain and heights. The scene uses the real street geometry, so the street is identifiable from its shape.

The speed limit

OpenStreetMap has no speed limit tagged on the loop. The signed limit is 30 mph (48.3 km/h) on the whole loop (signed, confirmed for MapMap on 23 Oct 2026), so the page applies a rule layer (Highway Code rule 124): The observed envelope is capped at the signed limit: the band's upper edge is the lower of the band as driven and the limit. Speed above the limit is over the limit, not inside the envelope, whoever drove it. Our own driver was over it on 9 of 12 passes, on both days, peaking at 54.0 km/h. The table gives every day and direction (counts against the rule, not pre-registered):

So the speed band as driven includes driving above the limit, which no envelope of lawful driving should include. The band is drawn capped at the limit, and speed above the limit is never counted inside it (decided for MapMap on 23 Oct 2026; not pre-registered). On the Speed measure the scene draws each pass in the hazard colour where it was over the limit, with a key beside it; on the other measures the passes are ink. In the 20 model samples no scored point, the driver's or the model's, is above the limit (0, 0 and 0 points), so the speed figures below are the same with the rule applied: model minus driver +13.0 points (−3.5 to 28.7) with route text. Where it does change a figure is the cross-day test, shown below with the rule applied and as driven.

Under the street, the road and verges are our own LiDAR (MEASURED): drive one's 3 cm cloud of 27 Sep 2026, ground returns only, within 14 m of the loop, one point per 0.2 m cell, thinned by MapMap; their reflectivity is the value, rasterised on the page into one texture at that cell size and draped on the terrain. The layer starts switched off (Layers turns it on): at that cell size its texture hid the carriageway in the opening view. Every point is a ground return under 0.25 m above the ground, so the layer can hold no person, vehicle or number plate; the cloud of moving objects was never read. The buildings are RENDERED from open data; the model's paths are MODELLED. Roads off the measured loop are drawn flat and dark: they are not measured, and nothing on this page reads as safe.

The observed human envelope

Per-station split-conformal band per direction: at each 5 m station a pass's score is its absolute distance from the station median of the calibration passes, and the band is the median plus or minus the k-th smallest score, k = ceil((n + 1) x level). It promises the level at each station, not for a whole pass at once.

The page draws the 80 % band, with the pass that each model sample starts from held out of it. A whole-pass (sup-norm) band, one promise for the whole lap. Its width is set by each pass's single most unusual station, so on 6 passes it is too wide to say anything; shown for honesty, not used for scoring.

The calibration test

The test asks that held-out passes fall inside the 95 % envelope between 93.0 to 97.0 % of the time. Result: fail. k = ceil((n + 1) x 0.95) = 4 > n = 3 per direction (7 > 6 for both directions); a 95 % envelope cannot be built from these passes. It needs 19 passes per stratum.

Within one session, leaving one pass out proves nothing. Within one session, leave-one-pass-out coverage is fixed by the number of passes, not by the driving: a held-out pass is outside exactly when it is the most extreme. With 6 passes that is 5 of 6 for a whole-pass band and 4 of 6 at each station for a min/max band. Reported, never used as evidence. The only real test here is across days: build the band from one day's passes and count the other day's stations inside it.

Intervals are 95 % percentile intervals from resampling the six held-out passes whole (5,000 resamples), because the stations of one pass are not independent. They count only how passes vary within the held-out day; the larger uncertainty, how one day differs from another, cannot be measured from two days. Treating each held-out pass as one yes-or-no observation instead gives far wider intervals (speed, wet day's band: 5.6 to 62.7 %). Either way, one driver's two days do not cover each other. Part of the position gap may be how the two drives were placed on the map (each was georeferenced on its own), so position is the weaker of the two findings; speed has no such confound.

The model's paths

The paths are the existing outputs of our left-hand bench run of nvidia/Alpamayo2-Super (2 Oct 2026): 20 samples on this loop, each with and without route text, six paths each (240 in all). No new inference was run. Each sample's ego history and future were placed on the rig's trajectory by a rigid fit (median error 0.04 m, worst 0.11 m). The page draws each mode's medoid path solid and the other five faintly. A point more than 6 m from every human pass is off the measured loop and is not scored. The model was fed pinhole views reprojected from our 360 camera, with the travel direction uncertain by about 5 degrees, not the cameras it was validated on: these are bench results, not its behaviour on the road.

The control matters more than the model. The driver's own next 6.4 s, scored against the same band the same way, is also often outside it:

Points outside the 80 % band, pooled over all 20 samples. The difference is model minus driver in percentage points, with a 95 % interval from resampling the 20 samples in pairs (5,000 resamples). It is clear, with the interval above zero, only for position and cornering; for speed and accel the interval includes zero in at least one mode, so we claim no difference there. The driver's own figure is well above the aim, which says as much about a band from five passes as about the model. The driver's future and the model's samples all come from the wet day, scored against a band of wet and dry passes, which inflates speed for both. Nothing here is a prediction or a pass mark.

Position and camera yaw

The model saw views whose travel direction is uncertain by about 5 degrees, and the LiDAR mount sits 1.5 degrees off the ego frame. A straight line fits each model path's drift from the band's middle well (median R squared 0.72 with route text, 0.78 without), at a median heading of -5.6 and -6.2 degrees (negative is toward oncoming traffic); the driver's own path drifts at -0.6 degrees. So much of the position figure could be a heading error of the size the bench cannot rule out. Turning every model path by the same angle, from minus 5 to plus 5 degrees, and scoring it again gives a position figure of 72.0 % to 80.6 % with route text and 73.3 % to 81.3 % without: one shared yaw error does not explain it away, but an error that differs from sample to sample could.

Position and camera yaw, per sample

So each sample's path was also turned on its own, by every whole degree from minus 5 to plus 5, and kept at the angle that makes it look best: the lowest position figure of the eleven. That is the best case for the model and the worst case for any claim about it. The rule and its pass mark were written down and hashed before this was run (pre-registration 5482301b0734): the position difference passes if its 95 % interval excludes zero with and without route text. Result: pass. One thing to weigh: the range of 5 degrees was fixed after the median heading-drift fit above (-5.6 and -6.2 degrees) was already known; it is the bench's stated camera-yaw uncertainty, not a range chosen to pass.

The driver's own figure, 30.7 %, is not turned: it is measured, not inferred. Given the same freedom for fairness, the driver's path would read 18.6 %; that row is reported, never used for the pass. Many chosen angles sit at the sweep's edge, so a heading error larger than the bench's 5 degrees could shrink the difference further. At the best-case headings the model's 5 m cells beyond the band number 73 right of it and 36 left with route text, 80 and 32 without. The strongest sentence this allows is that the position difference survives a per-sample yaw sweep; the mirrored-world control (below) fails its pre-registered bar, so the gap is not shown to come from left-hand traffic, and nothing here says the lean is a right-hand-traffic habit.

Not pre-registered, a sensitivity check only (added after an independent review): the same sweep with the range widened.

Left turns and right turns

The left and right counts above read like a lean toward oncoming traffic. Split by the driver's turn, they say something simpler. On a UK left turn the inside of the corner is the kerb, so a path beyond the band to the right is wide of the corner; on a right turn, beyond it to the left is wide. In these 20 samples the model's paths leave the band wide of the corner on both: too wide, not a side preference. The best-case headings agree: on clockwise passes the median chosen angle is -4.0 degrees, on anticlockwise ones 5.0 (with route text), a turn toward the inside of the loop both ways, where one constant heading error would turn every sample the same way. This split was not pre-registered; it was added after review.

Straight cells

Corners make lateral position noisy, so the scoring was repeated on straight road only: stations where the loop's curvature is under 1/150 per metre (radius over 150 m) at the station and both neighbours, 429 of 550 stations. Most samples start at a corner, so only 4 of 20 samples have any straight points scored (150 points of the model's path, 196 of the driver's), and those samples had no route text, so both modes gave the same paths. On them the model's path is beyond the band at 54.0 % of points and the driver's at 38.3 %: a difference of +15.7 points (−11.7 to 38.2), the same sign as on all cells, with an interval that includes zero. Too few samples to say more.

The page opens on the median sample by its own model-minus-driver position gap with route text (Sample 2 of 20: right turn, a gap of +51.3 points against a median of +51.5 points); every sample is in its menu. Choosing it by gap was not pre-registered.

Beyond the kerb line

Not pre-registered (added after review, following the drivable-area check in public planning benchmarks): for each sample, the furthest any point of the 6.4 s path lies beyond the kerb line. The kerb line is the OpenStreetMap road centreline plus half its width (6 m for a residential road unless the width is tagged): an estimate, not a measured edge. The model's path is more than 1 m beyond it in 12 of 20 (38.7 to 78.1 %) samples with route text (median of the furthest points 2.6 m, worst 12.2 m) and 10 of 20 (29.9 to 70.1 %) without (1.3 m, 13.1 m). The driver's own path is more than 1 m beyond it in 6 of 20 (14.5 to 51.9 %) (0.3 m, 6.4 m). A driver on the road crossing the line by metres means the line itself is out by metres in places, so this figure cannot settle whether a path is on the road. For the same reason the scene draws the line faint, never as the edge of the road.

The mirrored-world control

A per-sample yaw sweep cannot say why the model's path sits where it does. The mirrored-world control asks whether it is a right-hand-traffic habit: the model runs again on a left-to-right mirror of every sample (images flipped, left and right cameras swapped, the route text's left and right swapped), and its mirrored paths, mirrored back, are scored against the same band. Its pass mark is fixed in the pre-registration: the real paths beyond the band more often than the mirrored ones, with a 95 % interval above zero, with and without route text. Status: fail. Real minus mirror, position: −2.0 points (−7.9 to 3.9) with route text, −4.0 points (−8.9 to 0.6) without. The mirrored paths leave the band 78.1 % of the time with route text and 77.3 % without, against 76.1 % and 73.3 % for the real ones. The control fails its pre-registered bar: the gap is not shown to come from left-hand traffic. The model swings about as wide in a mirrored world as in the real one, so the lean is not a right-hand-traffic habit this test can see. It may come from how the model takes corners or from our input; this control cannot tell those two apart. One GPU run, 24 Oct 2026, on the same 20 blurred samples, flipped.

UK left-hand rules

Counts from rules written for UK left-hand traffic. Text rules count reasoning strings matching our rule, and none is called an error. On multi-lane UK roads “right lane” is often correct. Left and right counts carry the bench's camera-yaw uncertainty of about 5 degrees.

What it does not show

• Headway and gap acceptance: other road users are not measured.
• Roundabouts: the loop has none.
• Time of day: all twelve passes were driven at about the same time in the afternoon.
• Jerk: 10 Hz timestamp jitter makes it noise.
• New model inference: the paths are the left-hand bench's existing outputs.
• Camera frames: they stay unpublished; the page shows the paths only.

Sources and rebuilding

Map data © OpenStreetMap contributors (ODbL). Contains Environment Agency information © Environment Agency and database right, OGL v3.0. LiDAR and trajectories by MapMap.

Every number on the page and here is rebuilt by MapMap with one script from the rig drives, the bench outputs and the cached open data, and a rerun is checked to be byte-identical.