Roundabout envelope

The question

A driving envelope says what most human drivers do at a place: how fast they enter, how hard they brake, where they put the car. It is only useful if it is calibrated: a 95 % envelope should hold 95 % of drivers it has never seen, not 95 % of the ones it was built from. This page builds that envelope at real roundabouts and tests it on held-out entries, with the tests and thresholds written down and committed before the data was opened.

The data

The ACFR Five Roundabouts dataset (Alex Zyner and Stewart Worrall, The University of Sydney, 2019, CC BY 4.0): road users tracked at 25 Hz by a parked vehicle's LiDAR at five single-lane roundabouts in Sydney, two entrances recorded at each. Positions, speeds and classes only: no images, no plates, no faces. It holds 3,204,917 rows and 23,912 tracks, and the paper's own counts by movement are reproduced exactly.

Only cars are scored (1,016 trucks, 448 bikes and 252 other tracks are counted, not scored). 11,368 car entries passed the pre-registered cleaning:

The tracker picks a vehicle up about 15 m before the entrance line, so the rule asking for 2.0 s of samples before the line drops most entries that cover those 15 m quickly. The rule was registered before the counts were seen and was kept: the envelopes describe the slower entries. The cars it dropped crossed the line at a median 8.1 m/s against 6.2 m/s for those kept, and 35.7 % were first seen within 10 m of it (2.5 % of kept cars): mostly the faster entries, some picked up late (exploratory). Each track is one road user passing the recorder; the same commuter may recur over the recording hours.

What was measured

For each entry: the speed at the entrance line, the lowest speed, the hardest braking, the largest sideways acceleration and the time to clear, from 20 m before the line (or the first sample) to the last. The envelope is the set of entries whose Mahalanobis distance from the training entries' mean, under a Ledoit-Wolf shrunk covariance, is at most the conformal threshold: the k-th smallest distance among the calibration entries, k = ceil((n + 1) x 0.95).

The whole-path band follows the median path at 1 m stations: each entry's sideways offset from it, scaled by the median absolute deviation at each station, and one threshold on the largest scaled offset along the whole path, so the band holds for the whole path at once, not station by station. Across roundabouts, a band pooled by conformalised quantile regression on each roundabout's path shape (MODELLED) is tested on the roundabout it left out.

How it is scored

A stratum is one entrance and one movement; it decides the verdict when it has at least 750 clean car entries (6 did). The headline tests hold out time: each stratum's entries, in recorded order, are cut into five blocks of recording hours, and each block is scored by an envelope and a band built without it (Tests E and F). The shuffled maths check holds out a random fifth instead (Tests A and B); it is expected to pass when entries are exchangeable. Tests A, B, E and F pass when coverage is 93 to 97 %. The control asks whether the straight-on envelope is specific: it should hold at most 80 % of turning vehicles (Test D).

Calibrated, not specific. On 7,262 roundabout entries, each held out with its block of recording hours, 6 of 6 strata with at least 750 entries passed every pre-registered test (Tests A, B, E and F). The straight-on envelope covered 96.4 % of left-turning vehicles at Oliver-Wyndora A, 86.5 % of right-turning vehicles at Orchard-Mitchell A, 92.4 % of left-turning vehicles at Orchard-Mitchell A, 93.2 % of left-turning vehicles at Queen-Hanks A, 80.9 % of right-turning vehicles at Queen-Hanks B and 90.9 % of left-turning vehicles at Queen-Hanks B, so it is not specific there. Pooling across roundabouts by quantile regression on geometry held at 1 of 5 held-out sites (Queen-Hanks 95.2 %, Leith-Croydon 90.2 %, Roslyn-Crieff 98.1 %, Orchard-Mitchell 70.7 % and Oliver-Wyndora 99.3 %).

Held out by recording hours

The envelope's (Test E) and the whole-path band's (Test F) coverage of each block of recording hours, built without that block, against the 93 to 97 % target. Leith-Croydon's blocks fall on different days in 2017; the other roundabouts were each recorded on one day in January 2018. Four of five roundabouts were recorded on one day, so the hours held out there are hours of that day. Had nothing changed over time, each stratum would pass Tests E and F with a probability of 99.5 to 100.0 % (Monte Carlo, Amendment 1), so the pass is weak evidence of stability.

Reported, not a pass rule: two blocks of hours fell wholly below the 93 % target: Leith-Croydon A, left block 1, 85.5 % (95 % CI 80.3 to 89.6) (it spans recording days, and where cars cross the give way line there moves by 0.49 to 0.54 m between recording sessions for left turns and straight on alike, so the miss may come from the recording set-up, not from driving) and Orchard-Mitchell A, left block 5, 84.3 % (95 % CI 78.2 to 88.9).

Exploratory, chosen after the result was seen, never a pass rule: with each recording session's median crossing point moved to zero, Leith-Croydon A, left covered 95.1 % (95 % CI 93.7 to 96.2) overall; block 1 93.7 % (95 % CI 89.7 to 96.2) against 85.5 % as registered and block 2 91.8 % (95 % CI 87.4 to 94.8) against 94.1 % as registered.

Where the 28 misses in Orchard-Mitchell A, left, block 5 are furthest outside (exploratory): 22 in the band's first 3 m, where the tracker has just picked the car up and 6 in its last 5 m, where the band ends. Across the six strata, 238 of the 371 entries the whole-path band missed are furthest outside in its first 3 m (53), where the tracker has just picked cars up, or its last 5 m (185), where the band ends (exploratory). A sup-norm band is decided by an entry's worst station, so stations few tracks reach weigh as much as the middle of the path.

The calibration plot

From the shuffled folds: the envelope's held-out coverage at four nominal levels (80, 90, 95 and 99 %), each with its Wilson 95 % interval. A calibrated envelope sits on the diagonal.

Coverage by the number of entries

The same envelope built from the first n calibration entries in recorded order, scored on the full held-out folds. Below 19 entries no 95 % envelope exists: the threshold needs the 19th of 19.

Is it specific?

The straight-on envelope (fold 0) applied to the same entrance's turning cars. A specific envelope would hold at most 80 % of them. It held far more: entry speed, braking, sideways acceleration and time to clear, seen over the last 15 m and through the roundabout, do not tell a turning car from a straight one here. Slower entries only: 10,589 of 22,196 cars were seen for under 2 s before the line and left out. Slow, yielding entries look alike whatever their exit, which may be why the control fails; that is not tested here.

Pooled across roundabouts

Our own data, for contrast

Our own data, one driver on one Worthing street over two days, was also tested across time: the envelope from one day covered 9.8 to 19.3 % of the other day's speed cells and 17.9 to 21.5 % of lateral position cells against a 50 % nominal.

A different test from the one above: 5 m cells at a 50 % nominal, one day's envelope scored on the other day, beside 95 % envelopes held out by hours. All four variables the file holds are shown.

How this compares

The same data has been used before for a harder question. Zyner, Worrall and Nebot (arXiv 1807.09995, IEEE Transactions on Intelligent Transportation Systems, 2020) trained a recurrent neural network with a mixture-density output that gives each vehicle's likely exit and path: it separates left, straight and right from the approach, which our five kinematic features cannot (the failed control above). Hasan and others (arXiv 2104.11180) follow it on the rounD drone dataset, whose viewers draw tracks over an aerial photo, so every line sits on a road.

Conformal regions around human motion are standard in robotics (Lindemann and others, IEEE RA-L 2023, arXiv 2210.10254), and recent work argues they must be calibrated per scenario because intervals widen at junctions (arXiv 2512.05682). Against these, this page lacks a test on the path that tells movements apart, a test across many days at more than one site, a plain baseline envelope to show what the conformal step adds, and any interaction with circulating traffic. What it adds is a hashed pre-registration, a failed control shown plainly, and every figure rebuilt from the public file by one command. Three independent reruns inside MapMap, from the public zip, produced the same files.

What this is not

• This is Australian left-hand traffic used as a method check, not a UK result.
• Not a risk model and not a rating of these roundabouts: no collision data is used and no road is called anything.
• No automated vehicle is scored here.
• No driver identities, images or plates: positions, speeds and classes only.
• Two of each roundabout's entrances were recorded; the others are not assessed and are hatched on the page (arms per roundabout from the dataset's companion paper: Leith-Croydon is a T junction, the others have four arms; each hatched arm is where the recorded entrances' turning cars leave).
• The envelopes describe the slower entries the cleaning kept, and a commuter may recur over the hours.

Sources and licences

Five Roundabouts dataset, Alex Zyner and Stewart Worrall, The University of Sydney, 2019, CC BY 4.0 (doi:10.25910/5c64b8b7dba5b). Companion paper: A. Zyner, S. Worrall and E. Nebot, “ACFR Five Roundabouts Dataset: Naturalistic Driving at Unsignalised Intersections”, IEEE Intelligent Transportation Systems Magazine. Record: ses.library.usyd.edu.au/handle/2123/20001, DOI 10.25910/5c64b8b7dba5b. The dataset is used under the Creative Commons Attribution 4.0 licence; nothing here implies the authors' or the University's endorsement. MapMap modified it: tracks were cleaned by the pre-registered rules, resampled to 5 Hz for drawing and moved into one frame per roundabout. Cleaned, resampled, analysed and rendered by MapMap.

How it is rebuilt

Every figure on this page is rebuilt by MapMap from the public zip with one command: ingest, scoring, contrast, the exploratory step, the drawings and a final check. The exploratory step is post hoc and never a pass rule. The tests and thresholds are in the pre-registration (rules SHA-256 93ac6fbc51a2cadfe549110fcb19bc76e2e9d1f6824ad21600c9536fd6ee5734), committed before the test code existed and amended once before the data was downloaded. Zip SHA-256 2e63f402ba2a997e6ff166414c663e7beebb919e1d998fbb07a6fee274d547db.