Bounding-box accuracy on its own is a misleading metric for road AI. A working pothole-detection pipeline is: frame sampling, class-confident detection, spatial clustering, severity classification, operator review, training-loop feedback. The piece below walks through each step and explains why the raw per-frame accuracy number is less important than the five other things most comparisons ignore.
Every road-AI vendor publishes an accuracy number. The numbers are usually impressive and usually incomparable, because they measure different things. This piece is a guided tour of what actually happens inside a modern pothole-detection pipeline, and what the numbers on vendor decks do and don't tell you.
Step 1: frame sampling
A video stream from a windshield-mounted smartphone produces 25 to 60 frames per second. At normal urban speed (around 50 km/h) that is one frame every 25 to 55 cm. Feeding every frame into a detection model would waste 95% of the compute on redundant views of the same pothole. Instead, the capture app samples at a controlled interval, with extra frames triggered by IMU events (a sudden vertical acceleration suggests something underneath the vehicle worth looking at).
The choice of sampling rate is itself a design decision. Too sparse and you miss short-duration defects. Too dense and you waste compute without improving recall. RoadScout defaults to a rate chosen empirically to catch surface defects larger than a dinner plate at highway speed, and adjusts adaptively based on detected motion.
Step 2: class-confident detection
Each sampled frame passes through an object-detection network: in RoadScout's case, five production YOLO-family models covering surface distress, signs, markings, infrastructure and general assets. For every frame each model outputs a set of bounding boxes, each with a class label (for example "pothole" or "alligator cracking") and a confidence score.
The confidence score is the number most vendors publish as "accuracy." It is actually a much narrower concept: it is the probability the model assigns to this object matching the learned class. A 92% confidence detection does not mean 92% of the road needs repair. It means the model is 92% sure it has seen this geometric and textural pattern before in training data labelled with that class.
Step 3: spatial clustering
The same pothole gets captured across multiple frames as the vehicle approaches, passes, and recedes. Without consolidation the database fills with duplicates. RoadScout clusters detections of the same type within a 10 m radius into a single event. The event keeps the highest-confidence frames as its primary evidence, with the remaining frames attached as secondary evidence.
Spatial clustering is the step most often skipped in vendor demos because it doesn't photograph well. It is also one of the highest-leverage steps in the pipeline: a site that outputs un-clustered detections overwhelms the engineering team with 20 work orders for the same pothole.
Step 4: severity classification
Detection says "pothole is here." Severity says "this pothole is severe enough to trigger a work order." The severity rating is derived from three signals:
- Visual extent. Estimated depth, diameter and edge-spalling condition from the imagery.
- IMU response. The vertical-acceleration event as the vehicle passed over (or near) the defect. A high-amplitude event correlates strongly with a deep pothole; a flat trace suggests a surface-only defect.
- Network context. An edge defect on a single-lane rural road is a different severity from the same defect on an urban arterial. Context comes from the road-type classification layer.
The final severity rating aligns to the ASTM D6433 low / medium / high banding, so downstream pavement-management systems consume it directly.
Step 5: operator review
No road-AI pipeline is zero-touch. Every detection above the severity threshold is presented to the operator on the dashboard, with the evidence frames and accept/decline controls. The operator's decision is logged with the detection, which creates two valuable artefacts:
- An audit trail. Every decision to repair (or not repair) is traceable to a timestamped, evidenced detection. This is the piece that defends the engineering call if it is ever challenged.
- Training feedback. Declined detections that were false positives feed back into the training pipeline. Over 6 to 12 months of operation on a site, the site-specific false-positive rate falls substantially.
Step 6: training-loop feedback
The final piece is the one most vendor pitches don't mention: the model running on your network today is not the same model that will be running in six months. Declined detections (false positives) and missed detections (false negatives identified by operators via citizen complaints) both feed the retraining pipeline. The model evolves in the direction of your specific network, your specific vehicle fleet, your specific pavement age and material.
This is why site-specific accuracy is the number that actually matters, not the number on the vendor's initial deck. A 93% vendor-claimed accuracy on a North American training corpus can drop to 82% on a South African thermal-pavement network in week one, then climb past its starting point by month six as the model adapts. Vendors who can't show you the training-loop graph are showing you a static number that will bear no resemblance to your production experience.
The honest metric
When you're evaluating a road-AI pipeline, the question is not "what is your accuracy?" It is:
- What is the accepted-detection rate on our network after 90 days?
- What is the site-specific false-positive rate at the default confidence threshold?
- How do declined detections feed back into the training pipeline?
- How do you handle spatial duplication across frames?
- How is severity derived, and is it visible in your output?
Any vendor with a serious pipeline can answer all five. Any vendor that can only give you the raw per-frame accuracy number is showing you the sales-deck version, not the engineering version.