Results

Field deployment in Peter Street Basin produced three things worth reporting: a detector that survives turbid water, a trajectory with detections registered in real basin coordinates, and a head-up display that turns those coordinates into a direction and a distance.

Detector training progression

Collapsing the label space did more for accuracy than any other single change, and adding underwater imagery did the rest.

Configuration	Images	mAP50	Precision
YOLOv8n, TACO, multi-class	1,500	0.27	0.37
YOLOv8s, TACO, single-class	1,500	0.49	0.71
YOLOv8s, combined, single-class	12,177	—	0.921

TACO (terrestrial litter)

~1,500

TrashCan (underwater marine debris)

included

Collected by our team underwater

~1,000

Other public underwater repositories

remainder

Total training images

12,177

of which underwater scenes

5,720

Qualitative underwater detections

These two frames show the range of appearance the model has to handle: partial occlusion, low contrast, and heavy colour shift from attenuation and scattering in basin water.

Two underwater frames with bounding boxes: packaging debris in turbid water, and a small distant detection near the basin floor. — Detections from the Peter Street Basin deployment: packaging/plastic debris in turbid light (left) and a small, distant, low-contrast object near the basin floor (right).

Basin mapping

The COLMAP sparse reconstruction registered 796 of 823 above-water frames into one consistent model. Camera positions were extracted and transformed into basin coordinates through the ArUco anchor points. This run logged 130 detector hits, which are frame-level detection events along a continuous trajectory rather than 130 de-duplicated physical objects.

Plot of the perimeter survey path around Peter Street Basin with accumulated garbage detections in basin coordinates. — A representative survey trajectory in basin coordinates with accumulated detections. The path is derived from the COLMAP sparse reconstruction of 796 registered frames, starting at the southeast corner; this run logged 130 detector hits.

Head-up display output

The goggles report the target class, a range of 4.1 m, a compass arrow toward the selected target, and a miniaturised basin map with registered detections.

Head-up display through the Vuzix Smart Swim lens: a compass arrow, a 4.1 m range readout, the class label OTHER, and a small basin map. — The XR head-up display in operation: heading arrow, 4.1 m range readout, target class, and a miniaturised Peter Street Basin map with registered detections — all inside the small monocular display window.

Honest limitations

Repeated observations of the same debris inflate raw detection counts unless track association or spatial clustering is applied, so the 130 hits in the plotted run are detection events, not 130 distinct objects.
Underwater image degradation still causes false negatives for low-contrast or heavily occluded items.
The dome port only partially corrects refraction, and COLMAP's intrinsic calibration was performed for the above-water camera only.
The fixed transform between the above-water and underwater cameras (45°, 0.3 m) was measured roughly and propagates into underwater localisation error.
The submarine's 5 m maximum depth and ~30 min battery constrain survey area and duration for larger or deeper water bodies.

What comes next

De-duplicate the map

Multi-frame association and DBSCAN spatial clustering would turn repeated detection events into confirmed physical debris locations.

Enhance images before detection

Colour correction and dehazing ahead of inference would compensate for turbidity and wavelength-dependent attenuation.

Close the loop on collection

Given confirmed debris locations, the system could compute an efficient collection order by solving a shortest-path or vehicle-routing problem over the detection map, and prioritise items likely to drift out of reach.

Run inference on the goggles

A TFLite export running on the Vuzix Smart Swim's own Android processor would remove the host laptop from the pipeline for simple detection tasks.

Why the combination matters more than any one part

A detector on its own can say “there is probably garbage in this frame.” That is not something a swimmer can act on. Adding basin photogrammetry gives the detection a coordinate, and adding the XR display turns the coordinate into an instruction: this direction, this many metres.

The same reasoning drove the single-class decision. Knowing whether an item is a bottle or a can changes nothing for a person about to pick it up, so the label space was spent on finding objects instead of naming them. Classification returns as future work for waste composition analysis, where the answer feeds policy rather than a swimmer.