Give every detection a real place in the basin

Why this stage exists

A bounding box in a video frame is not actionable. To send a swimmer to a piece of debris, the detection has to become a coordinate in a frame that a person can navigate — metres east and north inside Peter Street Basin, at a depth.

How it was done

1. 1

   Survey the basin with a three-camera rig

   For the batch survey we walked the perimeter of the basin carrying three GoPro cameras. The top camera stayed above water at all times to capture surface structures and ArUco markers. The middle camera, in a 6-inch dome port, floated half above and half below the surface, imaging both regions in one frame while reducing refraction distortion; its buoyancy helped hold that half-submerged pose. The third camera was fully submerged, angled about 45° down toward the wall being scanned.

2. 2

   Reconstruct camera poses with COLMAP

   SIFT keypoints were extracted from the above-water sequence with COLMAP's GPU feature extractor, using the OPENCV_FISHEYE camera model for the GoPro's wide-angle lens. COLMAP jointly optimised intrinsics, converging to fx = 2440.5, fy = 2445.0 with four fisheye distortion coefficients.

3. 3

   Match sequentially instead of exhaustively

   Because a perimeter walk gives high overlap between consecutive frames, sequential matching replaced exhaustive matching and cut roughly 338,000 candidate image pairs down to neighbouring pairs.

4. 4

   Build the sparse model

   The incremental mapper registered 796 of 823 images into one consistent sparse reconstruction of about 415,000 3D points with a mean reprojection error near 1 pixel; the 27 frames that failed to register were discarded. Each registered frame yields a pose (R, t), from which the camera position is recovered as p = −Rᵀt.

5. 5

   Anchor the reconstruction to the real basin with ArUco markers

   ArUco markers from the 4×4_50 dictionary, 19.5 cm per side, were placed at known positions on the basin walls and detected in the above-water sequence with OpenCV. Their positions were read off a pre-existing Blender 3D model of the basin — a more precise reference than tape-measure field work — and used as ground-control points to fit a rigid transform from COLMAP coordinates into the basin frame. A further rotation aligns the basin's long axis and absorbs the roughly 25° offset between the basin frame and geographic north.

6. 6

   Project detections into basin coordinates

   With the camera pose (x, y, z) and heading ψ known, each accepted detection contributes a relative bearing θ and range r estimated from image geometry and calibration. The mapped point is x + r·cos(ψ+θ), y + r·sin(ψ+θ), z + Δz, and it is appended to the running basin map. For the handheld rig, the underwater camera's pose was derived from the above-water camera through a fixed transform: about 0.3 m offset at 45° to the left and downward.

Decisions and trade-offs

Where the code lives

What still bites

• Intrinsics were calibrated for the above-water camera only, and the dome port corrects refraction only partially.
• The rig transform between the above-water and underwater cameras was measured roughly, so underwater detections inherit that error.
• Repeated views of the same object inflate raw counts until track association or clustering is applied.