Get frames off the submarine and into the model
A host-side capture loop pulls the submarine's video stream, timestamps frames, and hands them to inference.
Why this stage exists
The detector is only as good as the frames it receives. Underwater frames arrive slowly, unevenly, and with wildly varying colour and contrast, so acquisition has to be its own stage rather than a line of glue code.
How it was done
- 1
Pull the stream on the host, not on the hull
The submarine exposes its camera stream over Wi-Fi through the antenna buoy. A host-side script connects to that stream and turns it into an ordered sequence of frames with capture times attached, which is what both the detector and the later spatial registration need.
- 2
Treat ~5 fps as the design point
At SVGA and roughly 5 fps, consecutive frames overlap heavily. That is good for redundancy — the same object is usually seen several times — and it is why detection counts later have to be read as detection events rather than distinct objects.
- 3
Keep raw frames for offline work
Every survey writes its frames to disk. The same footage feeds live detection, later re-training with newly collected underwater imagery, and the batch photogrammetry pass.
- 4
Filter detections at a confidence threshold
For a frame captured at time t, the detector returns a set of bounding boxes with confidence scores. Only detections above a threshold are forwarded to localisation, which keeps obvious turbidity artefacts out of the map.
Where the code lives
get_stream/Frame acquisition from the ESP32-S3 camera stream: connect, decode, timestamp, write to disk, and feed the detector.
- get_stream.py
What still bites
- Wi-Fi through a floating buoy is not a stable link; dropped frames are normal and the pipeline has to tolerate gaps.
- Automatic exposure swings between bright surface water and dark basin floor, which changes detector behaviour within a single run.