Abstract: In order to solve the problem of high real-time, and robustness to disturbances such as scene changes and wave reflections during visual detection in unmanned surface vehicle(USV) autonomous navigation, a real-time algorithm for visual detection of high-speed USV based on deep learning was proposed. First, a neural network MobileNet was arranged to quickly extract the full-image features. Then, a detection network based on SSD was used to fuse feature maps of each layer and achieve fast and multi-scale detection. Finally, the algorithm was implemented and validated on a hardware platform embedded GPU NVIDIA Jetson TX2. The results show that the proposed algorithm can quickly detect multiple types of specific obstacle on the water with strong robustness and multi-scale detection ability, and the detection speed of single-frame video within 50 ms.