Autonomous vehicles can only operate reliably if they have robust 6-degrees of freedom (6- DoF) localization capabilities. Basing critical autonomous decisions on highly uncertain localization information can potentially lead to catastrophic outcomes that not only risk the success of the autonomous vehicle's mission, but also endanger human lives.
Aruco markers are often used as an effective way for autonomous systems to be able to locate themselves in relation to rigid objects. Here we summarize an approach published in [1] where we developed a system for 6-DoF estimation of Aruco markers with associated uncertainties in the challenging underwater environment. A state-of-the-art object detection framework (EfficientDet) was adapted to predict the corner locations of Aruco markers (see Figure 1), while dropout sampling at inference time is used to estimate the predictive 6-DoF pose uncertainty.
Figure 1: Network structure. We use an EfficentNet backbone network for feature extraction, a bi-directional Feature Pyramid Network (BiFPN) for efficient feature fusion, and separate class and corner prediction heads.
A dataset of Aruco markers captured in a wide variety of turbidities, with ground truth position of the corner locations, was gathered and used to train the network to robustly predict the 6-DoF pose.
We report translational errors of 2.6 cm at low turbidity (8.5 m attenuation length) and up to 10.5cm at high turbidities (0.3 m attenuation length) while the associated uncertainty (inter-quartile range) ranges from 3.2 cm up to 27.9 cm. The rotational errors varied from 5.6◦ to 10.7◦ with uncertainty of 6.4◦ to 26.2◦. In Figure 2 we show results of the prediction of the Aruco corner with associated uncertainty at different turbidities.
|
|
|
Figure 2: Corner prediction overlaid on figure for attenuation lengths 8.6m, 0.7m and 0.3m. We have plotted iso-contours of the marginal probability of the different corner locations. The prediction is relatively stable across turbidities, except for at the highest turbidity where there is limited signal. At the highest turbidity we observe that the distributions are multimodal and considerably wider than at lower turbidites
One direct application of the proposed approach to 6-DoF pose estimation is for autonomous interventions subsea. If an Aruco marker is rigidly placed in relation to a fish-tail handle which the AUV should intervene with, the AUV can automatically position itself and the gripper in relation to the fish-tail. The gripping procedure can be adjusted according to the uncertainty of the pose estimate of the fish-tail. With high uncertainty, the movements can be slower, and the gripper can open up more before closing up the gripper. This will help reduce the risk of damaging the gripper and the fish-tail.
In Figure 3 we show an example where we have used the pose distribution (1 000 samples) given by the proposed algorithm when detecting an Aruco marker at an attenuation length of 0.7 m at 90 cm distance. We create a marginal probability volume around the gripper which tells us the probability of the fish-tail being present in that particular voxel. This is done by transforming the fish-tail handle with the 1 000 pose samples and for each time a voxel is inside the transformed model an accumulator is incremented for that voxel. When the gripper (the yellow model to the right in the figure) is closing its grip, we can report the probability of whether it is now gripping the fish-tail. The snapshot of the gripping process in the figure shows that the probability of the grip is 99 %.
[1] Petter Risholm, Peter Ørnulf Ivarsen, Karl Henrik Haugholt, Ahmed Mohammed. "Underwater marker-based pose-estimation with associated uncertainty", 2021 IEEE/CVF International Conference on Computer Vision Workshops, 2021.