Oyster reefs have significant advantages for the benthic marine ecosystem(s), including boosting species richness and offering habitat, sustenance, and protection for a wide range of marine organisms. Population decline over the 19th century has dramatically impacted oyster standing stocks in the Chesapeake Bay. Massive restoration efforts for oyster habitat are underway throughout the United States and Europe. Effectively tracking the development of oyster reefs is one of the main obstacles to advancing adapting the restoration process. General metrics for oyster reefs include areal dimensions, reef height, oyster density, and oyster size-frequency distribution. These measurements rely on the identification and counting of oysters by skilled human labor. Oyster reefs are manually subsampled with as few as 100 oysters per sampling site. Additionally, similarities between the bottom substrate and the oysters themselves make them difficult to distinguish for people and for computer algorithms. To streamline the process of oyster mapping, we use advancements in robotics and artificial intelligence to gather images from underwater Remotely Operated Vehicles (ROVs) and then automate oyster detection and density calculation. In this study, we provide a mathematical model to generate synthetic oyster image data and employ generative adversarial networks to facilitate the sim-2-real transfer. To the best of our knowledge, this is the first attempt to geometrically model oysters.
In this experiment, we train the convolutional neural network (CNN) using our real dataset (Oreal) and test using OysterNet and another method (DCO). The Intersection over Union (IoU) scores are 18.16% and 18.88% respectively which serves as the baseline for oyster segmentation results for our dataset. Both techniques perform similarly in these cases. Next, we evaluate the model performance using only the synthetic dataset for training (Osyn). We use both techniques to train on Osyn and test on O. The IoU score is lower than our baseline at 7.45% and 6.47%, respectively. Although the network has acquired the ability to detect synthetic oysters, the transfer from the sim to the real world is lacking. For training, we combined a tiny quantity of real data with synthetic data (Osyn_and_real), which yielded better results. In comparison to expert human-labeled ground truth, we achieved a state-of-the-art IoU Score of 24.54%, which is 35.1% better than utilizing only real datasets, and 12.7% better than DCO when trained on synthetic augmented real data.