On Monday 13 May, Witteveen + Bos and DigiShape organised a hackaton on underwater sound. With a diverse group of experts in the field of underwater acoustics, signal processing (signal processing) and/or machine learning and AI, we worked on the question:
What reprocessed parameters (with an eye on minimal data storage) can you derive from sound recordings that still give you enough input for vessel recognition with your favourite AI model?
While there is potentially a wealth of information available in the field of underwater noise (think of ship recognition, piling, explosions at sea and seismic surveys), the data are difficult to share widely because of the size of the sound files and the labour-intensive process of labelling. Witteveen + Bos and Rijkswaterstaat won the 2023 DigiShape seedmoney to identify ways in which sound files can be ‘stripped down’ so that they can be shared in a smaller format, without losing essential information.
Niels Kinneging from Rijkswaterstaat about underwater noise
We started the day with an introduction from Niels Kinneging of the Rijkswaterstaat’s Underwater Noise team. He was project leader of the European project Jomopans from 2018 to 2022, in which 11 institutes from all countries around the North Sea developed a framework for monitoring underwater noise in the North Sea using measurements. He discussed what the term underwater noise actually means, what natural noise sources there are and which sounds are caused by humans. He also explained what effects this noise can have on marine life and in what ways you can analyse it.
Niels: ‘Noise measurements contain a wealth of information. By making the measurements publicly available, it is possible for experts from different disciplines to look at the data in a different way. We all learn a lot from this.’
Hilde Hummel from CWI on Machine Learning and underwater sound
Hilde Hummel of Centrum Wiskunde & Informatica (CWI) then talked about her PhD research with the topic: The State-of-the-Art Machine Learning in Underwater Acoustics. She sees that in recent years there has been a real evolution in the number of applications in the field of Machine Learning and AI. This is good news because there is still a lot of unexplored territory when it comes to underwater acoustics. At the same time, researchers and developers still encounter many challenges, which include the complexity of the underwater environment and the limited amount of labelled data.
Hilde: ‘Very nice that DigiShape and Witteveen+Bos have brought together so many experts in this underwater field. Together we know more.’
Marieke Bezemer from Witteveen + Bos about the results of the hackaton
Marieke Bezemer, who is leading the project from Witteveen + Bos, was very pleased with the initial impuls that was given during the hackaton. ‘You can see that there is still quite a gap in the Netherlands between specialists in the field of underwater noise on the one hand and specialists in the field of Machine Learning on the other. Today we were able to bring these two fields closer together.’
As expected, the challenge is to reduce all fragments while making them suitable for the variety of intended applications. One possible solution route is to filter the audio fragments in advance, which gives you a better idea of which files are relevant. Based on relevance, you then decide how much information to store per fragment.
Marieke: ‘Depending on the background of the expert and the intended application, there may be different needs and preferences. We have seen that some participants preferred to convert the fragments into spectrograms and then feed them into an AI algorithm as a whole, while others further process these spectrograms into features and work with these reprocessed parameters. There was also extensive discussion during the hackaton about what makes a sound clip good or useful. Is this a fragment with sound sources well above the noise level? Or, on the contrary, do we want to use fragments, which are contaminated, for our algorithms? Again, the difference in the intended application plays a big role.’
At the end of the day, the various ideas for reducing the dataset were discussed in detail. These included:
- Selection of fragments based on labels. The distance of a ship from the hydrophone and the type of ship.
- Method of calculating a spectrogram.
- Division of the fragments based on wavelet analyses, where in fact a different reduction level can be achieved per frequency band.
- Deriving features based on the frequency analyses, calculating reprocessed parameters.
Possible follow-up
Clearly, when it comes to underwater noise, we are sitting on a goldmine of information. But in order to make this data available in an efficient and usable way, a lot of steps still need to be taken. During this hackaton, an initial impuls was given to this and in the process, as expected, potential barriers and questions were also raised. In the course of 2024, we will take stock of whether there is a need for a follow-up with this group and whoever wants to join it.
