Exploring Data through Sound - Victor Minces
Structured data vs. Unstructured data
- Sound waves, signals
Sounds are made of frequencies
Signal Generator
Spectrogram
Pink Trombone - Vocal Cavity Simulator
Frequencies of the Human Voice
Fundamental Frequency around 80/100hz
Many, many harmonics (~100)
Filtered harmonics
Data Science and Sound - Akshay Nagarajan, Victor Minces
Sound Representations of Data
Two directions
DataSound
SoundData
Data Sonification
Geiger Counter (1908)
The Geiger counter, one of the earliest examples of data sonification, converts radiation levels into audible clicks. It contains a small amount of gas that reacts to radiation, generating ions. Each ion produces a tiny amount of electricity, which is then converted into sound. More clicks indicate higher radioactivity.
Cosmic Sonification
Bullet cluster, dark matter
Data showing dark matter are represented by the lowest frequencies, while X-rays are represented by the highest frequencies
Machine Listening
Machine Listening
Machine listening is a fascinating field that enables machines to interpret and understand audio signals, beyond speech recognition including environmental sounds, animal calls, etc.
- Speech recognition: involves machines identifying and processing human speech to convert spoken words into text or commands
- Music Information Retrieval: analyzing music to extract information
- Environmental sound classification
Bestiary of Rhythms
Machine Listening Process
- Data Collection
- Preprocessing
- Model training: often CNN or RNN
- Testing and refinement
Tools for Machine Listening
- TensorFlow and PyTorch
- LibROSA
Data Collection in the Wild
Unstructured Bioacoustics
ARBIMON: classification based on distinct sound signatures for machine listening
Songlines as Memory Code
Naturalistic way of encoding memory in song lyrics