Last night I talked for hours with my dear friend David Holmes – a true thinker and inventor, expert in music and meditation, and uniquely talented writer (see his wonderful book on the mind, brain and its intimate relationship to sound and light). It was a crash course on harmonics, timbre, resonance, overtones and the missing fundamental. Most of all, he came up with some wonderful new ideas for sonifying brain activity. To summarize, consider:
- The harmonic relationship between the frequencies of neuronal oscillations: If there is (a heightened) activity of, say, 12 Hz, one could investigate its harmonics, at e.g. 24 Hz & 36 Hz.
- The tonal relationship between frequencies of neuronal oscillations: If there is activity of 16Hz or 23Hz, what ‘note’ would that corresponds to? (Answer: the first C and F#).
Now, the idea is to implement these in let’s call it the the Holmes patch, by doing the following:
- Identify a (peak) frequency. This might be done by finding a peak amplitude in the spectrum, possibly separately for high and low frequencies. It might be based on its dynamics over time (stable or not stable), or it might be selected by hand.
- Determine its harmonics. This might be a full harmonic series, only the odds, or others (see here and here).
- Play these frequencies corresponding to their musical notes (after transposing octave or so upwards), with their volume determined by their relative amplitude (relative to each other). This latter point neatly solves the issue of normalization of amplitude in cases where you want it to express power of a frequency.
What you get with this approach I leave to your imagination for now: it will sonify the relationship over time between a fundamental low frequency and its higher harmonics, creating a change of timbre over time. It has to be noted that this would be hard to implement in CV/gate, but trivial in MIDI.
Finally, we discussed briefly the possibility of using CV/gate to express the peak amplitude in a frequency-window, e.g. that of alpha (8-12Hz). One needs quite a time-window to have some resolution, e.g. 4 seconds for a minimum 0.25 Hz resolution to make it work. However, sliding time-windows will still make it a smooth experience. And how quickly would you be able to shift your peak alpha frequency anyway?
Further study today landed me on this great informative site of the acoustics lab of the University of South Wales. Besides their great material explaining sounds generation and acoustic concepts such as harmonics in wind instruments in general, I found their findings on the acoustics of the didjeridoo very interesting (this one published in Nature).
To conclude, a purpose of this blog is to share ideas for neuro-acoustic feedback that can be useful in any artistic, scientific, experiential (fun! exciting! inspiring!) or even therapeutic ways (although neurofeedback is a controversial issue to which we will have to return at some point). This new field of neuro-acoustic feedback is one of experimentation and exploration that still lies wide open to all our ideas.