The genetics of sound

Computers are starting to hear just like we do.

I learned about Melodyne software from Andrew Hearst's great site, panopticist. Melodyne is a piece of sound editing software made by the German company, Celemony. It's good for things like high quality time-stretching and pitch shifting, processes other software can also accomplish. Probably, Melodyne does it a little better than most. However, this fall they're planning to release Melodyne plugin 2 (full version plus plugin will go for $399), which many say is gonna revolutionize the sound studio. I think it's gonna revolutionize music, period. View their promo video if you want.


I've said "holy shit" a lot more than I normally do on a Friday morning. Melodyne's new feature, Direct Note Access, is the ability to analyze polyphonic audio content and separate notes within a chord. Once they're separate, you can alter each individually, shifting, stretching, muting little parts of the audio, not the whole file. And it really does. About 11 minutes into the video below, creator Peter Neubacher moves a trumpet melody around without changing a thing about the (minimal upright bass) accompaniment. Right now, the software might be hard pressed to pick a flute line out of a giant orchestral texture, but based on what it can do now, they'll surely advance the software in the next few years. Even within solo lines over accompaniment, glissandi are harder to deal with as opposed to straight pitches.



Mashups have become wildly popular (and sadly illegal). Those who adhere to antiquated value systems don't think Girltalk is artistic. They say the software is meant for altering one's own recording, but this new program will inevitably make the complex mixing and editing that Girltalk does a lot easier, and will therefore explode the genre. It will make more disparate songs easier to mold together. If more people can do what he does, then he'll have to step it up. This is great for DJ culture. Now we can change chord progressions, add dissonance, or any number of other things to existing recordings instead of just juxtaposing unrelated songs. It could be a great way to comment on other artists. Maybe a mashup explosion will push people to reevaluate what they consider original in music and--way, way, down the road--when a giant consumer market presents itself that would outweigh profits from untouched originals, the legal system will adapt to what people want to pay to hear.

Celemony may have deliberately called this plugin Direct Note Access because of its acronym, DNA. Much like the discovery of smaller and smaller parts of our genetic code, this audio DNA is pulling out smaller and smaller parts of audio information, and altering it. In biology, a genetic mutation usually means a catastrophe, but in digital sound, it can be quite the opposite.

Usually, theories are theories, and often evade reality. I like Neubacker's reversal of this norm: "the more I pondered the subject [of DNA], the more I began to see that what doesn't work in theory can still work in reality." This post seems like an advertisement for this company. It's not. I'm just excited.