Interactive Body Music App: Built with AudioKit

Marc-André Weibezahn has created Affine Tuning with AudioKit!  Use your body movements to create unique musical experiences with a collection of interactive compositions which react to your movements in real-time.

Don’t worry! You don’t have to be a dancer to use it. The musical feedback will always sound in tune, which lets you focus on your expression. We interviewed the developer and he shared his experience creating this fun new app. Plus, he has some advice for people building their first music app!


INTERVIEW WITH Marc-André Weibezahn:

What was your inspiration for making the app?

I had done reactive room installations before, which used body tracking and responded with sound as the only feedback channel. The main goal of these was to create an exclusive connection between movement and sound that goes both ways. Later, I wanted to create a similar kind of experience, but give the audience authority over the place, time and duration. This need for mobility, together with the good quality of iOS’ “Motion Capture” feature, led to the decision to make an app.

What was it like using AudioKit?

When started with AudioKit in late 2019 I already had some years of iOS development experience, but very little experience with audio programming and I only knew some synthesis basics from an introductory course at university. So my experience with AudioKit was dominated by learning both about coding for audio, and digital audio in general.

The example playgrounds helped me a lot, and I think now with the cookbook, AK should be even easier to pick up. I wont lie though, if you are coming to this completely fresh, at some point you will have to learn about the fundamentals of synthesis, sampling, and all the other stuff. I cannot imagine to having done this using iOS’ audio framework directly.

AK gave me easy access, including powerful functionality, like timestretch/pitchshift, bitcrusher, different reverbs and much more. I would often discover that something I’d like to try out was only a few lines of code away.

What did you learn from making the app – Any advice for people?

I feel I can best give advice to other “beginners”:

On the topic of audio programming: If you are coming from programming for graphic driven systems, especially in an OOP way, you are used to create and throw away graphical objects (views etc.) all the time. Do not translate this approach 1:1 to audio programming! Learn about the audio graph first, and then try to find a model that fits its more static behaviour. I had to rewrite the part of my system which interacts with the audio graph twice because I did not pay enough attention to this.

And in general, I think it is easy to underestimate the vast field that is digital audio, mostly because there is not as much general knowledge about it, as sound in media is very underrated, compared to graphics and images. The interactions of the smallest details, down to the level of the waveform, are very complex and, in my experience, less predictable than with graphics. But that also means that there are many interesting and exciting things to discover, if you bring time and patience for exploring.

Besides technical topics: Try out your ideas as quickly as possible and show them to other people! It will give you energy and motivate you to go on. It is much too easy to hide behind your screen and always stay on a technical level, but if you ultimately want other people to interact with your work, you should go out and meet them.

Thanks Marc-André!

Download Affine Tuning:

Available for iPhone & iPad

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.