Audio Visual Brief

This project was set out for myself to learn about new ways of creating music and visual effects. Since I’m very interested in both sound and design I thought that exploring the relationship between the two would be interesting. My initial idea was to circuit bend a keyboard and visually represent it’s frequencies somehow. After researching how to hack/circuit bend keyboards, I came across an image of an Oscilloscope that was being used to measure the circuit board's voltages so that the person editing the keyboard knew that the electronic signals were working. Oscilloscopes are typically used to measure different frequencies and voltages.

I was immediately drawn into the lines and shapes that the oscilloscope could create once sounds were directed into it’s probes. Jerobeam Fenderson, who is a visual artist, created a kickstarter video featuring a way to manipulate sound waves to create crazy visuals onto oscilloscopes. This has then inspired me to purchase my own from ebay and experiment with different sound waves. 

VIDEOS

As soon as I discovered the visual possibilites that could be created using oscilloscopes I was sold, the kickstarter video below that was made by jerobeam fenderson is amazing. I was not aware that sound waves could be manipulated into created illustrations and scenery. Its crazy. On youtube I searched through many different related videos and came across many more similar outcomes. Even without searching for Oscilloscopes I happened to find more examples of it being used. An example of this was when I went to listen to music from Techno musician 'Clark', who has a strong visual presence with his music. Some of his music videos are all accompanied by wave forms made on an oscilloscope. 

My Oscilloscope

I successfully purchased a Kenwood Oscilloscope from ebay. It actually works but unfortunately one of the channels is weaker than the other, which means I can't produce shapes using the XY mode. The picture below shows that one line is flat, this line is the weaker channel. As the channel is weaker than the other it can't reflect perfect shapes and only flat ones. This needs to be fixed, so hopefully I can get it serviced in Leeds soon.

Development

I managed to buy myself a working oscilloscope on ebay. It can produce waves and react to sound fine. However, since it was a used item, it’s not in the best condition and one of the channels doesn’t work very well, which means that I’m not able to produce spheres or cubes. This is quite limiting and without those possibilities there’s not a whole lot that can be done. There fore I decided to develop my idea on representing sound into a visual, by wanting to add different visual elements that can react to sound.

Another element that I thought of integrating into my audio-visual work was my guitar tuner, which shows the notes of the guitar strings when they are played. I thought that the letters in combination with the oscilloscope waves could make some interesting visuals. To test out these visual elements I needed to first produce some music. I decided to use musical hardware for my production, because it’s a preference when making music. 

My guitar tuner.

First of all, the drum samples were loaded onto my sampler because I wanted to create the drums firsst and build the rest of the musical sections around it. I wanted there to be some melody in the song, so I sampled my Casio keyboard and a Japanese jazz composer called Joe Hisaishi. I was able to sample some beautiful clips from one of his compositions. Once I had arranged the drums and chopped down my samples into different sections, I was ready to record the beat into Logic Pro.

I also added in a synthesiser and a bass to each beat I recorded, because in order for the oscilloscope to respond a strong wave sound needs to be produced. I scrolled through the different synths inside logic and picked the ones that produced the strongest waves.

Here are some examples of waves I made using logic's synthesisers.

I recorded 2 beats into logic.

Once the Beats had been arranged and exported from Logic, I could then begin to create the visuals. For me to capture the Oscilloscope waves, I needed to use my SLR camera in the dark. I had to do the same for my guitar tuner too as it looks most effective in the dark. I connected the oscilloscope probes to my laptop and played the synth sections of my beats, I was then able to record the sound waves. I did the same with the guitar tuner; I connected a 1/4 jack from my laptop into it and played the bass sections to the tracks. I’m surprised that the tuner corresponded to the software instruments. 

After I had recorded these visuals and synced them to the music in After Effects, I decided that the overall composition was lacking something. I wanted to include more elements. I decided to create a visual response to the drum section in one of the beats by using the ‘cloner’ effect in Cinema 4 D. I exported my beat with just the drums and placed it inside C4d. Spheres were made to bounce and wobble to the sound of the drums. Once I was happy with the colours and how the rendering looked, I exported the video file and overlaid it within After Effects.

3D Element

Compositing all the visual elements together in After Effects. Since I have experience with After Effects it was pretty easy to edit the video, the hardest part was syncing all the videos to the music. This was vital in order to create an effective visual, if it was out of sync then the overall video project would be useless.

FINAL OUTCOME // EVALUATION

I feel that the final outcome has been a success, as I have produced what I intended to do. Different elements of the beats I made have a visual response and are in sync with each other. The overall video & sound has been efficacious, since the desired aesthetics were achieved. Although, I am going to be extending this project for my end of year show, I will continue to make further amendments and improvements. The Oscilloscope waves worked well for the synth sections, however more interesting shapes and lines could be created if both of my channels worked together. I plan to get it serviced soon so that I can achieve more effects. There needs to be more elements that are synched in with the different sounds that are present within the musical compositions. I aim to integrate more 3D elements, because I felt that they can provide a stimulating visual, which can grab peoples attention.

Keeping it more consistent for the end of year show is something that needs to be done. I'm happy with this video but there needs to be some restrictions with the colours used. I still don't know if I should keep the whole video Inverted or to just stick with the green waves. The inverted pink waves create an appealing white space in the background which looks better with other elements when they're added over the top. Figuring out the technical set up for my end of year show display is essential too, I need to supply a television and DVD player and figure out how they will be connected. I really want to get my hands on a Hantarex television but you can't purchase them anywhere, they're only sold to museums and galleries. Creating more beats and pushing these visuals further will be my next step as well as obtaining the technical equipment to present the work.