"Creating Visual Music in Jitter"

I found this great article on visual representation and synthesis of music and sound. The article is titled: "Creating Visual Music in Jitter- Approaches and Techniques".

This was the initial inspiration for my final project. It was written by Randy Jones and Ben Nevile. If anyone is interested in this article, please comment or email me and I'll send it your way.

some testing with video

Technical overview Pt.1

I started out with a PD patch that I tailored to derive various data streams from an audio input.

The heart of this patch is the fiddle~ object with derives frequency, pitch (in MIDI note values), amplitude (in dB), and harmonic content (also known as individual sinosoidal components; I only printed 3 harmonics). Fiddle is also set up to derive attack values, but I haven't implemented that feature in the main patch yet.

If I precede the fiddle~ object with an adc~ (input from microphone), then I get lots of audio data streams. O Yea!

If I set up outlet points after each number box with the individual data streams, I can just plug this patch into a larger visually driven patch.

Please note: When writing a patch like this in Max/MSP, which I had to do after I realized that my project was best suited for Jitter, you need to download the fiddle object and place it in the Max/MSP externals folder. Otherwise this patch will not work.

Also note: As you will see, the audio input patch is like a master control for audio and MIDI. While I don't necessarily plan on using MIDI for the final presentation, this can be an added feature later on.

Final Projecting

So here's the rundown. I started my project using Pure Data. I was hoping to make an audio controlled "painting" program using Gem and Open GL objects. In other words, the project could be split up into 2 parts: The analog audio input from a microphone (which includes the information derived from its signal) and the visual output (created by Open GL).

After much frustration and many many hours of playing with examples from the PD Extended browser, I realized that using Jitter as a visual programing environment would be much easier and it would look pretty good too. With the generous help of my friend Borna Sammak, we set out to conquer Jitter with a similar concept as my original plan, but angled more towards the manipulation of images as they are displayed on a screen. The following posts will give a general procedural and technical overview of the final project as it exists in its current form. Since Today is Tuesday and the project isn't due until Thursday (I know this sounds absolutely ridiculous), I might decide to modify it if I find something inspiring.

PD madness!!

here's a snapshot of some of the objects I'm working with for the audio input component on the final project. Right now my computer is a mess of help screens.

improvements and discoveries

I fixed the problem with my serial connection. Apparently Processing was looking at the wrong serial port (it tried to find an input on my modem's serial port). I changed the input to port [3], which is the usb serial, and everything worked ok. Additionally, I've been trying to figure out how to get an analog signal into PD and create a data stream. I finally realized that the adc~ object connect to an env~ object, and then I can output a data stream in dB. I used the data stream (which ranged from 60-79dB) to control a few different images in the GEM extended examples in Pure Data. It was pretty cool to control an image with a sound input but I'm still a little bit far from my goal of creating a visual artwork based on the sound. I need to figure out how to scale my sounds so that I can have a wider data stream to control different visual elements. Perhaps some visual examples will follow this post...

serial problems

I've been exploring options for my final project and I settled on an idea featuring MIDI, Serial communication, and some sort of audio imput and spectral processing in Pure Data. Unfortunately, I've hit a roadblock on the very first step. For some reason, when I try to open up a serial port in Terminal, I get weird error messages. Here is what I've done so far. I opened up

/Applications/Processing\ 0133/libraries/serial/macosx_setup.command; exit

and then attempted to type in a new terminal shell the following:

screen/dev/tty.usbserial-5B1 9600

// but I get an error message that says this: No such file or directory


Any thoughts? I will try to solve this problem ASAP. In the meantime I'll start to post some sketches for a final project prototype.