THE PARSONS INSTITUTE. FOR INFORMATION MAPPING. 212 229 6825 piim.newschool.edu. URL http://filimowicz.com/pjim/. KEYWORDS Audio-visual ...
THE PARSONS INSTITUTE FOR INFORMATION MAPPING
68 Fifth Avenue New York, NY 10011
212 229 6825 piim.newschool.edu
Video Sonification M I CH A EL FI LI MO WICZ , MFA
URL
http://filimowicz.com/pjim/
K EYW O R D S Audio-visual hermeneutics, experimental music, experimental video, Fourier transform, site specificity, sound design, spectrographs, synaesthesia, video mapping, visual music
Video Sonification is the process of translating, or mapping, video information into sound. This process includes hybridizing aspects of sound design, visual music, experimental music, and spectrographic sonic representation. My work in video sonification begins with selecting highly structured images — typically taken from architectural contexts — that have the potential to map well into the acoustic domain. The high incidence of repetition in the built environment makes for ideal translations into stable spectrographs, which depend on the regularly-repeating bands of Fourier transforms. A composition is then shaped that follows both visual and acoustic sensibilities, creating an experience of audio-visual synaesthesia involving combinations of abstract and representational perceptions.
A BSTR A CT
M edia C l ips
The online presentation contains numerous links to video clips showing works of sonified video, as well as historical precedents in visualized music. T ech n olo g ies
Video sonification begins with digital video that is translated via a Fourier transform (such as SoundHack’s Quicktime coder) into an audio file. Because the video is used as a spectrograph for the resultant sound, footage containing a high degree of x-y axial structure (such as is found in architectural contexts) produces the richest spectral content. However, such translations of video into sound will typically produce disparate media types that are not of the same duration, due to lack of a common time-base. To correct for this, the original video clips will need to be time-altered in order to regain synchronicity with the audio file produced by the Fourier transform. Furthermore, this multi-stage process allows for the use of highdefinition video in Video Sonification works. Real-time
PIIM IS A RESEARCH AND DEVELOPMENT FACILITY AT THE NEW SCHOOL
Figure 1, Figure 2, and Figure 3: Screencaptures taken from http://filimowicz.com/pjim
© 2010 PARSONS JOURNAL FOR INFORMATION MAPPING AND PARSONS INSTITUTE FOR INFORMATION MAPPING
Video Sonification MICHAEL FILIMOWICZ, MFA
Fourier transforms of HD video is prohibitively processing intensive and software such as Cycling 74’s Jitter, used for real-time audiovisual processing, works better with lower resolution video files. With HD video footage, one can make low-resolution renderings to be processed by SoundHack, and the audio generated by SoundHack can subsequently be combined with the original HD video to produce synchronized HD Video Sonification works. In addition to SoundHack, the works of video sonification discussed here are completed in video editing and processing applications such as After Effects and Final Cut Pro. The works produced can then be presented in a range of formats such as site specific installation format, net art or linear video media for screenings. BI O G R A PHY
Michael Filimowicz is an interdisciplinary media artist working in the areas of sound, experimental video, creative writing, net art, public art, and digital photography. As a writer he has published poetry, fiction, and philosophy, and as a sound designer he has mixed soundtracks for film and television. He is on the faculty in the School of Interactive Arts and Technology at Simon Fraser University. A CK N O W LED G EME NTS
Web Design: Andrew Hawryskewich, MS School of Interactive Arts and Technology
Figure 4 and Figure 5: Screencaptures taken from http://filimowicz.com/pjim
PA R SON S journal FOR INFO RM ATIO N M APPING volume II issue 2, SPRING 2010 [page 2 ]
© 2010 PARSONS JOURNAL FOR INFORMATION MAPPING AND PARSONS INSTITUTE FOR INFORMATION MAPPING
