this article in PDF format
THE PARSONS INSTITUTE
FOR INFORMATION MAPPING
68 Fifth Avenue
New York, NY 10011
212 229 6825
piim.newschool.edu
Video Sonification
M I CH A EL FI LI MO WICZ , MFA
URL
http://filimowicz.com/pjim/
K EYW O R D S Audio-visual hermeneutics, experimental
music, experimental video, Fourier transform, site specificity,
sound design, spectrographs, synaesthesia, video mapping,
visual music
Video Sonification is the process of translating,
or mapping, video information into sound. This process
includes hybridizing aspects of sound design, visual music,
experimental music, and spectrographic sonic representation. My work in video sonification begins with selecting
highly structured images — typically taken from architectural contexts — that have the potential to map well into the
acoustic domain. The high incidence of repetition in the
built environment makes for ideal translations into stable
spectrographs, which depend on the regularly-repeating
bands of Fourier transforms. A composition is then shaped
that follows both visual and acoustic sensibilities, creating
an experience of audio-visual synaesthesia involving combinations of abstract and representational perceptions.
A BSTR A CT
M edia C l ips
The online presentation contains numerous links to video
clips showing works of sonified video, as well as historical
precedents in visualized music.
T ech n olo g ies
Video sonification begins with digital video that is translated via a Fourier transform (such as SoundHack’s Quicktime coder) into an audio file. Because the video is used as
a spectrograph for the resultant sound, footage containing
a high degree of x-y axial structure (such as is found in architectural contexts) produces the richest spectral content.
However, such translations of video into sound will typically produce disparate media types that are not of the
same duration, due to lack of a common time-base. To
correct for this, the original video clips will need to be
time-altered in order to regain synchronicity with the
audio file produced by the Fourier transform. Furthermore, this multi-stage process allows for the use of highdefinition video in Video Sonification works. Real-time
PIIM IS A RESEARCH AND DEVELOPMENT
FACILITY AT THE NEW SCHOOL
Figure 1, Figure 2, and Figure 3: Screencaptures taken
from http://filimowicz.com/pjim
© 2010 PARSONS JOURNAL FOR
INFORMATION MAPPING AND PARSONS
INSTITUTE FOR INFORMATION MAPPING
Video Sonification
MICHAEL FILIMOWICZ, MFA
Fourier transforms of HD video is prohibitively processing
intensive and software such as Cycling 74’s Jitter, used
for real-time audiovisual processing, works better with
lower resolution video files. With HD video footage, one
can make low-resolution renderings to be processed by
SoundHack, and the audio generated by SoundHack can
subsequently be combined with the original HD video to
produce synchronized HD Video Sonification works.
In addition to SoundHack, the works of video sonification discussed here are completed in video editing and
processing applications such as After Effects and Final Cut
Pro. The works produced can then be presented in a range
of formats such as site specific installation format, net art
or linear video media for screenings.
BI O G R A PHY
Michael Filimowicz is an interdisciplinary media artist
working in the areas of sound, experimental video,
creative writing, net art, public art, and digital photography. As a writer he has published poetry, fiction, and philosophy, and as a sound designer he has mixed soundtracks
for film and television. He is on the faculty in the School of
Interactive Arts and Technology at Simon Fraser University.
A CK N O W LED G EME NTS
Web Design: Andrew Hawryskewich, MS
School of Interactive Arts and Technology
Figure 4 and Figure 5: Screencaptures taken from
http://filimowicz.com/pjim
PA R SON S journal FOR INFO RM ATIO N M APPING
volume II issue 2, SPRING 2010
[page 2 ]
© 2010 PARSONS JOURNAL FOR
INFORMATION MAPPING AND PARSONS
INSTITUTE FOR INFORMATION MAPPING
Was this manual useful for you? yes no
Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Download PDF

advertising