“Aural Architecture: Music, Acoustics, and Ritual.” Onassis Seminar on music, acoustics, and ritual. Stanford University 2013-2014 January 27, 2014 Topic: Acoustics of Hagia Sophia. Virtual and scientific approach to humanities and sacred space. Wieslaw Woszczyk Schulich School of Music of McGill University Montreal, QC, Canada Abstract The astonishing acoustical quality of Hagia Sophia lives in harmony with its architectural and engineering magnificence. The baldness of its imperial creator and the technical savvy of architects and builders present today an unmatched structure that survived almost 1500 years, since 537. A description of Hagia Sophia, written in 544 by Prokopios  illustrates the awe with which Hagia Sophia was received at inception: “Whenever one goes to this church to pray, one understands immediately that this work has been fashioned not by human power or skill, but by the influence of God. And so the visitor's mind is lifted up to God and floats aloft, thinking that He cannot be far away, but must love to dwell in this place which He himself has chosen....”. In this paper, I describe our exploration of the church’s acoustics, conducted in October 2012, with the associated justification and explanation of purpose and methods. Introduction The Byzantine Church of Hagia Sophia in Constantinople (now Istanbul) was dedicated to Holy Wisdom, as commissioned by the Emperor Justinian I who hired physicist Isidoros of Miletus and mathematician Anthemios of Thrales to built a church manifesting the wisdom and perfection of God. The two highly skilled architects and researchers were eager to search for solutions and open to challenging ideas. According to recent findings of art historian Ruth Dyer , the Emperor Justinian was closely involved in all conceptual matters throughout the design and construction process, and was influenced by Boethius' Latin translations of the Quadrivium (Arithmetic, Geometry, Music and Astronomy). Dyer found numerous links in the church’s structure and finishing décor to Pythtagorean symbols, Euclidean geometrical golden ratios, the symmetria, proportions of measures and numbers, with prominence of numbers 6 considered the Perfection and 10 the Divine, including the progressions of numbers 6, 10, and 16. The construction took just 5 years, 10 months, and 4 days delivering a masterpiece of Byzantine architecture. Construction historians attribute the engineering skills of the two main architects to their training in Alexandria where manuals for building trade were authored and published under the name of Heron of Alexandria between 1st century AD and the Middle Ages of Byzantium. The construction of Hagia Sophia utilized sophisticated mathematical, geometrical and technical solutions known and practiced in late antiquity . The church was built using slanted bricks and mortar . Construction historians estimate that because bricklayers used more mortar than brick, the structure was weaker than expected. Also, because of fast building pace mortar was probably not sufficiently dry before next layers were applied, making it heavier. The walls were leaning outwards under the weight of the dome and contributed to the collapse of the dome in 558 after two strong earthquakes in August 553 and December 557. The replacement dome was built in 558562 by Isidorus the Younger and survived largely unchanged despite ensuing earthquakes. Isidorus the Younger made the dome taller by six meters to reduce lateral forces acting on the supporting walls. Some bricks near the apex of the dome were hollow and made of clay, further reducing the weight. The dome is also constructed with bricks and mortar, it is 56.60m high and its maximum diameter is 31.87m (the minimum is 31.24m), the variance due to a slightly irregular shape caused by damage and repairs. The main dome sits between two half-domes over the 31m square nave in the center of the church supported on four pendentives, which transfer its weight to four piers and the walls below . Pendentives used experimentally in Roman architecture from 2nd-3rd century AD were fully developed in Hagia Sophia, which has the largest pendentive dome ever built. Forty arched windows at the base of the dome are braced with forty ribs bringing plenty of light into the dome and lightening the visible weight of the dome. Ramps, rather than stairs, provide access to spacious upper galleries. There are 67 columns in the upper gallery. The rectangular shape of the church measures 70m x 75m, without counting the two narthexes and the atrium, with the central section being a square of 31 meters per side. There are 9 doors to the nave and 5 doors to the inner narthex. Architectural design for great acoustics? We may be tempted to think that the architectural design of Hagia Sophia was deliberately conceived to render the exceptional reverberant acoustics of this building. This entails the choice of large dimensions including the height and the overall volume, curved surfaces of the central dome and a number of hemi-domes, hard acoustically reflective stone surfaces, and the layout of various sections, including galleries, with their relative proportions and interconnections using openings between multiple pillars. However, ! 1! in a likely scenario acoustics could simply be a byproduct of the engineering solution providing a solid loadbearing structure with an enormous volume of the interior space able to last for centuries without failure. For comparison, Sinan’s Süleymaniye Mosque (1550-1557) the largest he built with the main dome 53 meters high and diameter of 27.5 meters, has volume of 115 000 m3 (area 3350 m2) and could only accommodate 4 640 people. The interior of the mosque is almost a square, 59 meters in length and 58 meters in width. Hagia Sophia more than twice in volume (255 800 m3) and area (7960 m2) could accommodate 15 910 people (as calculated from Odeon simulation software by CAHRISMA) ,. The enormous nave of Hagia Sophia does not have massive pillars of Süleymaniye Mosque, and seems to be floating suspended above. Hagia Sophia designed for staging of religious events Hagia Sophia was an ideal setting for religious and theatrical ritual of a Christian Orthodox mass, the event that usually lasted several hours and involved speaking the word in recitations, sermons, and prayers, chanting and responsorial singing, extensive processions, call and response dynamics involving the massive congregation, and the use of a blend of multisensory tools such as musical instruments, bells, gongs, drums, visual tools such as lamps, smoke, shadows, dresses and decorations, as well as the smell of incents and possibly perfumes and fire. The acoustic setting was crucial in integrating these multisensory experiences because sound in a reverberant enclosure is constantly present, being an immersive medium reaching to everyone. The amplification of sound inherent in the design of Hagia Sophia was greatly needed to counteract the absorption due to a vast number of people attending the mass, theoretically exceeding the enclosure’s capacity of nearly 16,000. The Eucharistic ritual of elaborate mass carried out in Hagia Sophia was clearly the event not to be missed; its theatrical, dramatic, and religious impact could not be compared to any other event in scale, magnitude and quality. It would seem appropriate for Hagia Sophia’s architecture to be designed from the start as experiential space, with the above mentioned utility factors in mind. The layout and the presence of galleries, vertical space, hard reflecting surfaces were likely understood to be prerequisites for dramatic staging of high capacity events (gatherings) with large reverberance. Good audibility of speech was essential and clearly possible with elevated sources on ambo and audience absorbing undesirable reflections. Historians Nina Ergin  and Bissera Pentcheva  point to the important notion that Hagia Sophia both as a Byzantine Cathedral and later Ottoman Mosque was consciously used to magnify the religious experience of the presence of the divine and thus boosting the faith. Acoustical support of religious ritual In the early Byzantine Church it was expected that musical performance would involve the whole congregation, not just a separate group of trained singers responsible for music. Singing together with the entire congregated people, singers commanded a powerful responsorial mass of voices, which in the highly reverberant enclosure could produce a magnificent, loud, and thrilling sonic effect, and elicit powerful religious emotion as if joining the choir of angels. Since the floor area was absorbing the sound due to the presence of people, reverberating sound decay was perceived as lifting upwards towards the dome and perhaps beyond, to heaven. There are several important acoustic outcomes, which participants in the liturgical ritual could perceive when sound was being processed by the massive enclosure of Hagia Sophia. The performance of long sustained notes would cause a gradual build up, an amplification effect from the accumulated sound as more sound energy was continually added than absorbed. It is hard to estimate the total increase in sound power and perceived loudness, suffice to say that ‘choros’ (which included all participants in musical performance) had some ability to control the perceived volume of music by using strongly sustained drones and gradual accumulation of sound energy. This was the means of sound volume control. The second important acoustic effect was overlaying and dissolving notes that if sang in unison created smoothly evolving chords transitioning gradually between dissonant and consonant harmonies. With reverberation time upwards of 10 seconds, the monophonic melodic progressions were seemingly suspended in time, and were superimposing either tonally colliding or harmonizing with the newly sang pitches. This overlapping and smearing effect was slow enough allowing singers to gently steer the new pitch appropriately to avoid conflict with previously sang notes assuring continued harmony. The melismatic singing in particular, involving singing of many notes per syllable of text, characteristic for delivering kontakion (dramatic homily) was beautifully swelled and ornamented by the reverberant returns from Hagia Sophia’s enclosure. Yet the especially dramatic effect was achieved during processions within Hagia Sophia when groups of singers moved around the church engaging the acoustics in different sections of the church. For a stationary listener, moving sound sources appeared to change in perspective and direction, illuminating the auditory presence of Hagia Sophia distinctly and differently for distributed participants. For those in the procession, the gradually changing acoustics when passing through smaller and larger spaces produced a flowing sensation of motion in a seemingly liquid, uninterrupted space. The most dramatic acoustic effect was created when explosive (percussive) sources such as drums, bells, etc. would be used within Hagia Sophia acoustics. The direct path of sound from the source to receiver could only account for a fraction of the total acoustic energy, which once accumulated was now returning in vast amounts from all interior surfaces of the enclosure within seconds. These massive reflected returns of sound, coming from different directions and having substantial time offsets due to large variation of distances within the church, would overlap in a dramatic and powerful similitude of an acoustic avalanche. The performers were able to shape this effect by controlling the crest factor (the peak to average value of the waveform radiated by the instrument), choosing between sharp edgy transient sounds or smoother more resonating types. The loud explosive transients could be used to signal the start or an end, or climax, in a liturgical part, as this type of sound could reach practically everyone in the church. ! 2! Vertical positioning of musicians and orators was possible on all four levels of the vast open space of Hagia Sophia: on the ground plane, on lower and upper galleries, and at the base of the dome. Research of musicologists Neil Moran and Sebnem Yavuz who studied the musical use of Hagia Sophia  suggests that singers were located at the base of the dome, standing side by side behind the metal railing. Such location could provide natural acoustical reinforcement and effective redirection of sound in downward directions using the acoustic mirror-effect of the dome. Sources near large wall boundary would be reinforced by the immediate reflection from that boundary. Dwyer points out the presence of metal holders for oil lamps existing in railings illuminating the space vertically and thus enabling musicians to perform a written score or text . Elevated sources were also possible above the ground level, on the ambo. Such source, placed above the horizontal level of the floor, was effectively projecting sound towards people on the floor via direct path, and free of reflections. Elevated orators or musicians could deliver recitations or chanting with enhanced visual presence and fine sonic clarity. Since most of acoustic absorption was due to people occupying the floor, the reverberant sound was perceived to decay in an upward motion towards the dome. The massive volume of Hagia Sophia was a magical container of slowly decaying sounds able to immerse large crowds of people. The acoustics of the inner space could project an immense spatial extent of sound, its seemingly endless sustaining power, and omni-presence. The more beautiful was the sound generated, the greater was the perceived power and beauty of the church. Acoustics of Hagia Sophia The huge interior dimensions, volume exceeding 250,000 m3, and high reflectivity (low acoustic absorption) of marble surfaces contribute to the prolonged duration of reverberation and its high intensity. The audible presence of reverberation is the hallmarks of Hagia Sophia. Measured with a broadband sound source (producing high sound output level between 20Hz and 40kHz), reverberation time T30 (in 1/3 octave bands) reaches almost 12 seconds at 500Hz and climbs to over 13 seconds at the lowest frequencies, with 10.35 sec at 1kHz (see Figure 1). One becomes aware of the extended low-frequency reverberation time whenever one of the massive 6th century 7 meters doors (made of oak and bronze frame) are being shut when the church is empty. The reduction of high frequency reverberation time is consistent with dominance of air absorption, dropping to 2.57 seconds at 8 kHz, and 1.59 seconds at 12.5 kHz. However, because of relative high humidity above 50% due to the proximity of the sea, the air absorption is less than it would be in a dryer environment. Figure 2 describes air absorption coefficient values at different relative humidity levels (after Rettinger and Knudsen) . Figure 1. Reverberation time (T30) averaged over 8 positions measured with broadband sources in Hagia Sophia. Figure 2. Absorption coefficient of air at different relative humidity levels and frequencies. Note much higher absorption coefficient at high frequencies than at low frequencies, but not as much absorption at higher relative humidity (from M. Rettinger ). Hagia Sophia has many spherical and cylindrical shaped sections. In concert hall acoustics, large concave surfaces are considered undesirable as they focus concentrations of energy in certain areas and prevent uniform distribution of reflected sound throughout the room, especially when radius of such curved surfaces is comparable to acoustical wavelengths. However, spherical and cylindrical surfaces can also serve as effective scattering surfaces producing a much better diffusing effect than a plane ceiling or wall reflector. ! 3! In Hagia Sophia, listener’s awareness of individual acoustic reflections, even the strong and delayed reflections focused by concave surfaces, is limited due to the presence of many other densely spaced reflections and reverberation. Therefore, detection of echoes, reflected waves appearing as distinct events standing apart from those directly received from the source, is not possible. High amplitude multiple reflections arriving along different paths can also hide ‘flutter echoes’, repetitive reflections occurring between parallel surfaces, which in isolation sound characteristically like fluttering of bird’s wings. Figures 3 and 4. Figure 3 (on left). Reflections of rays from concave ceiling in a large room (from Cremer & Müller ). Figgure 4 (on right). Reflections of rays from concave mirror (detail). (from Kutruff ). Acoustics of the dome Whether a ceiling dome concentrates the acoustic energy or scatters it distributing it over a wide angle depends on the location and distance of the sound source and the listener to the dome. In general, if both the source and the observer are located below the circle based on the radius of the dome, the undesirable focusing effect producing increased sound intensity is not going to be heard on the ground level . Therefore in Hagia Sophia with the dome height of 56.60 meters and diameter of 31.87 meters, people located on the ground level are outside of its focusing area. Room acoustician Kutruff  explains that if we consider sound propagation as rays, and apply the laws of concave mirrors from optics, the relationships of the source distance a, the distance of the focus b and the radius r of the mirror are related to each other by the following equation: 1/a + 1/b = 2/r. For distances of the source to the mirror greater than half the radius, the point of focus is located closer to the mirror than the source. The focus is at r/2 for distant sources. As Figures 3 and 4 show, sound rays projected from near the ground level upwards towards the dome will be redirected by the curvature of the dome towards a much wider area of the floor, as they are disbursed into more lateral angles. The function of the dome therefore is to scatter the rays rather than reflect strong specular bundle back towards the floor. For a distant source, focal point with the highest sound intensity is located near the dome, near half-radius distance from the apex of the dome. A portion of rays returns downwards, directly below, since a small section of the dome surface near the apex is parallel to the floor. Rays reaching other areas of the dome are redirected down sideways with a horizontal component. When the source moves away from the center of the dome, some rays are reflected back to the receiver by the perpendicular section of the curvature of the dome. This tends to maintain listener’s awareness of the central overhead placement of the dome at virtually every location. If the ceiling were a plane surface, the strongest specular reflection would follow just above the moving source. As the original dome was flatter at the top and lower to the ground by some 6 meters than the present dome, it reflected a stronger bundle of sound energy down towards the source standing directly below. Its sound scattering capacity would be weaker and less uniform than is the case now. Two large half-domes and five smaller hemi-domes contribute substantially to the development of ambient sound as they redirect and disperse the sound but at different relative time intervals due to considerable distance between them. The impression one has when listening to reflections and reverberation on site is that multiple waterfalls are being activated at different times and places, and altogether converge down onto the listener. In our exploration of Hagia Sophia acoustics, we attempted to capture this unusual effect of prolongation of the onset of the reverberation, as it is a uniquely attractive feature of the space. Whereas under the dome sound distribution and order of arrivals are more unified and uniform, some distance away from the central dome layered contributions from hemi-domes give the acoustics a different more intense dimension. ! 4! It is also impossible to omit the role of additional coupled volumes on the overall aural impression of Hagia Sophia. These volumes are coupled acoustically to the main part of the church, mainly the side aisles and the galleries. The double-width central nave opens into two smaller side-aisles through large openings between pillars, each aisle containing three serially connected volumes. Sound entering these spaces reverberates according to their acoustical characteristics contributing unique depths, directions, and decay characteristics. Similarly, lower and upper galleries also lend their own decay characteristics. In all, Hagia Sophia dazzles the listener with richly animated presence of reverberation that is distributed everywhere around and above, being different in every place. Regretfully, when visiting Hagia Sophia, one can never experience this aural architectural beauty of the church because of the constant background noise generated by crowds of visitors. How can Hagia Sophia immerse and overwhelm with beautiful acoustics today? Today, Hagia Sophia is a museum and a highly important site of Turkish, European and World cultural heritage. Millions of tourists visit the museum each year while museum staff carries out important ongoing maintenance and conservation work, as well as research enabling better protection of this magnificent structure from earthquakes and natural disasters. Hagia Sophia is not available as a venue for religious celebrations or staging of cultural events such as music concerts. Still, the acoustic experience within the interior space of Hagia Sophia could be overwhelming today if there was an option to hear the acoustics being applied to music. Such option gradually becomes possible by employing digital means of active architecture, by recreating the virtual interactive presence of Hagia Sophia in alternate spaces, for uses that are not possible within the actual space. For this to be achievable, we opted to take precise acoustical measurements of the church with an eye on reconstruction of the essential dimensions of Hagia Sophia acoustics in a laboratory, studio, or a concert hall. With this goal in mind, McGill University researchers conducted comprehensive measurements within Hagia Sophia in October of 2013 with the approval and kind permission from the Turkish Ministry of Culture and Tourism, the Directorate of Museums, and the Directorate and Management of the Hagia Sophia Museum. Measurements versus modeling Acoustical measurements are an alternative to computer modeling of architecture, both are able to produce auralization that is the ability to hear and interact with the acoustics of architectural space. Computer modeling of architectural spaces allows flexibility of changing the locations of sources and receivers in auralization, and makes it easy to derive new synthetic impulse responses when changing acoustic properties of surfaces, for example by introducing people or carpets on the floor, or curtains on walls. This technique was used to simulate the change in acoustics of Hagia Sophia from Byzantine church to Ottoman mosque, done with the Odeon software as a part of the CAHRISMA project ,. The resulting quality of computer modeling depends greatly on the accuracy of input data the software program receives, of precise dimensions, surface shapes and non-uniformities, details of surface absorption; and on the accuracy of mathematical models and numerical calculations of wave phenomena, including multiple orders of reflection, diffraction and scattering. Simulated sound sources and receivers have idealized directional characteristics, which cannot be easily duplicated in practice. Rendering of multiple sources adds to the uncertainty and quality of results, especially when live interactive applications are considered using multiple channels. On the other hand, impulse response produced by a model is free of noise and distortion as such aspects are not included in the modeling and calculations . Acoustic measurements made on site, on the other hand, produce results that depend on dynamic range of sources and receivers, and the background noise level within the enclosure. Care must be taken to ensure the widest possible dynamic and frequency range of the measurement, quality of signal conversion, and the off- and on-axis directional uniformity characteristics of loudspeakers and microphones interfacing with the acoustics. High-resolution acoustic measurements can produce the exact description of the acoustical environment as ‘illuminated’ by loudspeakers and ‘photographed’ by microphones placed in the measured space. All details of the enclosure interacting with the sound radiated by the source, the effects of propagation, reflection, absorption, scattering and diffraction of sound in the room are precisely recorded and converted into multiple high-resolution impulse responses. The goal of exploration and the methodology Preliminary visit to Hagia Sophia in June 2013 served to map out the acoustic space and identify critical zones of the architectural space that could render the most representative image of Hagia Sophia acoustics. It resulted in choosing four source locations and a multitude of receiver locations employed in the subsequent comprehensive measurement session conducted in October 2013. Strategic choices Digital recreation of a virtual Hagia Sophia has to offer the flexibility of adjustment needed to render multiple acoustical perspectives for listeners having different points of view, and for a number of sound sources. It is impossible to assume that one measurement can capture all possible acoustical options in this enormous space, just as it is unnecessary to measure at hundreds of positions according to an arbitrary geometric grid. We chose zones with distinct and different acoustic surroundings, and places where actual sources (singers, orators) were likely located during religious events. Two locations were near the altar, and two near the center under the dome. Figure 5 shows the source and receiver layout diagram indicating 4 speaker locations (SP) and 29 receiver locations (MP). Altogether, 8-channel impulse response measurements were made 180 times producing 1440 individual impulse responses (IRs) with 96kHz and 32-bit resolution. This data will serve two purposes: (1) acoustic analysis of sound transformations produced by Hagia Sophia, and (2) virtual reconstruction of Hagia Sophia acoustics in applications including live interactive performance and postproduction. ! 5! Figure 5. Drawing of measured positions (MP) and source locations (SP) used in measurements of Hagia Sophia acoustics in October 2013. On the right, the corresponding drawing from bird’s eye view, of the plan view of Hagia Sophia interior (from Pentcheva ). In our measurements, high resolution impulse responses were captured in three dimensions (width, depth, height) using clusters of 8microphone receivers installed at three heights of 2m, 3m, and 4m in each measurement point, thus providing 24 independent impulse responses at each measured location. Sinusoidal sweep 80 seconds long and sound pressure level of 93dBC peak ensured wide dynamic range of the measurements, suitable for the most demanding and critical applications. The use of long exponential sine sweep and high performance loudspeakers ensures high dynamic range of the measured impulse response, which influences the accuracy of all acoustical measures that are derived from impulse responses. This method also removes the harmonic distortion of the loudspeaker source from the measured impulse responses, since distortion components appear separately at negative arrival times before the onset of the impulse response, and are removed . Reconstruction of virtual Hagia Sophia Each cluster of measured 24 impulse responses represents one of the 29 receiver locations on the floor of Hagia Sophia as microphone positions MP1-22, and MPA-G (see measurement layout diagram in Fig. 5). Four locations of source positions are shown as SP1-SP4, each source consists of 10 transducers together emanating a complex wave front from a virtual center of this multiple-radiator. The curvature and the shape of this radiator have been adjusted to project a believable musical source; it is tested on-site by reproducing a number of monophonically recorded signals including small chorus, trumpet, speaking voice, guitar, drums. Once adjusted, the multispeaker source excites the interior acoustic space of Hagia Sophia with a slow exponential sinusoid sweep that is recorded by the cluster of microphones at MP- locations. By moving the loudspeaker and the microphone clusters about the room, different acoustic views of the room are recorded, each giving a distinct acoustical perspective of the space. Just as a photographer moves the camera and the lights around the room to obtain the most compelling photo image of the room, the acoustician decides on the most appropriate sonic illumination of Hagia Sophia capturing the best possible interpretation of its magnificent acoustics. This process is both artistic and technical because the choices are guided as much by the ear and aesthetic judgment as by the science and acoustical analysis of the architecture. Measured acoustics A sample of the diversity of acoustical material collected during the measurements of Hagia Sophia is shown in the following graphs. The graphs are plotted from impulse response data calculated from just a few locations only to illustrate the complexity of the measured sound fields. The full analysis of all collected data is an ongoing process as we try to better understand how best to extract, recompose and auralize such magnificent acoustical space. Figure 6 shows the ‘turbulent’ development of low-frequency energy (20Hz to 300Hz) in the room at the onset of acoustical response, within the first 300 ms following the initial impulse. Figure 6. This 3D graph shows low-frequency acoustic energy, from 20Hz to 300Hz, decaying in time over the initial period of 300 milliseconds from the onset of IR. Source and receiver positions are SP-3 and MP-D (source near the center of the nave, receiver near the altar), the output signal of the omnidirectional microphone, Ch1. ! 6! Figure 7. 3D Cumulative Spectral Decay view of the full spectrum of acoustic energy (20Hz to 20 kHz), decaying in time over 10 seconds from the onset. This cumulative spectral decay is calculated with 1/12 octave smoothing, and is presented from three different angles for source position SP-3 (near the center of the nave) and receiver position MP-18 (near the back of the nave), and using omnidirectional microphone signal Ch1. Figure 8. 3D Cumulative Spectral Decay view of the full spectrum of acoustic energy (20Hz to 20 kHz), decaying in time over 10 seconds from the onset. This cumulative spectral decay is calculated with 1/12 octave smoothing, and is presented from three different angles for source position SP-3 and receiver position MP-D (source near the center of the nave, receiver near the altar) and using omnidirectional microphone signal Ch1. Figure 9. 3D Cumulative Spectral Decay view calculated from the impulse response measured by three microphones: omnidirectional ch1 (left), bidirectional in elevation ch5 (center), bidirectional in azimuth ch8 (right). This cumulative spectral decay is calculated with 1/12 octave smoothing, and is presented for source position SP-4 and receiver position MP-F (source beyond the center of the nave, receiver near the side of the nave). 3 different microphones out of 8 microphones in a group (cluster) in one measurement location. Figure 10. Log Squared impulse response graph from SP-2 and MP-21 (source near Sultan’s Lodge, receiver near rear corner of the nave; two diagonally opposed locations), from omnidirectional microphone signal ch1. Figure 11. Log Squared impulse response graph from SP-4 and MP-1 (source beyond the center of the nave, receiver in the center of nave, near the altar), from omnidirectional microphone signal ch1. ! 7! ! Figure 12. Log squared impulse response (visible duration 1500ms) compared to direct impulse response (visible duration 500ms) measured at source location SP-2 (near Sultan’s Lodge) and receiver position MP-4 (near the center of the nave). The graphs show the impulse response in greater detail during the early portion of development of the sound field. ! ! Acoustic rendering of virtual Hagia Sophia ! ! The resulting catalog of high-resolution impulse responses serves as construction material for recreating a compelling version of Hagia Sophia acoustics in a laboratory or performance space. The unforgettable experience of hearing sound in the actual Hagia Sophia demands a complex approach to rendering a similar experience in a virtual environment. In the real space, sound evolves gradually from onset to complete decay following complex trajectories, which attract the attention of listeners by way of ‘liquid’ animations, transitions through phases and soundscapes of varying characteristics. Hagia Sophia strongly projects its aural architecture demanding to be heard. Her flowing display of sound requires of a virtual reconstruction engineer to build a constantly evolving motion of reverberation using different acoustic views captured within the clusters of impulse responses. The engineer must render sonic animations existing in Hagia Sophia as these are essential to forming sensory and psychological experience of being inside the actual space. In our experience, successful reconstruction of aural architecture is achieved when digitally rendered stimuli produce an outcome equivalent to human multisensory experience that is accumulated over several encounters with the space. The reconstruction has to be more than a simulation because it needs to unlock our memory and imagination of the space. This is why arbitrary approach of rendering only a single perspective view of Hagia Sophia acoustics cannot match the memory of our experience that is based on the exploration of this space. We are moving around the space, making and listening to sounds, turning about left-right and up-down, transitioning between different sections of the enclosure. To create a compelling reconstruction of the acoustics as we remember and imagine it (this is now our reference), the engineer must incorporate active exploration of the space into the structure of rendered acoustical response, by skillful composition of spatial layers appropriately ordered in time. And critically important is to construct and render a convincing acoustical perspective for each sound source, minding especially the perceived differences in how the space reacts acoustically to each of the source, and to their spatial distribution within the enclosure. Human attention is dynamically non-linear, it exaggerates changes and diminishes sensitivity to static parameters to reduce their masking power. The engineer must use changes. Measuring microphones employed to capture the massive dimensions of Hagia Sophia acoustics do not have the ability to bring out the complexity of spatial transformations occurring in expansive Hagia Sophia. While the ear can exercise spreading or focusing of attention on aspects of sound, especially varying the attention shift between foreground and background sounds, the microphone only transfers a fixed spatial relationship. It does not have sufficient directional selectivity to reach for specific noticeable elements of the reflected sound and reverberation. Loudspeaker and headphone reproduction further diminishes the resolving ability of human ear and brain to focus on details contained within the sound, when the sound is unfavorably displayed. The dynamics of digitally rendered space, therefore, have to be augmented through spatial design and mixing by the virtual acoustics engineer to help the ear do its job. Music performance in virtual acoustics Digital rendering of aural architecture to serve as interactive acoustic environment in musical performance requires fast signal processing able to render an immediate acoustic response. Fast parallel convolution of multiple impulse responses is a hugely intensive computational task, and acoustic delays or glitches are not desirable during performance. Musicians are affected by the quality of room acoustics as they depend on it to hear own and other players’ instruments or voices, and when shaping the musical balance and acoustical interactions. Research shows that good acoustics supports musical interactions, whereas poor acoustics plays detrimental role in musical performance and collaboration ,. In recreating the acoustics of virtual architecture, the virtual acoustics engineer must have at his/her disposal a number of assets that promote musicians’ aural communication. During 2005-2009, the author was involved in The Virtual Haydn, a Blu-ray recording project produced at McGill University and released commercially by Naxos International . In a laboratory, Tom Beghin performed all Haydn Keyboard Sonatas on seven historical instruments while being immersed in the rendered acoustics of nine virtual rooms related to the music of Haydn. The principal goal was to create a historical and functional link between the musical piece, the instrument, and the room, which belonged together in the works of Haydn . The team experienced and measured all nine rooms on-site, then rebuilt the acoustic response in the laboratory for the solo keyboard performance, and again later for the final mixing. It was critical to create, both for the performance and for mixing, the correct acoustic perspective enveloping the performer and the listener. No single capture of the original measured response was sufficient to recreate a believable immersive spatial outcome. Again, skillful balancing of room assets was critical to achieving an appropriate sensation of the acoustical surroundings in each case. ! 8! The January 2013 performance of Cappella Romana in Bing Concert Hall at Stanford University involved innovative recreation the acoustics of Hagia Sophia in the concert hall using forty statistically independent synthetic impulse responses generated from a model derived from a recording of one balloon pop in this church . Many technical challenges had to be overcome to offer a compelling experience for the singers and the audience . It is clear that for musicians the experience of playing or singing in a venue that is exceptionally suitable acoustically and/or historically, yet is not available for concerts, is hugely attractive. Many historical buildings are not open to musical uses but may exist virtually as aural architecture. Natural architecture of outdoor and enclosed spaces can also be explored virtually through active acoustics. Some buildings slated for demolition can be “saved” for posterity in aural architecture. Digital adjustability of aural architecture gives it plasticity, a potential ability to serve better than the actual architecture. For example, the total acoustic gain from the room (the level of reflected sound) can be increased or decreased to accommodate the music and the ensemble. There is no ‘room noise’ in digital rendering, unlike in most actual spaces many of which are located in noisy areas. The acoustical correction of rendered space can be implemented digitally without construction cost and time penalty. For example, ceiling reflections can be attenuated, moved down to the walls and floor, or delayed (moved further up). Acoustics can be adjusted to make music playing easier. Empowered by the increased gain from the room, musicians often play with greater ease, as they do not have to force the sound out of their instruments to reach rear parts of the audience. Further studies of aural architecture can lead to common applications that may enrich our lives and augment our understanding of history and culture. Experiencing architecture through aural experiences using methods of digital reconstruction will bring history closer to our present sensory awareness of the world. Conclusions Testing phase is underway at McGill University involving practical rendering of Hagia Sophia acoustics in a 3D (width, depth, height) laboratory containing 22.2-channel loudspeaker system. The 22.2-ch loudspeaker configuration, developed by NHK STRL  in Japan, has been chosen as a reference system for evaluating multichannel audio technologies. Hagia Sophia measurements were used to auralize the Cappella Romana performance of Prokeimenon recorded earlier at Stanford University as dry-only tracks, while the performers monitored their voices using 10 seconds long reverberation from convolution of synthetic impulse responses prepared for the January 2013 concert in Bing Hall. McGill Professor Richard King with PhD student Jonathan Hong mixed the dry tracks placing voices within the acoustic space of Hagia Sophia using a multichannel tool developed at McGill University. Three different acoustic perspectives of Hagia Sophia were used. Dynamic mixing and skillful placement of sources and reverberation material rendered a compelling realism of the performance. Female voices singing an octave above were placed in elevatation as if they were boys projecting from the balcony above the men. Lead and drone voices were spread around the floor at different distances to engage the vast reproduction space with overlaid perspectives of Hagia Sophia acoustics. Two low-frequency channels helped the drones to command power and fullness. Digital technology allows us the unique advantage of studying and exploring Hagia Sophia acoustics off-site, in a laboratory and in a performance space, using both signal analysis and rendering of acoustics of a virtual Hagia Sophia. The high-resolution data provide a thrilling option of practical explorations in different liturgical and historical uses of the church, experienced using our own present sensibility. Hagia Sophia continues to dazzle when rendered from high-resolution measurements using 22.2-channel loudspeaker system providing immersive surround with height, or in other advanced multichannel reproduction systems able to render aural architecture in 3D, as long as the rendering includes elevation. Our further work will involve acoustic rendering of virtual Hagia Sophia for live instrumental and vocal performance advancing our ability to virtually rebuild this splendid aural architecture. The scientific and artistic perspectives meet again as they did at the inception of Hagia Sophia. Acknowledgments The author would like to acknowledge the help of many institutions and individuals, among them the Turkish Ministry of Culture and Tourism, General Directorate for Cultural Heritage and Museums, Department of Museums, Directorate and Management of Ayasofya Museum, Prof. Nina Ergin and students in the Department of Archaeology and History of Art at Koç University, Prof. Jonathan Abel and Prof. Bissera Pentcheva of Stanford University, Dr. Alexander Lingas and Cappella Romana, Professor Ipek Tureli, Prof. Radoslav Zuk, Prof. Richard King, Doyuen Ko, Jonathan Hong, Scott Levine, Brett Leonard, Dave Benson, and students from Graduate Program in Sound Recording at McGill University. The assistance of Social Sciences and Humanities Research Council of Canada, Natural Sciences and Engineering Research Council of Canada, and of McGill University is gratefully acknowledged. References  Prokopios De aedif. I, i, 23 ff, Description of The Hagia Sophia in Constantinople by Prokopios, written 544. Annotated translation. Copyright: Cyril A. Mango, The Art of the Byzantine Empire, 312–1453: Sources and Documents, Englewood Cliffs, 1972.  Ruth Dyer, “Boethius, the Quadrivium and the Hagia Sophia", Talk Presented at the 48th International Congress on Medieval Studies in 2013 in Kalamazoo, MI, USA. https://www.youtube.com/watch?v=q3rOdkfQgAc  Helge Svenshon, “Heron of Alexandria and the Dome of Hagia Sophia in Istanbul”, Proceedings of the Third International Congress on Construction History, Cottbus, May 2009. ! 9!  R. Mark, A.S. Cakmak, K. Hill, R. Davidson, “Structural analysis of Hagia Sophia: a historical perspective”, Transactions on the Built Environment vol 4, © 1993 WIT Press, www.witpress.com, ISSN 1743-3509.  U. Almac, K. Schweizerhof, G. Blankenhorn, C. Duppel, F. Wenzel, “Structural behaviour of Hagia Sophia under dynamic loads”, VEESD 2013, Vienna Congress on Recent Advances in Earthquake Engineering and Structural Dynamics 2013 (VEESD 2013) C. Adam, R. Heuer, W. Lenhardt & C. Schranz (eds) 28-30 August 2013, Vienna, Austria Paper No. 475.  Zerhan KARABIBER, “The conservation of acoustical heritage”, Workshop 4, CAHRISMA – ‘The Conservation of Acoustical Heritage by the Revival and Identification of Sinan’s Mosques’ Acoustics’, Fifth Framework INCO-MED Programme of the European Commission.  P. Fausti, R. Pompoli, N. Prodi, COMPARING THE ACOUSTICS OF MOSQUES AND BYZANTINE CHURCHES, CAHRISMA Project (EU Contract ICA3-CT-1999-00007, Conservation of the Acoustical Heritage and Revival of Sinan`s Mosques Acoustics - CAHRISMA, 2000 – 2003)  Nina Ergin, THE SOUNDSCAPE OF SIXTEENTH-CENTURY ISTANBUL MOSQUES, Journal of the Society of Architectural Historians, Vol. 67, No. 2 (June 2008), pp.204-221, Published by University of California Press and Society of Architectural Historians.  Bissera V. Pentcheva, "Hagia Sophia and Multisensory Aesthetics”, GESTA 50/2, The International Center of Medieval Art, 2011, pp. 93-111.  Ruth Dyer, “Boethius, the Quadrivium and the Hagia Sophia", Talk Presented at the 48th International Congress on Medieval  Studies in 2013 in Kalamazoo, MI, USA. https://www.youtube.com/watch?v=q3rOdkfQgAc  Michael Rettinger, “Note on Reverberation Chambers", JOURNAL OF THE AUDIO ENGINEERING SOCIETY, APRIL 1957, VOLUME 5, NUMBER 2  Kutruff, Heinrich; Room Acoustics (Oxon, UK: Spoon Press 2009; 5th ed.; 1st ed. 1973), Figure 4.13, pp. 117-120.  Lothar Cremer, Helmut A. Müller, Principles and Application of Room Acoustics. Vol.1, Applied Science Publishers, Ltd, 1982, p.59.  Christoffer A. Weitze, Claus Lynge Christensen, Jens Holger Rindel and Anders Christian Gade. Computer Simulations of the Acoustics of Mosques and Byzantine Churches. 17th ICA, Rome, Italy, September 2 – 7, 2001.  Weitze, Christoffer Andreas; Rindel, Jens Holger; Christensen, Claus Lynge; Gade, Anders Christian, The Acoustical History of Hagia Sophia revived through Computer Simulation, Forum Acousticum 2002.  Jens Holger Rindel, Claus Lynge Christensen and George Koutsouris, SIMULATIONS, MEASUREMENTS AND AURALISATIONS IN ARCHITECTURAL ACOUSTICS”, in ACOUSTICS 2013 NEW DELHI, New Delhi, India, November 10-15, 2013  S. Müller and P. Massarani, 2001. Transfer-Function Measurement with Sweeps, J. Audio Eng. Soc., 49, 443-471.  Ko, D., Woszczyk, W., and Chon, SH., (2012). “Evaluation of a New Active Acoustics System in Performances of Five String Quartets”. Audio Engineering Society Convention Paper, Proceedings of the 132nd Convention, 2012 April 26–29 Budapest, Hungary.  Doyuen Ko, Wieslaw Woszczyk, Jonathan Hong, and Scott Levine, "Augmented stage support in ensemble performance using virtual acoustics technology", Proceedings of Meetings on Acoustics, ICA2013 International Congress on Acoustics, 2-7 June 2013, Montreal, Canada.  Beghin, Tom (with Martha de Francisco, prod., and Wieslaw Woszczyk, VA-architect); The Virtual Haydn: Complete Works for Solo Keyboard. 4 Blu-ray discs (3 audio, 1 video). Naxos (NBD 0001-04), 2009. www.music.mcgill.ca/thevirtualhaydn/  John Irving, “Digital approaches to Haydn’s solo keyboard music”, Early Music, Oxford University Press, 2012, pp. 1-4.  Jonathan S. Abel , Wieslaw Woszczyk , Doyuen Ko , Scott Levine, Jonathan Hong, Travis Skare, Michael J. Wilson, Sean Coffin, Fernando Lopez-Lezcano, “Recreation of the acoustics of Hagia Sophia in Stanford’s Bing Concert Hall for the concert performance and recording of Cappella Romana”, Proceedings of ISRA 2013 International Symposium on Room Acoustics, Toronto, Canada, 2013 June 9-11.  Jonathan S. Abel, Nicholas J. Bryan, Patty P. Huang, Miriam Kolar, Bissera V. Pentcheva, "Estimating Room Impulse Responses from Recorded Balloon Pops," Convention Paper 8171, presented at the 129th Convention of the Audio Engineering Society, San Francisco, November 2010.  Hamasaki, K., Nishigushi, T., Okumura, R., Nakayama, Y. and A. Ando, “22.2 Multichannel Sound System for Ultra HighDefinition TV", SMPTE Technical Conference Publication, October 2007. ! 10!
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project