UCLA’s Visualization Portal presented one of the first truly “mixed” virtual reality performances, where live performers – the singing group UCLA Sounds - were placed acoustically in a virtual model of the medieval cathedral Santiago de Compostela.  The model of Santiago de Compostela was created for Professor John Dagenais, Spanish and Portuguese, as a teaching aid for a course on medieval literature inspired by the pilgrim route. 

The concert was not simply a matter of adding reverb to the voices of the singers with the model serving as a static backdrop for the singers, but rather an attempt to create a living, breathing virtual soundscape through dynamic, immersive sound spatialization.

The performance was a showcase for David Beaudry's sound server, an audio application for real-time audio processing for virtual reality, performance and computer gaming which has been in development for three years at Academic Technology Services. The approach David has taken with his sound server differs significantly from current and past approaches to sound for virtual reality in that the concern is not so much with replicating the physical acoustics of the space (which is a secondary concern), but rather a more dynamic approach to actual content - what makes up the soundscape of a particular model and how to make this soundscape as engaging, dynamic, and performative as possible. In other words, the specifics of the acoustics are sacrificed in exchange for a more robust engaging experience.

For the concert, David, who also functioned as the lead sound designer for this project, and undergraduate Theater student Ben Story, created four scenes in which the singers performed. The concert started with the singers in a room outside the Portal.  Virtually the men were place on the altar and the women in the choir. As the audience entered the cathedral from the east entrance, the audience was able to experience what it would have sounded like entering and moving through the cathedral.

Immediately the cathedral came to life. The virtual placement and spatialization of the singers (all done with real-time audio processing with only 40 milliseconds of latency between sound coming into the microphone and leaving the speakers) was especially obvious since the musical material featured many 'call-and-response' pieces between the two choirs.

The second scene brought the audience outside to explore the town surrounding the cathedral. The singers were physically located behind the audience, much like a group of minstrels following a crowd as they strolled through the town. This scene featured secular vocal and instrumental works.

The third scene brought the audience back into the cathedral. The singers were then physically placed in front of the audience, although virtually they were in the balcony overlooking the altar and choir. For the last scene the singers left the physical space of the Portal. Virtually the women were placed near the altar finishing with a couple of high pitched liturgical pieces. The men were positioned outside the cathedral singing a secular pilgrim marching song. As visitors left the cathedral they could not only hear the spatialization of the two groups, but also the difference in acoustics between the women (who were in church) and the men (who were outside). This effect provided a dramatic recessional and conclusion to a wonderful, and one-of-a-kind concert.

After the concert, the same sound server was used to run a new sound model of the cathedral. Here David and Ben tried to replicate what it would have sounded like in the cathedral, with crowds of pilgrims milling about the church, singers (both liturgical and secular), as well as sounds of people and markets in the surrounding town.