An Augmented Reality System of Linked Audio

Joey Rozier and Karrie Karahalios
Professor Judith Donath


Hear&There allows people to virtually drop sounds at any location in the real world. Once one of these "SoundSpots" has been created, an individual using the Hear&There system will be able to hear it. We envision these sounds being recordings of personal thoughts or anecdotes, and music or other sounds that are associated with a given area.

We hope that this system will be used to build a sense of community in a location and to make places feel more alive. Over time, an area such as the Media Lab Courtyard can be filled with sounds from many members of the community so that new members can get a sense of who others in the community are. Then, the new member can drop his or her own sound into the space, adding to the collective definition.

To make the augmented environment as realistic as possible, we use spatialized (or 3D) audio, using Java 3D. This provides important cues to the explorer roaming the augmented environment, as it allows sounds to "appear" to be coming from a particular location in space.

In addition to being able to drop sounds in a space, Hear&There includes a graphical user interface to allow precise control over where a sound exists in space, how large it is, and various properites of the audio.

An additional interesting application of this project is the notion of activation networks. Although the user of the system can choose to explore all of the SoundSpots, they may also choose to take a more guided route. Using this approach, most of the SoundSpots in an area are "turned off." Whenever a user moves into a SoundSpot that is turned on, he or she is presented with the option of turning on other SoundSpots that the SoundSpot's author suggests.

This is the first stage of a project that will branch into new areas in the future. Some questions we may address in the future are the notion of temporal information (so that a SoundSpot changes over time), augmented communication channels within a space, and moving sounds.

More Information

(c) 1999 Sociable Media Group at the MIT Media Lab