Designing Telematic Sculpture for Mediated Conversation

Judith Donath and the Sociable Media Group, MIT Media Lab
Karrie Karahalios and the Sociable Spaces Group, UIUC dept of computer science

This IAP course runs from Jan 16-19 (Monday - Thursday). We will meet in E15-383 from 10:30-12:30.

Please see the Chit Chat Club proposal for ISEA for useful background information


Day Topic
Monday Introduction: What is a communicative telematic sculpture? What is ISEA? What are the big research and artistic questions we are addressing? In this session we'll describe the first Chit Chat Club installation and discuss what we would like to accomplish with this new version of it. We'll show some of the works that have influenced our thinking and discuss the cognitvie basis for our approach. We'll discuss the tradeoff between abstraction and realism in a personal representation (avatar) and how scale affects this.

Reading assignment: Chit Chat Club website

Fully Embodied Conversational Avatars: Making Communicative Behaviors Autonomous, Cassell, J. and Vilhjálmsson, H. (1999). Autonomous Agents and Multi-Agent Systems 2(1): 45-64.

Chat Circles. Fernanda Viegas and Judith Donath. In Proceedings of CHI 99.

Design problem: create an initial design for two avatar chairs. Specify their looks and what changes their appearance.

Things to think about:

- Is the design in a frame - is it 2D or 3D?

- What is the scale of the piece?

- How does this interact with with the rest of the space? With other people?

- How do the local users know when it is inhabited - how does it indicate that it is neutral, but present? What is the ipresion that the neutral face conveys

- Does it use a face? Text? Are there ways of indicating direction or attention?

- How does it look when no one is inhabiting ?

Tuesday Designing remote present: We'll look in greater depth at the difference between a physical and on-screen avatar. We'll review design sketches, looking at how they address the issues of scale, controllability, and agency. We will also discuss feasibility: which designs can be built and which require greater machine intelligence etc. than is possible. We will evaluate the designs both from a communication design standpoint (how does this help the participants communicate better) and from an artistic perspective (what does this say about communication? humanity? cafes?)

Reading assignment: read chapter 5 of Social Catalysts . Eric Paulos and John Canny PRoP: Personal Roving Presence

Design problem: redesign/refine 2 of the designs we discussed in class to dayavatar chair. Include in your design the interface for the remote user as well. At least one of your designs must use TEXT as the primary form of input.

Things to think about

- how does the remote particpant know if people are at the table? Follow the conversation? know how he or she appears to others? What control does this person have on the avatars appearance?

- how are entrances and exits managed?

- Keep it practical. These installations will need to be robust - used in a public setting by many people, in a real cafe with food and drink. We are avoiding robotics for a number of reasons, primarily because the likelihood of their breaking down is too high. Think of ways to use lights or changing the size of a video image instead to achive similar effects.

- Think about how to incorporate text. Some of the remote users may have only text / mouse input. Text display can be quite poetic - think about its shape, motion, etc.

- describe the scale of your piece. Why have you chosen that size?

- How does the piece work? Highlight any places where the piece is making complex analyses about word meanings or physical relatonships- these may need to be reformulated.

-Think about the users' motivation. If we expect them to behave a certain way or to provide certain imput, what inspires them to do so?

Wednesday The remote interface: What the remote user see? How do they interact with those present? How do they know how they appear? How do they know which chair to inhabit ? We'll review the designs, looking both at the remote and local participants' experience. We'll come up with 3 conceptual models to bring to final design stage.

Design problem:  Create a final design and spec for one of the 3 models.

1. The text avatar: This avatar has a text only interface. The remote user hears audio and sees video, but inputs text. The text will be very visual. What is the shape of the sculpture? How does the text move? Is there any special input for effect?

2. The abstract audio avatar: This sculpture has images and audio for input. There is a simple graphical mode that is in efect when the sculpture is occupied. There is also a shape that pulsates with the volume of the audio input. In addition, the user can draw or add pictures to it. What sort of shape should this sculpture be? How does the remote user become motivated to decorate this design?

3. The facial avatar. This one has audio input and has a face. What is the shape of this sculpture? is it freestanding or hanging? What is the material? How is the face controlled? Is it a control of emotion? indication of attention? Is the face abstract - does it behave like a normal face or do say the eyes appear sometime and the lips at others? how does it related to the table?

For all of these:

  • how do they look when unoccupied? When occuped but idle? When in active use? What are the entrances and exits like?
  • Would it make sense to incorporate ideas such as growing and shrinking scale of projection?
  • What about other lights, etc.
  • How do they relate to the other people in the space? Are they moveable?
  • Where does the equipment go? Where is the projector? How many camersa/ microphones does it need?
  • Will text or sounds from past conversations be incorporated in the piece?
  • What can we learn from this piece? What do we want to observe about how people use it (or all 3)? Are there experiments we would want to run?
Thursday Technology and experiements: How can we implement the pieces? What equipment is needed? What are the steps to building it? What do want to learn from the installations?