This project will focus on the creation of PC-based, multiparticipant, networked virtual worlds. These worlds will consist of three dimensional computer generated models and include animated characters. The environments and their inhabitants will be rendered in real-time as texture-mapped polygons. The users (which we call participants) will be represented in these worlds as 3D models and referred to as avatars. Participants will communicate in two ways, through conversation, with voice input rather than typed text messages, and through symbolic gestures and movements of their avatars.

We will explore both abstract and realistic representations of virtual space with a focus on the look and feel of the environment and the integration of multiple virtual participants. This will take into consideration the often ignored function of pacing and rhythm as it applies to the interactive experience.

Our first and primary focus will be in the design of movement as it relates to the animation of avatars and other forms of artificial life; for they will bring these virtual worlds to life and will prove most challenging to integrate in a networked environment. We will initially design a three dimensional model of a social environment which includes an animated performance.

This research will focus on the numerous aesthetic issues involved in the creation of effective, coherent, well-designed virtual environments; environments that are populated by intriguing representations of the networked community. This concern for aesthetics has been sorely lacking in existing research.

The research goals are as follows:

• to research existing user interface design and develop new design paradigms that enhance the sense of immersion in a virtual world,

• to develop and evaluate appropriate design criteria for effective navigation through multiple forms of data representations,

• to design graphic representations of avatars and other articulated characters based on their function and environment.

• to develop techniques for character animation and interaction including computer generated procedural behaviors,

• to explore the role of movement and gestures as a form of communication in a virtual space.

This research will be presented in prototype form as an interactive, computer generated three-dimensional environment with animated characters. Our goal is to generate examples that will assist in establishing aesthetic criteria and will lead to a design "template". This tempate will provide the underlining rules to be applied in the development of active virtual environments.

MUDs and chat rooms are popular forms of communication on the net. Until very recently these have consisted of text only, a participant uses a keyboard to type conversations, stories and descriptions of places and characters. Influenced by video games such as Doom and the expanding capabilities of 3D graphics and faster computers, there has recently been developments, from both research and commercial concerns, in the design of MUDs and chat rooms as three dimensional environments with 3D avatars.

When a virtual environment changes from a text description to a visual representation of avatars and their surroundings, many new questions are introduced. It is much more difficult to build and animate intriguing computer generated worlds than to create text descriptions. How should these worlds be designed? How should avatars be visually represented and how should they behave? What are the techniques for effective navigation?

This form of communication is truly global and therefore must communicate to all cultures. Based on research we will seek to identify and develop gestures and other methods of communication that have meaning to a global community. The study of non-verbal communication and the significance of rituals, dance, puppetry, masks and performance across many cultures will provide a basis when designing movements and forms that will enhance the communication experience in a networked environment.

This project will draw on my twenty years of experience in research-oriented interactive multimedia and three dimensional computer animation work. This includes my recent experience in video game development which utilized the most current real-time 3D technology. It will also serve as a continuation of research I began in 1979, when I pioneered techniques for the animation of computer generated characters based on the study of human motion and non-verbal communication. I can now extend this work to the realm of interactive virtual communities.

The following report is meant to briefly summarize current work and discuss future plans for the research project titled, "Aesthetic Issues in the Design of Multiparticipant Virtual Environments".

This project commenced in late January 1997 after receiving the required computer equipment from Intel. This report will refer to our progress over the last seven months as well as the anticipated progress in the remaining five months of Year 1. As a summary, it will not include detailed technical specifications; that can be provided in a separate document.

Let me begin by saying that, at this point, we have achieved much more than expected thanks to our exceptionally talented and dedicated team, comprised of students from both the computer science and design departments. In addition, we have received crucial support from the staff of the Center for the Digital Arts (CDA).

In order to create virtual environments, we first needed to build a fast and flexible 3D graphics engine, capable of rendering texture-mapped, animated, polygonal models and supporting synchronized audio data.

After the initial software design phase, which involved research of real-time rendering systems and familiarity with PC-based tools, we began development of a customized 3D engine using the Direct3D API. This proved to be unsatisfactory so we converted the system to Open GL.

One of the major requirements of this system, which we call "Emergence", is to handle hierarchical motion data from articulated 3D models, including motion-captured animation. This is important for the impression of life and interaction within the world.

We selected the NT version of Softimage as our 3D modeling and animation software and built a production pathway that converts 3D models, including texture, material and hierarchical animation data from the Softimage format to one read by Emergence.

Emergence supports multiple camera viewpoints. Each inhabitant, including the avatar, can be viewed from a first or third person perspective. We have found it most effective to attach a camera to the avatar with a certain distance and lag. There are also free roaming cameras, allowing exploration of the world independent of a participant's avatar.

The computer-controlled inhabitants of the virtual world are designed with interactivity in mind. Each of these inhabitants or agents has vision and sound sensors, and a range of individual characteristics, controlling features such as dynamics. Agents also have a set of parameters that govern their behavior towards other inhabitants of the world. An agent can have a "feeling", (a parameterized value), such as love, like, dislike, hate or indifference which affects the agent's reaction to an inhabitant when in its vicinity. A complex social environment can be achieved from the interaction of simple agent behaviors.

Any agent can also be a member of a species which allows for emergent group behaviors. Examples of emergent behaviors present in the world are group flocking, following and fleeing.

The next area of development in agent behaviors is to allow the agents' "feelings" to adapt over time. For example, a participant's avatar could harass an agent that likes it (by getting in its way and following it closely), causing the agent to gradually dislike the avatar, or a character could come to life only after an avatar interacts with it.

In examining our Year 1 Research Plan, it appears that we will accomplish our goals. The primary change to that plan involves a more extensive focus on artificial life. Procedural behaviors applied to articulated 3D models proved to be a powerful method for invoking the sense of an alive, complex, social and interactive environment.

We expanded this area to include the development of a flexible, behavior scripting language that allows non-programmers to design behaviors and relationships between characters and objects. This scripting language will continue to be developed and enhanced over the next year.

We are currently editing and processing motion capture data which can be overlaid on certain behaviors. This will allow us to explore forms of Performance Art in a virtual environment.

An interactive demonstration of our work was presented at Intel's Complexity Forum on May 14, 1997. The Emergence.system was first presented publicly, at the 1st International CAiiA Research Conference titled, "Consciousness Reframed: Art and Consciousness in the Post-Biological Era", which took place, July 5-7, 1997, at the University of Wales, Newport, Wales. I presented a paper titled "The Bush Soul: Traveling Consciousness in an Unreal World" which discussed theories on the location of consciousness and its relevance to avatars and virtual environments. "The Bush Soul" was presented as an interactive installation at the international conference and exhibition, "Art Futura", which took place in Madrid, Spain from October 15-19, 1997.