Productions

Home Publications Teaching Activities Productions Resources Bio/CV


Real-time Aesthetics and Experience Lab

We formed a new lab to research aspects of aethetics and visual design in building interactive experiences.
click here to go to the lab's website

Lighting System (ELE)

As part of my dissertation, I have developed an automatic lighting system called ELE (Expressive Lighting Engine), which uses expressive lighting techniques derived from the film and theatre lighting design theory to make lighting decisions in real time to satisfy several design goals, including establishing visibility, portraying dramatic tension, and directing viewer's attention to important artifacts while maintaining visual continuity between frames. ELE uses non-linear optimization algorithms to place lights and adjust their angles and colors given the stage configuration and placements of characters on the stage, the importance of each character, and the dramatic tension of the moment.

ELE is divided into three components:

  • a system for choosing angles, depending on a set of artistic constraints specified by the artist or inferenced by ELE depanding on the story event and the action. I have compiled a demo that shows the variation of some artistic constraints and their effect on the lighting angle.

    Click here to see a demo.

  • a system for choosing colors, depending on a set of constraints artistic constraints specified by the artist or inferenced by ELE depending on the story event and the action. I have compiled a demo that shows the variation of some artistic constraints and their effect on the lighting colors, visual tension, and mood portrayed

    Click here to see a demo.

  • a system for choosing a lighting configuration for the scene, and managing lights by monitoring the areas as they go in view or out view.

To show ELE in action in a complete scene where time is a factor, please click here. This demo shows one scene from Mirage played with ELE configuring the lights and selecting angles and colors to suit the interaction.

MIRAGE

I developed an interactive story called Mirage, funded by CIRA (Center for Interdisplinary Research in Arts). Mirage, based on the ancient Greek tragedy Electra, is an interactive graphical (visual and audio) experience where a participant plays the role of a character. The participant is presented with a number of choices involving relationships, justice, redemption, family ties, etc. The character of the participant is modeled in four-dimensions: violent, self-interested, coward, and reluctant hero. During interaction with Mirage, the participant will be asked to make choices that affect the character he/she projects in the narrative. This projected character will directly affect the story. I recruited and led the team of artists who developed Mirage. The team included a sketch artist, a screenwriter, two animators, four voice actors, a sound designer, a music composer, and a visual effects artist (who won an Academy Award for his animation work in Starship Troopers ).

For more information about Mirage Click here

Authoring System

To make the job of implementing Mirage easier, in addition to the lighting system (ELE), I developed an interactive narrative architecture that is composed of a number of systems, including a camera system, a characters controller system, and a director agent. The camera and characters controller systems move the characters and the camera depending on a specific script dictated by the artist. The director agent ensures right timing of behavior execution, facilitates interaction, and synchronizes behaviors among camera, lights, and character controllers.

This system was used by students in the interactive narrative class to build their interactive stories.

FLAME

My master's research focused on developing believable characters. Previous research showed that simulating the emotional process is necessary for building believable characters (OZ project, see Resources' page). My research aimed to investigate the role of the learning process on believability. I designed a model called FLAME (Fuzzy Logic Adaptive Model of Emotions) (El-Nasr et al. 2000, see publications' page ). Emotional states have no definite boundaries. Therefore, one possible method for representing these states is to use fuzzy linguistic variables and fuzzy sets. The model used Fuzzy Logic inference rules and OCC (Ortony's work on Emotions, see Resources' page) to derive an emotional state based on the character's goals and expectations. I used reinforcement learning, conditional learning, among other learning algorithms, to derive expectations given a situation. Emotional states were translated to behaviors based on a number of rules derived from psychology research.

Using FLAME I was able investigate the impact of emotions on character believability. Additionally, I investigated the impact of learning on believability. The model was tested through a simulation called PETEE (a PET with Evolving Emotional Intelligence), where users rated the believability of the character, among other character qualities. The results collected from these experiments demonstrated that learning has a major impact on the character's perceived believability.
This work won 'best student paper' award at the Autonomous Agents Conference '99.


PETEEI (A PET with Evolving Emotional Intelligence)

PETEEI is a simulation of a pet using the FLAME model. The pet learns from its interaction with its user. It eventually aquires emotions such as shame and pride depending on its interaction with the user. Additionally, PETEEI was designed to recognize and cope with the various moods and emotional reponses of its owner.

An Emotionally Expressive Baby

In this project, we have integrated the emotional agent model simulated in PETEEI with visualization models for mapping emotions into facial expressions to create an interactive simulation of an agent (a baby agent) that produces appropriate facial expressions in a dynamic environment.