Researcher List

Department of Media Science

Sensory Media

Sensory Media

Sensory Media

People have a variety of sense organs: eyes, ears, nose, tongue, and skin. Senses also play an important role in conveying a huge amount of computer based information. Our group has been studying human-computer interaction technology, to represent information by using various sensory channels. Specifically, we are developing several interfaces, such as immersive display systems for enhancing spatial presence, haptic display interfaces for the sense of touch, and locomotion interfaces for simulation of human mobility.

Future goal

To develop applications that maximize the power of virtual reality environments created by computers, we are focusing on surgical simulators, rehabilitation equipment to support the elderly when they exercise, and an emergency evacuation simulator.

Human Interaction Design

Guide Dog Robot/Communication Robot
Guide Dog Robot/Communication Robot
Cooking Assistant Robot
Cooking Assistant Robot

By defining our research as pertaining to people and their surrounding environments, we are working on roughly the following three interaction design research themes.

  1. Navigation design research: Reliable guidance systems, such as guide dog robots
  2. Human communication design: Systems capable of understanding human intentions/emotions and forming natural expressions, such as communication robots
  3. Cognitive function design research: Systems that support the lives of the elderly and physically challenged, such as cooking assistant robots

Culturally-aware Human-agent Interaction

Sensory Media

Human-Agent Interaction research (HAI) focuses on the interaction and communication between people and virtual agents. Virtual agent-based interaction has matured over the last decade, and agents have become increasingly sophisticated in their verbal as well as nonverbal behavior, such as facial expressions and gestures. Now that such "natural" communication channels are available for expressing not only task-relevant, but also socially and psychologically relevant information, we have to take into account subtle influences that are not easy to implement, such as emotions and cultural heuristics (culturally relevant behavior). These influences have a huge impact on the success of an interaction, but people are seldom conscious of generating or interpreting these signals.
Our research focuses on culture as a vital aspect of successful human-agent interactions, by analyzing how culture influences the expression of multimodal behavior in agents. Our goal is to enhance cultural understanding by making virtual agents culturally aware.

Face and Gesture

Facial feature detection and tracking in a sequence of images
  1. Fundamental research on face recognition
    1. Facial feature detection and tracking in a sequence of images
    2. Face recognition based on detected facial features and their relationships
    3. Face expression recognition based on the motion of facial features
  2. Fundamental research on gesture recognition
    1. Body feature detection (hands, feet, face) and tracking in a sequence of images
    2. Gesture recognition based on motion of detected body features
  3. Application of these face and gesture recognition methods to a variety of fields
    1. Monitoring and security
    2. Virtual conductor systems
    3. Virtual instruments
    4. Game playing
    5. Presentation control, and so on

Three-Dimensional Image Processing

Three-Dimensional Image Processing

We are studying ways to process three-dimensional image and shape data, for example, how to acquire three-dimensional shapes by using readily available digital still cameras, and how to create 360-degree stereoscopy by adding special lenses. We are also investigating ways to acquire human movement data using a monocular camcorder, and then edit and use the acquired motion data.

Randomness-based Scene Analysis for Cooperative Decision Making



Despite the intrinsic uncertainty of landmark allocation in conditions of unstable ambient light, the randomness that is a natural part of complex scenes provides robust features for inducing the multitude of cooperative perception. Fractal sampling of subtle chromatic diversity is transferred to a satellite image to extend a GPS track towards a future route point, as illustrated in (a) below. Associated bird's eye views beyond a physical-geometric perspective are downloaded and used to adapt on-vehicle vision to a predicted fluctuation of saliency patterns to be noticed in subsequent scenes, as shown in (b) below.