Laboratories in Department of Media Science
- Human Interface Laboratory - Prof. Tomoko KODA
- Multimedia Applications Laboratory - Prof. Makoto J. HIRAYAMA
- Speech and Music Information Processing Laboratory - Prof. Motoyuki SUZUKI
- Sensory Media Laboratory - Prof. Wataru HASHIMOTO
- Spatial Imaging System Laboratory - Prof. Masahiro KAWAKITA
- Natural Language Processing Laboratory - Prof. Hirotoshi TAIRA
- Interaction Design Laboratory - Prof. Mutsuo SANO
- Sensor Information Processing Laboratory - Associate Prof. Kenzaburo MIYAWAKI
- Visual Information Processing Laboratory - Associate Prof. Norihiko KAWAI
- Visual Computing Laboratory - Associate Prof. Yuta MURAKI
- Communication Design Laboratory - Associate Prof. Taku FUKUSHIMA
- Media Interaction Laboratory - Assistant Prof. Sho OI
Human Interface Laboratory
Prof. Tomoko KODA
Our research areas are "Intelligent Virtual Agents (IVA)" and Human-Agent Interaction (HAI)". Our research focuses on new engineering issues such as how intelligent virtual agents and humanoid robots can get along well with their human users, and what kind of interactions should be designed between humans and agents in order to achieve this.
Specifically, in order to give agents and robots "humanity and human-like qualities," we will evaluate how to realize emotions and non-verbal behaviors that make humans human-like in computer models, and how much human interaction is improved as a result. We will apply the results of research on humans in cognitive science and psychology to information science.
Main Research Topics
- Development and interaction evaluation of intelligent virtual agents and humanoid robots with human-like behaviors
- Research on cross-cultural communication using agents and robots with culturally adapted behaviors
Multimedia Applications Laboratory
Prof. Makoto J. HIRAYAMA
With IT technology at the core, we develop multimedia applications and create works of art using a combination of video, images, computer graphics, sound, voice, sensation, and motion. We will contribute to society by creating useful things for people. We will contribute to society by developing and producing multimedia applications using a combination of images, computer graphics, sound, voice, sensation, and motion, and by creating useful products. We will implement these systems in PCs and mobile terminals by means of software programs, and conduct research while conducting demonstration experiments. As basic research, we are making a model of tongue movement during speech. We are also focusing on information technology standardization.
Main Research Topics
- Production of visual and audio contents and technological development (human body animation, clay animation, sound production, virtual studio)
- Basic research on speech production and production of models of tongue movements during speech production
- Development of assistive technologies for the visually impaired (make-up support, audio guidance for movies, e-books, web accessibility)
- Development of assistive technologies for the hearing impaired (conversation support applications, summary writing support, sign language recognition and synthesis)
- Information technology standardization in the multimedia field
Speech and Music Information Processing Laboratory
Prof. Motoyuki SUZUKI
Humans interact with others by sensing their emotions which can be felt from their voice. For example, you say, "Let's go bowling," and your friend answered "Oh, bowling.” If he seems to be having fun, it means "Let's go," but if he seems to be bewildered, it means "I don't want to.” We are developing a system that can recognize these differences accurately and add appropriate emotions to the system's own speech, so that it can interact naturally with humans. We are also developing several speech-related systems; a music retrieval system which uses singing voice with lyrics, humming, onomatopoeia as a retrieval key. And a "lip reading" system that automatically generates speech from lip images.
Main Research Topics
- Research on emotion recognition from speech
- Research on music retrieval systems using singing voice
- Research on more natural spoken dialogue systems
- Research on automatic speech generation from lip images
Sensory Media Laboratory
Prof. Wataru HASHIMOTO
Humans have a variety of senses. Humans perceive information in the outside world through these senses. It is important to use these senses to express information without relying on sight and hearing to efficiently convey the vast amount of information handled by computers to humans. In the Sensory Media Laboratory, we are researching information media technologies that fully utilize the senses, such as information expression using force and touch, and immersive visual media that generate a sense of presence.
Main Research Topics
- Research on information presentation using tactile and force sensations
- Research on immersive visual media
Spatial Imaging System Laboratory
Prof. Masahiro KAWAKITA
Augmented reality (AR) and virtual reality (VR), which are highly realistic display technologies, have spread throughout game and entertainment industry, and new sensors, displays, and other equipment for consumer use are now being commercialized. Studies are also underway on next-generation highly realistic spatial images, such as three-dimensional (3D) and aerial images, that do not require special glasses to reproduce multiple parallax images and groups of light rays to achieve realistic image displays. In our laboratories, we are researching 3D camera and display system based on the light-field method, which is a spatial imaging technology, can reproduce natural 3D images in accordance with the viewer’s position by faithfully reproducing light rays from objects. We are also studying several 3D interactive video systems by using aerial images and an ultra-high-definition video to improve the quality of spatial images.
Main Research Topics
- 3D camera and display
- 3D and Aerial interactive video system
- Ultra-high-definition video system
Natural Language Processing Laboratory
Prof. Hirotoshi TAIRA
The world is flooded with large amounts of data, including text, images, video, and sensor information. However, if the data just exists, it is a waste of treasure. In order to easily access the desired items from a large amount of data, technology that links words and data, is essential. In the future, there will also be a great need for technology that allows robots and humans to communicate with each other in words, for example, in the use of robots in nursing care. In order to solve such problems, our laboratory conducts research on modeling of human language processing and creation of semantic analyzers and language generators using machine learning methods.
Main Research Topics
- Research on semantic analysis of language
- Research on artificial intelligence that understands the meaning of words and responds to them
- Research on information processing that integrates language with other media such as sound and images
- Research on machine reading comprehension
- Development of applications based on language processing, such as machine translation, dialogue processing, search, and text mining
- Research on algorithms and machine learning methods that form the basis of natural language processing
Interaction Design Laboratory
Prof. Mutsuo SANO
In recent years, new interfaces, and information systems such as those used in smart phones allow us to enjoy virtual reality and improved life navigation. Robots and avatars can be controlled with our minds, and they are having a great impact on human society. With these new technologies, the way they interact with people, objects, and information is becoming more and more important. In this laboratory, we focus on the relationship between humans and information media using imaging, acoustics, and the sense of touch, and conduct joint research with medical professionals on methods of communication design that can recognize emotions to reduce stress. The development of these media interfaces should be kind to people and provide a sense of security, and promote awareness and motivation.
Main Research Topics
- An ADL training system based on xR (VR, AR, MR, SR…) technologies and AI
- Avatars and communication robots which can be controlled with our minds through behavior and emotion recognition
- A VR rehabilitation for patients with cognitive disability
Sensor Information Processing Laboratory
Associate Prof. Kenzaburo MIYAWAKI
We are researching and developing software that processes information obtained from various sensors for systems that support human behavior, such as robots that move accurately.
Specifically, by processing and integrating the output of various sensors, such as cameras, laser range finders, and 3D depth sensors, and extracting information on human behavior and the surrounding environment (obstacles and terrain around the robot), we aim to realize systems that can operate safely and efficiently in real environments and assist humans.
Main Research Topics
- Research on indoor and outdoor navigation robots
- Research on support systems for daily life activities such as cooking
- Dialogue systems that take into account the human condition
- Android devices and applications using their built-in sensors
Visual Information Processing Laboratory
Associate Prof. Norihiko KAWAI
In this laboratory, we are engaged in research and development to obtain valuable images, three-dimensional models, and recognition results through image processing and recognition by inputting image information obtained from cameras and depth sensors. We are also working on the application of such technologies to augmented reality (AR) and virtual reality (VR).
Main Research Topics
- Computer Vision
- Augmented Reality
- Virtual Reality
- Diminished Reality
- Image and video inpainting
Visual Computing Laboratory
Associate Prof. Yuta MURAKI
We conduct research on the representation of computer graphics and aim to integrate information technology with various fields. Specifically, we are researching various shape processing techniques for point cloud data obtained from 3D scanners, and developing a high-precision digital archive system. We are also conducting research on curved surface representation technology and applying it to the compression of CAD data. We will also develop applications using image processing technology.
Main Research Topics
- Research on 3D measurement
- Research on information extraction from measured point clouds
- Research on curved surface representation
- Research on CAD data compression based on curved surface representation
- Application development based on image processing
Communication Design Laboratory
Associate Prof. Taku FUKUSHIMA
Our research focuses on the support of communication between people using information technology. We are working to solve various problems in society through computers. In recent years, a variety of communication tools using information technology have been widely used. However, there are still various gaps between people in terms of language, culture, knowledge, and so on. Therefore, we are conducting research on how to enable people to communicate naturally and smoothly using various information technologies and devices.
Main Research Topics
- Research on multilingual communication support for medical field
- Research on evacuation drill support system that can be used in daily life
- Research on human-to-human communication support using information technology
- Research on support of motivation maintenance using information technology
Media Interaction Laboratory
Assistant Prof. Sho OI
Media Interaction Laboratory (MixLab) studies media technologies to improve the quality of life (QOL) in various fields such as education, medical care, nursing care, entertainment, artificial intelligence, and robotics through the combination of various media technologies and interaction between people and society and media technologies.
Main Research Topics
- Research on cognitive rehabilitation
- Research on educational support applications
- Research on entertainment
- Research on behavior analysis