Graduate School of Science and Engineering
Information and Computer Science
Spoken Language Processing Laboratory
Aiming to achieve systems to assist communication by spoken language
- Robust speech recognition for conversational speech
- Multimodal information processing for conversation
- Developing speech recognition systems for Japanese-accented English
- Constructing computer assisted language learning systems
- Machine translation by corpora and machine learning
- Spoken dialogue system
- Stochastic software keyboard for smart devices
Research background and goalsThe world is globalizing and opportunities to communicate in foreign languages are increasing. Opportunities are also increasing to give computers and robots various instructions by voice and receive information from them.
Speech (spoken language) is said to be more context dependent than text (written language). There are mainly omitted items easily presumed from the context where the conversation is taking place, and there is also much ungrammatical phenomena such as restarts, filled pauses, and so on, because of “thinking while speaking, speaking while thinking” speech behavior. What kind of effects of this spoken language have when trying to communicate with speaker of different mother tongue and computers and robots in conversational speech. Conversely, in what way can we accurately and efficiently communicate with these subjects.
We are searching for mechanisms to establish such speech communication from various perspectives and researching and developing technologies assisted such conversations. Specifically, we are researching topics such as computer assisted language learning (CALL) to support efficient foreign language learning. To develop such systems we are developing robust speech recognition technologies for conversational speech whose acoustic feature varies in speaking. We also developing technologies that can recognize Japanese-accented English speech.
Approach and methods to solve these issuesAs a technique for natural language processing, a rule-based approach to develop processing rules based on developer’s introspection has been primarily used in the past. A rule-based approach is a valuable knowledge source that concentrates the many years of experience of those developers, but for a large-scale system, maintaining uniformity and maintenance are difficult issues because many developers are involved. With the increases in computer processing ability and machine readable corpora (texts with added information such as the part of speech, etc.) useable by computers, a corpus-based approach, an approach to automatically acquire knowledge by machine learning from corpora, is attracting attention. Centered on a corpus-based approach that applies machine learning techniques to foreign language learner’s corpora, speech databases, and parallel texts, we are researching and developing CALL systems and speech dialogue systems with robots.
The corpus-based approach is a powerful approach to natural language processing and spoken language processing, but this does not mean a corpus-based approach is good at everything. How to incorporate the natural knowledge of humans is also an important research theme. We are advancing research on how humans understand spoken language.
Specific research themesTo support the efficient learning of foreign language learners, their abilities must be accurately understand and problems given according to those abilities. For this, methods are required to objectively measure the kinds of abilities below and to measure the difficulty of problems. This research and development requires a large-scale research corpus and development of a speech recognition system. Additionally, in order to extract problems which arise in the actual use, we will develop a system integrating these technologies.
Furthermore, Kato is in charge of new research topics, "spoken dialogue systems" and "stochastic software keyboard for smart devices".
Speech recognition technology
- Developing acoustic models and language models suitable to Japanese-accented English speech
- Developing a speech recognition system to recognize Japanese-accented English
- Reliability evaluation technology for translations from a large-scale English text corpus
- Corpus-based translation technology such as statistical translation technology
- Developing Japanese to English translation data and English speech data by people with a variety of English abilities
- Automatic measurement method for English text construction ability (English speech ability) based on the distance, etc., from a reference translation
- In ability measurements, a method to select appropriate problems with different level of difficulty
- Analysis of difference between difference features in conversations in mother tongue (L1) and foreign language (L2)
- Analysis of difference between eye movement in conversations in L1 and L2
- Researching a difficulty evaluation scale when translating the given Japanese text into English
- Researching an automated difficulty evaluation scale for Japanese text translated to English
- Dialogue-based CALL system for English conversations to point out a learner’s problems
- Integrate technologies such as English speech recognition, dialogist ability measurement, translation problem selection, and translation technologies
- Statistical understanding model based on neural networks
- Development of stochastic touch model
- Online user adaptation of the stochastic touch model
- Situation recognition and adaptation baased on analysis of multimodal touch data including eye gaze and acceleration data
- New user interface design for elderly
- Speech recognition
- Natural language processing
- Nonlinear speech signal processing
- Acquiring foreign language (L2) ability
- Spoken language processing