Two-way speech-to-speech translation on handheld devices

This paper presents a two-way speech translation system that is completely hosted on an off-the-shelf handheld device. Specifically, this end-to-end system includes an HMM-based large vocabulary continuous speech recognizer (LVCSR) for both English and Chinese using statistical n-grams, a two-way translation system between English and Chinese, and, a multilingual speech synthesis system that outputs speech in the target language. This paper describes the system development and the functionality of the major components, focusing on the optimization efforts employed to achieve real time results on such a limited platform.