Iain Matthews, Gerasimos Potamianos, et al.
ICME 2001
This paper addresses the problem of audio-visual information fusion to provide highly robust speech recognition. We investigate methods that make different assumptions about asynchrony and conditional dependence across streams and propose a technique based on composite HMMs that can account for stream asynchrony and different levels of information integration. We show how these models can be trained jointly based on maximum likelihood estimation. Experiments, performed for a speaker-independent large vocabulary continuous speech recognition task and different integration methods, show that best performance is obtained by asynchronous stream integration. This system reduces the error rate at a 8.5 dB SNR with additive speech "babble" noise by 27% relative over audio-only models and by 12% relative over traditional audio-visual models using concatenative feature fusion.
Iain Matthews, Gerasimos Potamianos, et al.
ICME 2001
Ambrish Tyagi, James W. Davis, et al.
WMVC 2008
Roland Goecke, Gerasimos Potamianos, et al.
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Patrick Lucey, Gerasimos Potamianos, et al.
AVSP 2008