(click here for main project page and user registration)

SONIC is a toolkit for enabling research and development of new algorithms for continuous speech recognition.  Since March of 2001, SONIC has been used as our test bed for integrating new ideas and for supporting research activities that include speech recognition as core components at the Center for Spoken Language Research.   While not a general HMM modeling toolkit, SONIC is specifically designed for speech recognition research with careful attention applied for speed and efficiency needed for real-time use in live applications.

SONIC utilizes state-of-the-art statistical acoustic and language modeling methods.  The system acoustic models are decision-tree state-clustered Hidden Markov Models (HMMs) with associated gamma probability density functions to model state-durations.  The recognizer implements a two-pass search strategy.  The first pass consists of a time-synchronous, beam-pruned Viterbi token-passing search through a lexical prefix tree.  Cross-word acoustic models and trigram or four-gram language models are applied in the first pass of search.  During the second pass, the resulting word-lattice is converted into a word-graph.  One can dump lattices in Finite State Machine (FSM) format compatible with AT&T tools. Also one can generate n-best lists using the A* algorithm. Finally, word-posterior probabilities can be calculated to provide a measure of word confidence.

The recognizer also incorporates both model-based and feature-based speaker adaptation methods. Model-based adaptation methods include: Maximum Likelihood Linear Regression (MLLR), Structured MAP Linear Regressions (SMAPLR).  In addition, SONIC also includes implementation of feature-based adaptation methods such as Vocal Tract Length Normalization (VTLN), cepstral mean & variance normalization, and Constrained MLLR (CMLLR).  Finally, advanced language-modeling strategies such as concept language models can be used to improve performance for dialog system recognition tasks.

Phonetic Aligner

• Provides word, phone, and HMM state-level boundaries for acoustic training.  
• Decision-tree based trainable letter-to-sound prediction module
• Multilingual lexicon support
• API for integration into applications (e.g., lip-synchronization)

Phonetic Decision Tree Acoustic Trainer

• Estimates parameters of state-clustered continuous density Hidden Markov Models
• Incorporates phonetic position & context questions
• Distributed / parallel acoustic trainer (multiple machines, operating systems, CPUs)

Core Recognizer

• Token-passing based recognition using a lexical-prefix tree search
• Cross-word triphone models & up to 4-gram language model in first-pass
• HMM state durations modeled using Gamma distributions
• N-best list output; Lattice dumping, and second-pass rescoring functionality
• Word confidence computed from word-posteriors of word-graph
• Class-based, word-based, and concept-based n-gram language models
• Dynamically switched statistical language models (dialog state-conditioned LMs)
• Keyword & regular expression based grammar spotting with confidence
• Phonetic fast-match constrained ASR search for improved decoding speed
• Mel Frequency Cepstral Coefficient (MFCC) feature representation

Speaker Adaptation

• (Confidence Weighted) Maximum Likelihood Linear Regression (MLLR)
• Lattice-based MLLR (Lattice-MLLR)
• Constrained MLLR (CMLLR)
  
• Structural Maximum a Posterior Linear Regression (SMAPLR)
• Vocal Tract Length Normalization (VTLN)
• Cepstral mean and variance normalization

Live-Mode Recognition and Voice Activity Detection

• API for streaming audio for keyword/grammars and continuous speech recognition
• Internal, HMM-based voice activity detection method

Language Portability

• Designed to incorporate new phone sets and foreign vocabularies.  
• SONIC has been ported to German, Spanish, French, Italian, Croatian, Arabic, Russian, Portuguese, Korean, Turkish, and Japanese

Speech Compression Interface

• Libraries and APIs for utilizing the Speex subband CELP coding system are provided within SONIC. 
• SONIC in server mode accepts both raw PCM and compressed bitstreams.

Application Programming Interface (API)

• API environment for linking and designing speech enabled applications.
• Batch-mode and simulated live-mode example
• Socket-based Tcl/Tk client / C-code server example

Supported Operating Systems

• Linux, Sun Solaris, Microsoft Windows, and Mac OS X

Project Management:

Bryan Pellom	Research Assistant Professor, CSLR / Department of Computer Science pellom@cslr.colorado.edu
Kadri Hacioglu	Research Associate, CSLR / Institute for Cognitive Science hacioglu@cslr.colorado.edu

Software Contributors:

Bryan Pellom	Research Assistant Professor, CSLR / Department of Computer Science
Kadri Hacioglu	Research Associate, CSLR / Institute for Cognitive Science
Wayne Ward	Research Professor CSLR / Institute for Cognitive Science
Umit Yapanel	PhD student, Eletrical Engineering
Min Tang	PhD student, Computer Science
Andreas Hagen	PhD student, Computer Science
Daniel Cer	PhD student, Computer Science / Institute for Cognitive Science
Nicholas Romanyshyn	Senior, Undergraduate, Computer Science

Past Software Contributors:

Ruhi Sarikaya	PhD, Electrical Engineering, 2000 currently with IBM Watson Speech Group
Keith Herold	MS, Computer Science / Linguistics currently with Lumenvox Speech Group

Recent Related Publications:

Core System Description
Bryan Pellom, Kadri Hacioglu, "Recent Improvements in the CU SONIC ASR System for Noisy Speech: The SPINE Task", in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hong Kong, April, 2003 (pdf)

Bryan Pellom, "SONIC: The University of Colorado Continuous Speech Recognizer", University of Colorado, tech report #TR-CSLR-2001-01, Boulder, Colorado, March, 2001 (pdf)

Children's Speech Recognition
R. Cole, S. van Vuuren, B. Pellom, K. Hacioglu, J. Ma, J. Movellan, S. Schwartz, D. Wade-Stein, W. Ward, J. Yan , "Perceptive Animated Interfaces: First Steps Toward a New Paradigm for Human Computer Interaction", in Proceedings of the IEEE: Special Issue on Human Computer Interaction, vol. 91, no. 9, pp. 1391-1405, September, 2003 (pdf)

Andreas Hagen, Bryan Pellom, Ronald Cole, "Children's Speech Recognition with Application to Interactive Books and Tutors", in IEEE Automatic Speech Recognition and Understanding (ASRU) Workshop, St. Thomas, USA, December, 2003 (pdf)

Andreas Hagen, Daniel Connors, Bryan L. Pellom , "The Analysis and Design of Architecture Systems for Speech Recognition on Modern Hand-Held Computing Devices", First IEEE/ACM/IFIP International Conference on Hardware/Software Codesign & System Synthesis, pp. 65-70, Newport Beach, California, USA, October, 2003 (pdf)

Accented Speech Recognition
Ayako Ikeno, Bryan Pellom, Dan Cer, Ashley Thornton, Jason Brenier, Dan Jurafsky, Wayne Ward, William Byrne , "Issues in Recognition of Spanish-Accented Spontaneous English", in ISCA & IEEE Workshop on Spontaneous Speech Processing and Recognition, Tokyo, Japan, April, 2003 (pdf)

Distributed Speech Recognition
Kadri Hacioglu, Bryan Pellom, "A Distributed Architecture for Robust Automatic Speech Recognition", in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hong Kong, April, 2003 (pdf)

Multilingual Speech Recognition
Kadri Hacioglu, Bryan Pellom, Tolga Ciloglu, Ozlem Ozturk, Mikko Kurimo, Mathias Creutz, "On Lexicon Creation for Turkish LVCSR", in Eurospeech - Interspeech, pp. 1165-1168, Geneva, Switzerland, September, 2003 (pdf)

B. L. Pellom and R. Cole, " The CSLR International Workshop ," (Sponsored by NSF), Summer 2003.  SONIC was ported to French, German, Italian, and Spanish (Mexican & Chilean) for use in interactive books and tutors.

O. Salor, B. L. Pellom, T. Ciloglu, K. Hacioglu, M. Demirekler, "On Developing New Text and Audio Corpora and Speech Recognition Tools for the Turkish Language", in International Conference on Spoken Language Processing (ICSLP), pp. 349-352, Denver, Colorado, September, 2002 (pdf)

Call Center Speech Recognition
Min Tang, Bryan Pellom, Kadri Hacioglu, "Call-Type Classification and Unsupervised Training for the Call Center Domain", in IEEE Automatic Speech Recognition and Understanding (ASRU) Workshop, St. Thomas, USA, December, 2003 (pdf )

Confidence Assessment for Speech Recognition
Kadri Hacioglu & Wayne Ward, "A Concept Graph based Confidence Measure", Proc. IEEE International Conference on Acoustic, Speech, and Signal Processing (ICASSP), Orlando Florida, May 2002. (pdf)

Kadri Hacioglu, Wayne Ward, "A Word Graph Interface for a Flexible Concept Based Speech Understanding Framework," Proc. Eurospeech, Aalborg Denmark, September 2001. (pdf)

Kadri Hacioglu, Wayne Ward, "Dialog-Context Dependent Language Modeling Using N-grams and Stochastic Context-Free Grammars", Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Salt Lake City, May 2001. (pdf)

Robust Speech Recognition
Umit Yapanel, John H.L.Hansen, "A new perspective on Feature Extraction for Robust In-vehicle Speech Recognition", Proceedings of Eurospeech'03, Geneva, Sept. 2003 (pdf)

Umit Yapanel, Xian Xian Zhang, J.H.L. Hansen, "High Performance Digit Recognition In Real Car Environments," Inter. Conf. on Spoken Language Processing (ICSLP)., vol. 2, pp. 793-796, Denver, CO, Sept. 2002 (pdf)