Model-Based Margin Estimation for Hidden Markov Model Learning and Generalization

Recently, speech scientists have been motivated by the great success of building margin-based classifiers, and have thus proposed novel methods to estimate continuous-density hidden Markov model (HMM) for automatic speech recognition (ASR) according to the notion that the decision boundaries determined by 14 the estimated HMMs attain the maximum classfication margin as in learning support vector machines (SVMs). Although a good performance has been observed, the margin used in the ASR community is often specifid as a parameter that has no explicit relationship with the HMM parameters. The issues of how the margin is related to the HMM parameters and how it directly characterizes the generalization capability of HMM-based classfiers have not been addressed so far in the community. In this paper we attempt to formulate the margin used in the soft margin estimation (SME) framework as a function of the HMM parameters. The key idea is to relate the standard distance-based margin with the concept of divergence among competing HMM state Gaussian mixture model densities. Experimental results show that the proposed model-based margin function is a good indication about the quality of HMMs on a given ASR task without the conventional needs of running experiments extensively using a separate set of test samples.