Speech recognition and understanding [1] of spontaneous speech have been a goal ofresearch sine 1970. It is a process of conversion of speech to text. The object of humanspeech is not just a way to convey words from one person to another but also to make theother person to understand the depth of the spoken words. Spontaneous speech is highlyvariable and rarely conforms to conventional assumptions and linguistically definedpronunciation rules. Specifically, there may be many different continuous speech realizationsfor each expertly defined phonetic unit in the dictionary. The phones may be realized in a clean and complete fashion as in read speech, or they may be realized in a sloppy andincomplete fashion as in highly spontaneous speech. For spontaneous speech, therefore, itmay be beneficial to model incompletely realized variants of any phonetic unit as separateunits. The purpose of this paper is to develop a method to build automatic spontaneousspeech recognition for Punjabi language. Initially, the Spontaneous Speech Recognitionmodel has been implemented for small vocabulary. We successfully build the acoustic modelfor Punjabi language using Sphinx4 and java programming and Language model fordecoding.