Laxmi Narayana Mangalagiri

  
  Home | Research | Publications | Personal Interests | Darshan Quiz

Research Interests

Natural Language Processing, Speech Processing, Digital Image Processing.

Research during ME in IISc, Bangalore

1. Natural Language Processing for Tamil TTS

Text to Speech Synthesis is an automated encoding process which converts a sequence of symbols conveying linguistic information (text) into an acoustic waveform (speech). The two major components of a TTS system are Natural language processing (NLP) module and Digital Signal processing (DSP) module. The NLP module in a Text to Speech Synthesizer is responsible for the determination of the phonetic transcription of the incoming text. A Text normalizer, a Grapheme to Phoneme converter and a foreign word lexicon have been developed to form the NLP module for our Tamil TTS system. The text normalizer expands the non-standard words and number sequences in the input text into the corresponding Tamil graphemic form. The Grapheme to Phoneme converter generates the phonetic transcription of the normalized orthographic text. The NLP module has been designed to be used for Tamil TTS synthesis in both Windows platform and Festival (Linux) environment.

2. Stress Labeling of Syllables in Tamil TTS Corpus

Speech synthesis based on syllables seems to be a good possibility to enhance the quality of synthesized speech compared to mono-phone and diphone-based synthesis. Syllable is the minimum possible speech segment, which can be spoken without the help of the succeeding or the preceding phone. Keeping this in mind, the syllable structure is dynamically chosen such that a syllable contains all the coarticulation effects of the phones present in it. A method for Stress labeling of syllables based on the combination of their Pitch, Energy and Duration (PED) is proposed. Two types of clustering are carried out for classifying the stressed and unstressed syllables. K-means clustering algorithm after variance normalization of data showed better results in classifying the stressed and unstressed syllables. A more robust method of stress labeling is carried out which gives stress rating to the syllables based on the distance of the syllable vectors from a reference vector constructed from maximum feature values. Perception experiments are conducted to compare the stress rating given by the proposed stress labeling strategy.

3. Preliminary Prosody Model for Tamil TTS

 

The byproduct of the stress labeling of the database is the Prosodic knowledge that the first syllable in Tamil is stressed (in most cases, except when there is some emphasis of a particular syllable in a word) and the second syllable is stressed if the first syllable is hrasva. This confirms the earlier studies of some linguists. The fact has been verified acoustically too.

 

4. Dealing with Missing Phonetic contexts in Speech Synthesis

 

Any corpus under consideration, though phonetically rich, cannot cover all the possible contexts of a particular phone or syllable. An attempt has been made to handle situations in Text to Speech Synthesis, where a particular phonetic or syllabic context is not available in the corpus. A novel way of systematic replacement of missing phones has been proposed for speech synthesis. The idea is to replace any such context by another one which is ‘similar’. The ‘similarity’ of phones or syllables lies in the inability of listeners to distinguish between them when placed in a particular context. Such phones were found linguistically in two south Indian languages - Tamil and Telugu and also in English, by performing listening tests and acoustically, through a phone classification experiment with Mel Frequency Cepstral Coefficients as features. Maximum Likelihood classifier is used to find the most confusing phones in Tamil and English languages. The knowledge of the pairs of phones, confused for each other, is incorporated in Tamil TTS and experiments show that the proposed phone replacement strategy is fairly successful. The confused phones in English are also found, whose knowledge can be used for future purposes of developing a bilingual TTS.

 

5. Study on natural variability in Human Speech

 

A human being does not always speak a given sentence/phrase/word in exactly the same manner every time she/he speaks. The different realizations from the speaker vary even within words, not just from one complete utterance to the next. A study of the parameters of speech which vary when same speech is spoken at different times, for one speaker and across different speakers, is conducted which helps in making a synthesizer speak with slightly different characteristics at different times.

 

Research in TCS Innovation Lab - Mumbai

 

1. Pronunciation Lexicon Creation for Proper Names

 

Development of a pronunciation lexicon for proper names in general and Indian proper names in particular is usually a manual effort which can not be avoided. Grapheme to phoneme (G2P) modules developed in literature are usually rule based and work best for non-proper names in a particular language. Proper names are foreign to a G2P conversion module. For the construction of a semi-automatic pronunciation dictionary for proper names, an optimization problem is formulated in the current work. The idea is to construct a ‘small’ orthogonal set of words (basis) which can span the complete set of names in a database. The transcription lexicon of all the proper names in a database can be produced by the manual transcription of only the basis words. A cost function based on a set of features is constructed for this purpose and the minimization of this cost function results in an orthogonal basis.

 

2. Effect of Noise-in-Speech on MFCC Parameters

 

Manifestation of noise in speech is inherent unless a conscious effort is made to minimize the disturbances in the surroundings while recording speech. The performance of a speech recognition system often degrades in the presence of noise. In this paper, we study the effect of noise in the speech signal on the extracted speech features that are used in speech recognition. Mel frequency cepstral coefficients (MFCCs) are the most popularly used speech features in speech and speaker recognition applications. We first show theoretically, how additive Gaussian noise with mean, μ and variance, s2 effects the speech parameters (MFCCs). The mean and variance of the error in MFCC due to noise in speech is related to the mean and variance of the noise added. We experimentally verify that additive Gaussian noise in speech results in an error in MFCC parameter estimation which is also Gaussian.

 

3. The use of inherent Stress Information in speech for Speaker Recognition

 

ASR systems and Speaker recognition systems find difficulty when the speaker is under stress or emotional. The way a person expresses his emotion and stresses certain syllables is unique to her/him and most of the times this style of speaking is consistent. The current work explores to find a mechanism where the inherent stress information in the speech can be used as an additional feature for speaker recognition (identification/verification).

 

4. Time scale Modification of Speech signals

 

Time compress the speech signal without change in pitch; To what extent the time can be compressed such that the speech recognition accuracy is reasonable. (Fast browsing of audio/speech data); once a time compressed signal is obtained, extract features (MFCC) and compare with the original features.