professor c: And  Hans -  , Hans - Guenter will be here , I think by next next Tuesday or so . professor c: So he 's he 's going to be here for about three weeks , phd b: Oh ! That 's nice . professor c: So he 's gonna look in on everything we 're doing phd d:   grad e: Th - that 's his spectral subtraction group ? professor c: Yeah , grad e: Is that right ? professor c: yeah . He 's , very , very , easygoing , easy to talk to , and , very interested in everything . professor c: he 's he 's he 's he 's phd a: Wh - Back when I was a grad student he was here for a ,  a year or n six months .  , I went around and talked to everybody , and it seemed like they they had some new results but rather than them coming up and telling me I figured we should just wait a week and they can tell both you know , all of us . So , why don't we why don't we start with you , Dave , and then , we can go on . grad e: So , since we 're looking at putting this ,  mean log m magnitude spectral subtraction , into the SmartKom system , I I did a test seeing if , it would work using past only and plus the present to calculate the mean . So , I did a test , where I used twelve seconds from the past and the present frame to , calculate the mean . And phd a: Twelve seconds Twelve twelve seconds back from the current frame , is that what you mean ? grad e:  Twelve seconds , counting back from the end of the current frame , phd a: OK , OK . So it was , twen I think it was twenty - one frames and that worked out to about twelve seconds . grad e: And compared to , do using a twelve second centered window , I think there was a drop in performance but it was just a slight drop . grad e: Is is that right ? professor c: yeah , it was pretty it was pretty tiny . And , that that  , that 's encouraging for for the idea of using it in an interactive system like And , another issue I 'm I 'm thinking about is in the SmartKom system . So say twe twelve seconds in the earlier test seemed like a good length of time , but what happens if you have less than twelve seconds ? And ,  So I w bef before ,  Back in May , I did some experiments using , say , two seconds , or four seconds , or six seconds . In those I trained the models using mean subtraction with the means calculated over two seconds , or four seconds , or six seconds . And , here , I was curious , what if I trained the models using twelve seconds but I f I gave it a situation where the test set I was subtracted using two seconds , or four seconds , or six seconds . And , I th I think it was , four se I think I think it was , something like four seconds and , six seconds , and eight seconds . And it seems like it it it hurts compared to if you actually train the models using th that same length of time but it it doesn't hurt that much .  , u usually less than point five percent , although I think I did see one where it was a point eight percent or so rise in word error rate . But this is , w where , even if I train on the , model , and mean subtracted it with the same length of time as in the test , it the word error rate is around , ten percent or nine percent . professor c: But it but looking at it the other way , isn't it what you 're saying that it didn't help you to have the longer time for training , if you were going to have a short time for grad e: That that 's true .  , professor c: why would you do it , if you knew that you were going to have short windows in testing . grad e: Wa phd a: Yeah , it seems like for your  , in normal situations you would never get twelve seconds of speech , right ? I 'm not e u phd b: You need twelve seconds in the past to estimate , right ? Or l or you 're looking at six sec seconds in future and six in professor c: Yeah . phd a: Is this twelve seconds of  , regardless of speech or silence ? Or twelve seconds of speech ? grad e: Of of speech . professor c: The other thing , which maybe relates a little bit to something else we 've talked about in terms of windowing and so on is , that , I wonder if you trained with twelve seconds , and then when you were two seconds in you used two seconds , and when you were four seconds in , you used four seconds , and when you were six and you basically build up to the twelve seconds . grad e: But s so I g So I guess the que the question I was trying to get at with those experiments is , " does it matter what models you use ? Does it matter how much time y you use to calculate the mean when you were , tra doing the training data ? " professor c: Right . But  the other thing is that that 's  , the other way of looking at this , going back to , mean cepstral subtraction versus RASTA kind of things , is that you could look at mean cepstral subtraction , especially the way you 're doing it , as being a kind of filter . You know , basically you 're you 're you 're doing a high - pass filter or a band - pass filter of some sort and and just design a filter . And then , you know , a filter will have a certain behavior and you loo can look at the start up behavior when you start up with nothing . professor c: And and , you know , it will , if you have an IIR filter for instance , it will , not behave in the steady - state way that you would like it to behave until you get a long enough period , but , by just constraining yourself to have your filter be only a subtraction of the mean , you 're kind of , you know , tying your hands behind your back because there 's filters have all sorts of be temporal and spectral behaviors . professor c: And the only thing , you know , consistent that we know about is that you want to get rid of the very low frequency component . phd b: But do you really want to calculate the mean ? And you neglect all the silence regions or you just use everything that 's twelve seconds , and grad e: you do you mean in my tests so far ? phd b: Ye - yeah . grad e: th  , yeah , I guess it 's something I need to play with more to decide how to set that up for the SmartKom system . Like , may maybe if I trained on six seconds it would work better when I only had two seconds or four seconds , and professor c: Yeah . professor c: Yeah , and again , if you take this filtering perspective and if you essentially have it build up over time .  , if you computed means over two and then over four , and over six , essentially what you 're getting at is a kind of , ramp up of a filter anyway . But , if you do that , then , in practice somebody using the SmartKom system , one would think if they 're using it for a while , it means that their first utterance , instead of , you know , getting , a forty percent error rate reduction , they 'll get a  , over what , you 'd get without this , policy , you get thirty percent . And then the second utterance that you give , they get the full you know , full benefit of it if it 's this ongoing thing . phd a: Oh , so you you cache the utterances ? That 's how you get your ,  professor c: Well , I 'm saying in practice , yeah , grad e: M phd a: Ah . professor c: that 's If somebody 's using a system to ask for directions or something , phd a: OK . And and to begin with if it doesn't get them quite right , ma m maybe they 'll come back and say , " excuse me ? " phd a:   And so , there 's a higher probability of it making an error , in the first utterance . phd a: What would be really cool is if you could have  , this probably users would never like this but if you had could have a system where , before they began to use it they had to introduce themselves , verbally . professor c: Oh , the other thing I guess which which , I don't know much about as much as I should about the rest of the system but but , couldn't you , if you if you sort of did a first pass I don't know what kind of , capability we have at the moment for for doing second passes on on , some kind of little small lattice , or a graph , or confusion network , or something . But if you did first pass with , the with either without the mean sub subtraction or with a a very short time one , and then , once you , actually had the whole utterance in , if you did , the , longer time version then , based on everything that you had , and then at that point only used it to distinguish between , you know , top N , possible utterances or something , you you might it might not take very much time .  , I know in the large vocabulary stu  , systems , people were evaluating on in the past , some people really pushed everything in to make it in one pass but other people didn't and had multiple passes . And , the argument , against multiple passes was u u has often been " but we want to this to be r you know have a nice interactive response " . And the counterargument to that which , say , BBN I think had , was " yeah , but our second responses are second , passes and third passes are really , really fast " . grad e: S so , the the idea of the second pass would be waiting till you have more recorded speech ? Or ? professor c: Yeah , so if it turned out to be a problem , that you didn't have enough speech because you need a longer longer window to do this processing , then , one tactic is you know , looking at the larger system and not just at the front - end stuff is to take in , the speech with some simpler mechanism or shorter time mechanism , grad e:   professor c: do the best you can , and come up with some al possible alternates of what might have been said . And , either in the form of an N - best list or in the form of a lattice , or or confusion network , or whatever . professor c: And then the decoding of that is much , much faster or can be much , much faster if it isn't a big bushy network . And you can decode that now with speech that you 've actually processed using this longer time , subtraction . professor c: So  , it 's it 's common that people do this sort of thing where they do more things that are more complex or require looking over more time , whatever , in some kind of second pass . professor c: and again , if the second pass is really , really fast  , another one I 've heard of is is in in connected digit stuff , going back and l and through backtrace and finding regions that are considered to be a d a digit , but , which have very low energy . professor c: So , there 's lots of things you can do in second passes , at all sorts of levels . phd a: So is that ,  that it ? grad e: I guess that 's it .  , so , the last two weeks was , like So I 've been working on that Wiener filtering . And , found that , s single like , I just do a s normal Wiener filtering , like the standard method of Wiener filtering . And that doesn't actually give me any improvement over like  , b it actually improves over the baseline but it 's not like it doesn't meet something like fifty percent or something . So , I 've been playing with the v phd a: Improves over the base line MFCC system ? Yeah . professor c: Is that using in combination with something else ? phd b: No , just just one stage Wiener filter professor c: With with a phd b: which is a standard Wiener filter . professor c: No , no , but  in combination with our on - line normalization or with the LDA ? phd b: Yeah , yeah , yeah , yeah . phd b: So , I di i di professor c: So , does it g does that mean it gets worse ? Or ? phd b: No . Like I have an LDA f LDA plus on - line normalization , and then I plug in the Wiener filter in that , professor c: Yeah ? phd b: so it improves over not having the Wiener filter . So it improves but it it doesn't take it like be beyond like thirty percent over the baseline . So professor c: But that 's what I 'm confused about , cuz I think I thought that our system was more like forty percent without the Wiener filtering . phd a: Is this with the v new VAD ? phd b: well , these are not No , it 's the old VAD . So my baseline was , nine This is like w the baseline is ninety - five point six eight , and eighty - nine , and professor c: So  , if you can do all these in word errors it 's a lot a lot easier actually . phd b: What was that ? Sorry ? professor c: If you do all these in word error rates it 's a lot easier , right ? phd b: Oh , OK , OK , OK . phd d: The baseline is something similar to a w  , the t the the baseline that you are talking about is the MFCC baseline , right ? phd b: The t yeah , there are two baselines . So the baseline One baseline is MFCC baseline that When I said thirty percent improvement it 's like MFCC baseline . professor c: So so so what 's it start on ? The MFCC baseline is is what ? Is at what level ? phd b: It 's the it 's just the mel frequency and that 's it . professor c: No , what 's what 's the number ? phd b: so I I don't have that number here . What 's ten point seven ? phd b: It 's a medium misma OK , sorry . phd b: So professor c: OK , four point three , ten point seven , phd b: And forty forty . professor c: And what were you just describing ? phd b: Oh , the one is this one is just the baseline plus the , Wiener filter plugged into it . professor c: But where 's the , on - line normalization and so on ? phd b: Oh , OK . So , with the with the on - line normalization , the performance was , ten OK , so it 's like four point three . So the h well matched has like literally not changed by adding on - line or LDA on it . professor c: OK , and what kind of number an and what are we talking about here ? phd b: It 's the It - it 's Italian . professor c: Is this TI - digits phd b: I 'm talking about Italian , professor c: or Italian ? phd b: yeah . professor c: And what did So , what was the , corresponding number , say , for , the Alcatel system for instance ? phd b: Mmm .  professor c: Do you know ? phd d: Yeah , so it looks to be ,  phd b: You have it ? phd d: Yep , it 's three point four , eight point , seven , and , thirteen point seven . phd b: So , this is the single stage Wiener filter , with The noise estimation was based on first ten frames . phd b: Actually I started with using the VAD to estimate the noise and then I found that it works it doesn't work for Finnish and Spanish because the VAD endpoints are not good to estimate the noise because it cuts into the speech sometimes , so I end up overestimating the noise and getting a worse result . phd b: So ,  so this was ,  And so this was giving  , this this was like not improving a lot on this baseline of not having the Wiener filter on it . And , so , I ran this stuff with one more stage of Wiener filtering on it but the second time , what I did was I estimated the new Wiener filter based on the cleaned up speech , and did , smoothing in the frequency to to reduce the variance professor c:   phd b: I have I 've I 've observed there are , like , a lot of bumps in the frequency when I do this Wiener filtering which is more like a musical noise or something . And so by adding another stage of Wiener filtering , the results on the SpeechDat - Car was like ,  So , I still don't have the word error rate . And the rest is like the LDA plu and the on - line normalization all remaining the same .  , so this was , like , compared to ,  Fifty - seven is what you got by using the French Telecom system , right ? phd d: No , I don't think so . phd b: Y i phd d: Is it on Italian ? phd b: No , this is over the whole SpeechDat - Car . phd b: Yeah , so the new the new Wiener filtering schema is like some fifty - six point four six which is like one percent still less than what you got using the French Telecom system . But again , you 're you 're more or less doing what they were doing , right ? phd b: It 's it 's different in a sense like I 'm actually cleaning up the cleaned up spectrum which they 're not doing . They 're d what they 're doing is , they have two stage stages of estimating the Wiener filter , but the final filter , what they do is they they take it to their time domain by doing an inverse Fourier transform . phd b: And they filter the original signal using that fil filter , professor c:  - huh . phd b: which is like final filter is acting on the input noisy speech rather than on the cleaned up . So this is more like I 'm doing Wiener filter twice , but the only thing is that the second time I 'm actually smoothing the filter and then cleaning up the cleaned up spectrum first level . phd b: And actually I tried it on s the original clean  , the original spectrum where , like , I the second time I estimate the filter but actually clean up the noisy speech rather the c s first output of the first stage and that doesn't seems to be a giving , that much improvement . And and what I t what I tried was , by using the same thing but  , so we actually found that the VAD is very , like , crucial . phd b: by instead of using the current VAD , if you just take up the VAD output from the channel zero , when instead of using channel zero and channel one , because that was the p that was the reason why I was not getting a lot of improvement for estimating the noise . So I just used the channel zero VAD to estimate the noise so that it gives me some reliable mar markers for this noise estimation . professor c: What 's a channel zero VAD ? phd b: professor c: I 'm I 'm confused about that . phd b: so , it 's like phd d: So it 's the close - talking microphone . phd b: Yeah , the close - talking without professor c: Oh , oh , oh , oh . phd b: So because the channel zero and channel one are like the same speech , but only w  , the same endpoints . professor c:  phd b: But the only thing is that the speech is very noisy for channel one , so you can actually use the output of the channel zero for channel one for the VAD .  , so a are they going to pro What are they doing to do , do we know yet ? about as far as what they 're what the rules are going to be and what we can use ? phd d: Yeah , so actually I received a a new document , describing this . phd b: Yeah , that 's phd d: And what they did finally is to , mmm , not to align the utterances but to perform recognition , only on the close - talking microphone , phd b: Which is the channel zero . professor c: So it 's not like that 's being done in one place or one time . phd d: And professor c: That 's that 's just a rule and we 'd you you were permitted to do that . Is is that it ? phd d: I think they will send , files but we we don't Well , apparently professor c: Oh , so they will send files so everybody will have the same boundaries to work with ? phd d: Yeah . phd d: Oh , i Yeah , so what happened here is that , the overall improvement that they have with this method So Well , to be more precise , what they have is , they have these alignments and then they drop the beginning silence and and the end silence but they keep , two hundred milliseconds before speech and two hundred after speech .  , and the overall improvement over the MFCC baseline So , when they just , add this frame dropping in addition it 's r  , forty percent , right ? professor c:   phd b: Yeah , which is phd d: which is , t which is the overall improvement . phd b: It gives like negative Well , in in like some Italian and TI - digits , phd d: Yeah , some @ @ . So by using the endpointed speech , actually it 's worse than the baseline in some instances , which could be due to the word pattern . professor c: Yeah , phd d: And Yeah , the other thing also is that fourteen percent is less than what you obtain using a real VAD . phd d: So  So I think this shows that there is still work  , well , working on the VAD is still still important I think . professor c: Yeah , c phd d:  phd a: Can I ask just a a high level question ? Can you just say like one or two sentences about Wiener filtering and why why are people doing that ? phd b:  . phd a: What 's what 's the deal with that ? phd b: OK , so the Wiener filter , it 's it 's like it 's like you try to minimize  , so the basic principle of Wiener filter is like you try to minimize the , d  , difference between the noisy signal and the clean signal if you have two channels . Like let 's say you have a clean t signal and you have an additional channel where you know what is the noisy signal . And you get you can do that  , if if you have only a c noisy signal , at a level which you , you w try to estimate the noise from the w assuming that the first few frames are noise or if you have a w voice activity detector , you estimate the noise spectrum . phd b: And then you phd a: Do you assume the noise is the same ? phd b: Yeah . phd b: So but that 's not the case in , many many of our cases but it works reasonably well . And then you do this  , this is the transfer function of the Wiener filter , so " SF " is a clean speech spectrum , power spectrum phd a:   professor c: Right phd b: And , professor c: actually , I guess phd b: Yeah . phd b: So but the thing is that you have to estimate the SF from the noisy spectrum , what you have . So you estimate the NF from the initial noise portions and then you subtract that from the current noisy spectrum to get an estimate of the SF . phd a: So is this , basically s  , similar to just regular spectral subtraction ? phd b: It professor c: It 's all pretty related , phd b: Yeah . It 's it 's there 's a di there 's a whole class of techniques where you try in some sense to minimize the noise . phd a: Do people use the Wiener filtering in combination with the spectral subtraction typically , or is i are they sort of competing techniques ? phd b: Not seen . phd b: So it 's like I haven't seen anybody using s Wiener filter with spectral subtraction . professor c: but y but there you make different approximations , and in spectral subtraction , for instance , there 's a a an estimation factor . professor c: You sometimes will figure out what the noise is and you 'll multiply that noise spectrum times some constant and subtract that rather than and sometimes people even though this really should be in the power domain , sometimes people s work in the magnitude domain because it it it works better . phd a: So why did you choose , Wiener filtering over some other one of these other techniques ? phd b: the reason was , like , we had this choice of using spectral subtraction , Wiener filtering , and there was one more thing which I which I 'm trying , is this sub space approach . phd b: So I picked up phd a: So you 're sort of trying @ @ them all . phd b: Y Yeah , phd a: Ah , phd b: we just wanted to have a few noise production compensation techniques phd a: I see . professor c: I m  yeah , there 's Car - Carmen 's working on another , on the vector Taylor series . professor c: So they were just kind of trying to cover a bunch of different things with this task and see , you know , what are what are the issues for each of them . phd b: So so one of one of the things that I tried , like I said , was to remove those zeros in the fri filter by doing some smoothing of the filter . phd b: Like , you estimate the edge of square and then you do a f smoothing across the frequency so that those zeros get , like , flattened out . So what I did was like I p did this and then you I plugged in the one more the same thing but with the smoothed filter the second time . phd b: So that 's where I got like fifty - six point five percent improvement on SpeechDat - Car with that . And So the other thing what I tried was I used still the ten frames of noise estimate but I used this channel zero VAD to drop the frames . And that has taken the performance to like sixty - seven percent in SpeechDat - Car , which is which which like sort of shows that by using a proper VAD you can just take it to further , better levels . phd a: So that 's sort of like , you know , best - case performance ? phd b: Yeah , so far I 've seen sixty - seven  , no , I haven't seen s like sixty - seven percent . And , using the channel zero VAD to estimate the noise also seems to be improving but I don't have the results for all the cases with that . So I used channel zero VAD to estimate noise as a lesser 2 x frame , which is like , everywhere I use the channel zero VAD . And that seems to be the best combination , rather than using a few frames to estimate and then drop a channel . phd b: so we c so  , which which means , like , by using this technique what we improve just the VAD professor c: Yes . And , w  Yeah , but this all these things , I have to still try it on the TI - digits , which is like I 'm just running . And there seems to be not improving a a lot on the TI - digits , so I 'm like investigating that , why it 's not . So ,  so the other the other thing is like I 've been I 'm doing all this stuff on the power spectrum . So Tried this stuff on the mel as well mel and the magnitude , and mel magnitude , and all those things . So , one of one of reasons I thought like doing the averaging , after the filtering using the mel filter bank , that seems to be maybe helping rather than trying it on the mel filter ba filtered outputs . phd b: Yeah , th that 's that 's the only thing that I could think of why why it 's giving improvement on the mel . professor c: how about the subspace stuff ? phd b: Subspace , I 'm I 'm like that 's still in a little bit in the back burner because I 've been p putting a lot effort on this to make it work , on tuning things and other stuff .  , So to r to remind you a little bit of of what I did before , is just to apply some spectral subtraction with an overestimation factor also to get , an estimate of the noise , spectrum , and subtract this estimation of the noise spectrum from the , signal spectrum , but subtracting more when the SNR is is , low , which is a technique that it 's often used . phd a: " Subtracting more " , meaning ? phd d: So you overestimate the noise spectrum . phd d: So , above twenty DB , it 's one , so you just subtract the noise . phd d: And then it 's b Generally Well , I use , actually , a linear , function of the SNR , phd a:   phd d: which is bounded to , like , two or three , when the SNR is below zero DB . phd d: doing just this , either on the FFT bins or on the mel bands , t doesn't yield any improvement professor c: Oh !  , what are you doing with negative , powers ? phd d: o Yeah . So there is also a threshold , of course , because after subtraction you can have negative energies , phd a:   phd d: and So what I I just do is to put ,  to to add to put the threshold first and then to add a small amount of noise , which right now is speech - shaped .  phd a: Speech - shaped ? phd d: Yeah , so it 's a it has the overall overall energy ,  pow it has the overall power spectrum of speech . phd a: So when y when you talk about there being something less than zero after subtracting the noise , is that at a particular frequency bin ? phd d: i  - huh . phd a: And so when you say you 're adding something that has the overall shape of speech , is that in a in a particular frequency bin ? Or you 're adding something across all the frequencies when you get these negatives ? phd d: For each frequencies I a I 'm adding some , noise , but the a the amount of the amount of noise I add is not the same for all the frequency bins . Right now I don't think if it makes sense to add something that 's speech - shaped , because then you have silence portion that have some spectra similar to the sp the overall speech spectra . So this is something I can still work on , phd a: So what does that mean ? phd d: but  . phd a: I 'm trying to understand what it means when you do the spectral subtraction and you get a negative . It means that at that particular frequency range you subtracted more energy than there was actually phd d: That means that   So so yeah , you have an an estimation of the noise spectrum , but sometimes , of course , it 's as the noise is not perfectly stationary , sometimes this estimation can be , too small , so you don't subtract enough . phd a: So in in an ideal word i world if the noise were always the same , then , when you subtracted it the worst that i you would get would be a zero .  , the lowest you would get would be a zero , cuz i if there was no other energy there you 're just subtracting exactly the noise . phd d:  -  , professor c: Yep , there 's all there 's all sorts of , deviations from the ideal here . professor c: for instance , you 're you 're talking about the signal and noise , at a particular point . And even if something is sort of stationary in ster terms of statistics , there 's no guarantee that any particular instantiation or piece of it is exactly a particular number or bounded by a particular range . professor c: So , you 're figuring out from some chunk of of of the signal what you think the noise is . professor c: and there 's absolutely no reason to think that you 'd know that it wouldn't , be negative in some places . professor c: on the other hand that just means that in some sense you 've made a mistake because you certainly have stra subtracted a bigger number than is due to the noise . professor c:  Also , we speak the whole where all this stuff comes from is from an assumption that signal and noise are uncorrelated . And that certainly makes sense in s in in a statistical interpretation , that , you know , over , all possible realizations that they 're uncorrelated phd a:   professor c: or assuming , ergodicity that i that i  , across time , it 's uncorrelated . But if you just look at a quarter second , and you cross - multiply the two things , you could very well , end up with something that sums to something that 's not zero . But if down the road you 're making use of something as if it is a power spectrum , then it can be bad to have something negative . Now , the other thing I wonder about actually is , what if you left it negative ? What happens ? phd b: Is that the log ? professor c: because  , are you taking the log before you add them up to the mel ? phd b: After that . So the thing is , I wonder how if you put your thresholds after that , I wonder how often you would end up with ,  with negative values . phd b: But you will But you end up reducing some neighboring frequency bins @ @ in the average , right ? When you add the negative to the positive value which is the true estimate . But nonetheless , you know , these are it 's another f kind of smoothing , right ? that you 're doing . So , you 've done your best shot at figuring out what the noise should be , and now i then you 've subtracted it off . And then after that , instead of instead of , leaving it as is and adding things adding up some neighbors , you artificially push it up . professor c: Which is , you know , it 's there 's no particular reason that that 's the right thing to do either , right ? phd b: Yeah , yeah . professor c: So , i in fact , what you 'd be doing is saying , " well , we 're d we 're we 're going to definitely diminish the effect of this frequency in this little frequency bin in the in the overall mel summation " . I d I don't know if it would be phd a: Sort of the opposite of that would be if if you find out you 're going to get a negative number , you don't do the subtraction for that bin . professor c: although phd a: That would be almost the opposite , right ? Instead of leaving it negative , you don't do it . If your if your subtraction 's going to result in a negative number , you you don't do subtraction in that . professor c: Yeah , but that means that in a situation where you thought that that the bin was almost entirely noise , you left it . phd d: And , yeah , some people also if it 's a negative value they , re - compute it using inter interpolation from the edges and bins . professor c: People can also , reflect it back up and essentially do a full wave rectification instead of a instead of half wave . Well , actually I tried , something else based on this , is to to put some smoothing , because it seems to to help or it seems to help the Wiener filtering professor c:   Actually , when you do spectral subtraction you can , find this this equivalent in the s in the spectral domain . You can  compute , y you can say that d your spectral subtraction is a filter , and the gain of this filter is the , signal energy minus what you subtract , divided by the signal energy . And this is a gain that varies over time , and , you know , of course , depending on the s on the noise spectrum and on the speech spectrum . And what happen actually is that during low SNR values , the gain is close to zero but it varies a lot . Mmm , and this this is the cause of musical noise and all these the the fact you we go below zero one frame and then you can have an energy that 's above zero . But it 's the smoothing is nonlinear in the sense that I tried to not smooth if the gain is high , because in this case we know that , the estimate of the gain is correct because we we are not close to to to zero , and to do more smoothing if the gain is low . So , well , basically that 's this idea , and it seems to give pretty good results , although I 've just just tested on Italian and Finnish . And on Italian it seems my result seems to be a little bit better than the Wiener filtering , phd b:   phd d: I don't know if you have these improvement the detailed improvements for Italian , Finnish , and Spanish there phd b: Fff . No , I don't have , for each , phd d: or you have just have your own . professor c: So these numbers he was giving before with the four point three , and the ten point one , and so forth , those were Italian , right ? phd b: Yeah , yeah , yeah . phd d:  phd b: I actually didn't give you the number which is the final one , phd d: no , we 've phd b: which is , after two stages of Wiener filtering . phd d:  professor c: But do you have numbers in terms of word error rates on on Italian ? So just so you have some sense of reference ? phd d: Yeah . professor c: And this is , spectral subtraction plus what ? phd d: Plus plus nonlinear smoothing . Well , it 's the system it 's exactly the sys the same system as Sunil tried , professor c: On - line normalization and LDA ? phd d: but professor c: Yeah . But instead of double stage Wiener filtering , it 's it 's this smoothed spectral subtraction . phd a: for Do they use spectral subtraction , or Wiener filtering , or ? phd b: They use spectral subtraction , right . phd d: It it 's Wiener filtering , phd b: Oh , it 's it 's Wiener filtering . professor c: Yeah , plus , I guess they have some sort of cepstral normalization , as well . phd b: s They have like yeah , th the just noise compensation technique is a variant of Wiener filtering , phd d:   phd d: One in the time domain and one in the frequency domain by just taking the first , coefficients of the impulse response .  , what you did , it 's similar phd b: It 's similar in the smoothing and phd d: because you have also two two kind of smoothing . phd d: One in the time domain , and one in the frequency domain , phd b: Yeah . phd a: Does the smoothing in the time domain help phd d:  phd a: Well , do you get this musical noise stuff with Wiener filtering or is that only with , spectral subtraction ? phd b: No , you get it with Wiener filtering also . phd a: Does the smoothing in the time domain help with that ? Or some other smoothing ? phd b: Oh , no , you still end up with zeros in the s spectrum . phd b: the phd d: Well , I cannot you cannot hear beca well , actually what I d did not say is that this is not in the FFT bins .  So , it could be seen as a f a a smoothing in the frequency domain because I used , in ad mel bands in addition and then the other phase of smoothing in the time domain . But , when you look at the spectrogram , if you don't have an any smoothing , you clearly see , like in silence portions , and at the beginning and end of speech , you see spots of high energy randomly distributed over the the spectrogram . phd d:  phd a: That 's the musical noise ? phd d: Which is musical noise , phd a:   phd d: yeah , if if it If you listen to it  , if you do this in the FFT bins , then you have spots of energy randomly distributing . And if you f if you re - synthesize these spot sounds as , like , sounds , phd a:   phd d:  professor c: Well , none of these systems , by the way , have  , y you both are are working with , our system that does not have the neural net , phd d: And phd b: Yep . So one would hope , presumably , that the neural net part of it would would improve things further as as they did before .  Yeah , although if if we , look at the result from the proposals , one of the reason , the n system with the neural net was , more than well , around five percent better , is that it was much better on highly mismatched condition . I 'm thinking , for instance , on the TI - digits trained on clean speech and tested on noisy speech . phd d: And if we have no , spectral subtraction or Wiener filtering , i the system is  , we thought the neural neural network is much better than before , even in these cases of high mismatch . phd a: Could you train a neural net to do spectral subtraction ? professor c: Yeah , it could do a nonlinear spectral subtraction phd d:   professor c: but I don't know if it  , you have to figure out what your targets are . phd a: Yeah , I was thinking if you had a clean version of the signal and and a noisy version , and your targets were the M F -  , you know , whatever , frequency bins phd d:   professor c: Yeah , well , that 's not so much spectral subtraction then , phd d:   professor c: but but but it 's but at any rate , yeah , people ,  phd a: People do that ? professor c: y yeah , in fact , we had visitors here who did that I think when you were here ba way back when . professor c: The objection everyone always raises , which has some truth to it is that , it 's good for mapping from a particular noise to clean but then you get a different noise . professor c: And the experiments we saw that visitors did here showed that it there was at least some , gentleness to the degradation when you switched to different noises . phd a: How did it compare on  , for for good cases where it it  , stuff that it was trained on ? Did it do pretty well ? professor c: Oh , yeah , it did very well .  , we 're not doing exactly that , we 're not trying to generate good examples but by trying to do the best classifier you possibly can , for these little phonetic categories , phd a: it we 'll just have to see whether it helps more or less the same , but I would imagine it would help some . professor c: So in any event , all of this I was just confirming that all of this was with a simpler system .  , Yeah , so this is th the ,  Well , actually , this was kind of the first try with this spectral subtraction plus smoothing , professor c:   And it seems that the one that I chose for the first experiment was the optimal one , so  , professor c: It 's amazing how often that happens .  Yeah , another thing that I it 's important to mention is , that this has a this has some additional latency . Because when I do the smoothing , it 's a recursion that estimated the means , so of the g of the gain curve . So , instead o of using the current estimated mean to , subtract the current frame , it 's better to use an estimate that 's some somewhere in the future . phd b: You mean , the m the mean is computed o based on some frames in the future also ? professor c:   phd b: Or or no ? phd d: It 's the recursion , so it 's it 's the center recursion , right ? phd b:   professor c: One five ? phd d:  professor c: One five ? Five zero ? phd d: Five zero , professor c: Five zero . phd b: why why is that delay coming ? Like , you estimate the mean ? phd d: Yeah , the mean estimation has some delay , right ? phd b: Oh , yeah . phd b: It isn't OK , so it 's like it looks into the future also . professor c: What if you just look into the past ? phd d: It 's , not as good . professor c: How m by how much ? phd d: it helps a lot over the ba the baseline but , mmm professor c: By how much ? phd d: it It 's around three percent , relative . phd d: mmm So ,  professor c: It 's depending on how all this stuff comes out we may or may not be able to add any latency . Yeah , b but I don't think we have to worry too much on that right now while you kno . professor c: s Yeah , I think the only thing is that phd d: So professor c: I would worry about it a little . professor c: Because if we completely ignore latency , and then we discover that we really have to do something about it , we 're going to be find ourselves in a bind . professor c: Yeah , just , you know , just be be a little conservative phd d: Oh yes . professor c: because we may end up with this crunch where all of a sudden we have to cut the latency in half or something . So , yeah , there are other things in the , algorithm that I didn't , @ @ a lot yet , phd a: Oh ! phd d: which phd a: Sorry . If if there 's another part of the system that causes a latency of a hundred milliseconds , is this an additive thing ? Or c or is yours hidden in that ? phd d:   We can do something in parallel also , in some like some cases like , if you wanted to do voice activity detection . phd b: So you can make a decision on that voice activity detection and then you decide whether you want to filter or not . phd a: couldn't ,  I Couldn't you just also  , i if you know that the l the largest latency in the system is two hundred milliseconds , don't you couldn't you just buffer up that number of frames and then everything uses that buffer ? phd b: Yeah . phd a: And that way it 's not additive ? professor c: Well , in fact , everything is sent over in buffers cuz of isn't it the TCP buffer some ? phd b: You mean , the the data , the super frame or something ? phd d:   phd b: Yeah , but that has a variable latency because the last frame doesn't have any latency phd d:   phd b: So But the only thing is that the first frame in that twenty - four frame buffer has a twenty - four frame latency . phd b: Because it just goes as phd a: Yeah , I wasn't thinking of that one in particular phd b: Yeah . phd a: but more of , you know , if if there is some part of your system that has to buffer twenty frames , can't the other parts of the system draw out of that buffer and therefore not add to the latency ? professor c: Yeah . And and that 's sort of one of the all of that sort of stuff is things that they 're debating in their standards committee . Like , I played a little bit with this overestimation factor , but I still have to to look more at this , at the level of noise I add after .  , I know that adding noise helped , the system just using spectral subtraction without smoothing , but I don't know right now if it 's still important or not , and if the level I choose before is still the right one .  , and another thing is to Yeah , for this I just use as noise estimate the mean , spectrum of the first twenty frames of each utterance . I don't remember for this experiment what did you use for these two stage phd b: I used ten just ten frames . Yeah , because phd d: The ten frames ? phd b: the reason was like in TI - digits I don't have a lot . But , so what 's this result you told me about , the fact that if you use more than ten frames you can improve by t phd b: Well , that 's that 's using the channel zero . phd b: Which phd d: But this is ten frames plus plus phd b: Channel zero dropping . phd d: no , these results with two stage Wiener filtering is ten frames phd b: t Oh , this phd d: but possibly more . The second thing I was working on is to , try to look at noise estimation , mmm , and using some technique that doesn't need voice activity detection .  , and for this I u simply used some code that , I had from from Belgium , which is technique that , takes a bunch of frame , and for each frequency bands of this frame , takes a look at the minima of the energy . And then average these minima and take this as an an energy estimate of the noise for this particular frequency band . So , I compute an FFT based on the long , signal frame which is sixty - four millisecond phd a: So you have one minimum for each frequency ? phd d: What what I what I d  , I do actually , is to take a bunch of to take a tile on the spectrogram and this tile is five hundred milliseconds long and two hundred hertz wide . phd d: And this tile  , in this tile appears , like , the harmonics if you have a voiced sound , because it 's it 's the FTT bins . And when you take the m the minima of of these this tile , when you don't have speech , these minima will give you some noise level estimate , If you have voiced speech , these minima will still give you some noise estimate because the minima are between the harmonics . And If you have other other kind of speech sounds then it 's not the case , but if the time frame is long enough , like s five hundred milliseconds seems to be long enough , you still have portions which , are very close whi which minima are very close to the noise energy . You said five hundred milliseconds phd d: Mmm ? professor c: but you said sixty - four milliseconds . Which is which ? What ? phd d: Sixty - four milliseconds is to compute the FFT , bins . phd d: actually it 's better to use sixty - four milliseconds because , if you use thirty milliseconds , then , because of the this short windowing and at low pitch , sounds , the harmonics are not , wha  , correctly separated . phd d: So if you take these minima , it b they will overestimate the noise a lot . professor c: So you take sixty - four millisecond F F Ts and then you average them over five hundred ? Or ?  , what do you do over five hundred ? phd d: So I take to I take a bunch of these sixty - four millisecond frame to cover five hundred milliseconds , professor c: Ah . phd d: on the on on the bunch of  fifty frames , right ? professor c: I see . So the interest of this is that , as y with this technique you can estimate u some reasonable noise spectra with only five hundred milliseconds of of signal , so if the the n the noise varies a lot , you can track better track the noise , professor c:   The only requirement is that you must have , in these five hundred milliseconds segment , you must have voiced sound at least . So what I did is just to simply replace the VAD - based , noise estimate by this estimate , first on SpeechDat - Car Well , only on SpeechDat - Car actually . And it 's , slightly worse , like one percent relative compared to the VAD - based estimates .  , I think the reason why it 's not better , is that the SpeechDat - Car noises are all stationary . So , u y y there really is no need to have something that 's adaptive professor c:   But , I expect s maybe some improvement on TI - digits because , nnn , in this case the noises are all sometimes very variable . professor c: But are you comparing with something e I 'm I 'm p s a little confused again , i it  , when you compare it with the V A D - based , phd d:   professor c: VAD - Is this is this the ? phd d: It 's It 's the France - Telecom - based spectra , s  , Wiener filtering and VAD . professor c: Oh , you 're not doing this with our system ? phd d: In i I 'm not No , no . Actually , th the best system that we still have is , our system but with their noise compensation scheme , right ? professor c: Right . But phd d: So I 'm trying to improve on this , and by by replacing their noise estimate by , something that might be better . professor c: Couldn't you try this for that ? phd d: But I di professor c: Do you think it might help ? phd d: Not yet , because I did this in parallel , professor c: I see , phd d: and I was working on one and the other . phd b: So I 'm also using that n new noise estimate technique on this Wiener filtering what I 'm trying . phd b: So I I have , like , some experiments running , I don't have the results .  , the idea is just to  , flatten the log , spectrum , and to flatten it more if the the probability of silence is higher . So in this way , you can also reduce somewhat reduce the musical noise and you reduce the variability if you have different noise shapes , because the the spectrum becomes more flat in the silence portions . With this , no improvement , but there are a lot of parameters that we can play with and ,  Actually , this this could be seen as a soft version of the frame dropping because , you could just put the threshold and say that " below the threshold , I will flatten comp completely flatten the the spectrum " . So it would be like frame dropping , because during the silence portions which are below the threshold of voice activity probability , w you would have some kind of dummy frame which is a perfectly flat spectrum . And this , whitening is something that 's more soft because , you whiten you just , have a function the whitening is a function of the speech probability , so it 's not a hard decision . phd d: so I think maybe it can be used together with frame dropping and when we are not sure about if it 's speech or silence , well , maybe it has something do with this .  , you know , in in JRASTA we were essentially adding in , white  , white noise dependent on our estimate of the noise . professor c: You could imagine one that that that made use of where where the amount that you added in was , a function of the probability of it being s speech or noise . Yeah , w Yeah , right now it 's a constant that just depending on the the noise spectrum . professor c: Cuz that that brings in sort of powers of classifiers that we don't really have in , this other estimate . professor c: What what what point does the , system stop recording ? How much phd a: It 'll keep going till I guess when they run out of disk space , professor c: It went a little long ?  , disk phd a: but I think we 're OK .  Yeah , so there are with this technique there are some I just did something exactly the same as as the Ericsson proposal but , the probability of speech is not computed the same way . And I think , i for yeah , for a lot of things , actually a g a good speech probability is important . Like for frame dropping you improve , like you can improve from ten percent as Sunil showed , if you use the channel zero speech probabilities .  , so yeah , the next thing I started to do is to , try to develop a better voice activity detector . And ,  I d  yeah , for this I think we can maybe try to train the neural network for voice activity detection on all the data that we have , including all the SpeechDat - Car data .  , and the way I mi I do that is that I just use the HTK system but I train it only on the close - talking microphone .  It seems to be ,  i Actually what I observed is that for Italian it doesn't seem Th - there seems to be a problem . Because What ? phd b: you mean their d the frame dropping , right ? Yeah , it doesn't phd d: Yeah . phd d: so  , the c the current VAD that we have was trained on , t SPINE , right ? phd b: TI - digits . And actually we observed we listened to some of the utterances and sometimes for Finnish there is music in the recordings and strange things , right ? phd b: Yeah . phd d:  Yeah , so the idea was to train all the databases and obtain an alignment to train on these databases , and , also to , try different kind of features , as input to the VAD network . And we came up with a bunch of features that we want to try like , the spectral slope , the , the degree o degree of voicing with the features that , we started to develop with Carmen , e with , the correlation between bands and different kind of features , phd b: Yeah . Well , Hans - Guenter will be here next week so I think he 'll be interested in all all of these things <doc-sep>Actually , I think he he redirected it to everybody also so  the PDA mikes  have a big bunch of energy at at  five hertz  where this came up was that  I was showing off these wave forms that we have on the web and and  I just sort of hadn't noticed this , but that the major , major component in the wave in the second wave form in that pair of wave forms is actually the air conditioner . I I have to be more careful about using that as a as a as a good illustration , in fact it 's not , of  of the effects of room reverberation . And then we had this other discussion about  whether this affects the dynamic range , cuz I know , although we start off with thirty two bits , you end up with  sixteen bits and you know , are we getting hurt there ? But  Dan is pretty confident that we 're not , that that quantization error is not is still not a significant factor there . So there was a question of whether we should change things here , whether we should change a capacitor on the input box for that or whether we should phd b: Yeah , he suggested a smaller capacitor , right ? professor a: Right . But then I had some other  thing discussions with him phd b: For the P D professor a: and the feeling was once we start monk monkeying with that , many other problems could ha happen . professor a: A simple thing to do is he he he has a I forget if it this was in that mail or in the following mail , but he has a a simple filter , a digital filter that he suggested . professor a:  The other thing that I don't know the answer to , but when people are using Feacalc here ,  whether they 're using it with the high - pass filter option or not . So when we 're doing all these things using our software there is  if it 's if it 's based on the RASTA - PLP program , which does both PLP and RASTA - PLP  then  there is an option there which then comes up through to Feacalc which  allows you to do high - pass filtering and in general we like to do that , because of things like this and it 's it 's pretty it 's not a very severe filter . Doesn't affect speech frequencies , even pretty low speech frequencies , at all , but it 's phd b: What 's the cut - off frequency it used ? professor a: Oh . I don't know I wrote this a while ago phd b: Is it like twenty ? professor a: Something like that .  I think there 's some effect above twenty but it 's it 's it 's it 's mild . So ,  it probably there 's probably some effect up to a hundred hertz or something but it 's it 's pretty mild . I don't know in the in the STRUT implementation of the stuff is there a high - pass filter or a pre pre - emphasis or something in the phd f:  . We we we want to go and check that in i for anything that we 're going to use the P D A mike for .  He says that there 's a pretty good roll off in the PZM mikes so we don't need need to worry about them one way or the other but if we do make use of the cheap mikes ,  we want to be sure to do that that filtering before we process it . And then again if it 's  depending on the option that the our our software is being run with , it 's it 's quite possible that 's already being taken care of .  the thing is it was since I was talking about reverberation and showing this thing that was noise , it wasn't a good match , but it certainly was still  an indication of the fact that you get noise with distant mikes .  It 's just not a great example because not only isn't it reverberation but it 's a noise that we definitely know what to do . professor a: So , it doesn't take deep a new bold new methods to get rid of  five hertz noise , so . So it was it was a bad example in that way , but it 's it still is it 's the real thing that we did get out of the microphone at distance , so it wasn't it w it w wasn't wrong it was inappropriate . So  , but  , Yeah , someone noticed it later pointed it out to me , and I went " oh , man .  So I think we 'll change our our picture on the web , when we 're @ @ . One of the things I was  , I was trying to think about what what 's the best way to show the difference an and I had a couple of thoughts one was , that spectrogram that we show is O K , but the thing is the eyes  and the the brain behind them are so good at picking out patterns from from noise that in first glance you look at them it doesn't seem like it 's that bad  because there 's many features that are still preserved . So one thing to do might be to just take a piece of the spec  of the spectrogram where you can see that something looks different , an and blow it up , and have that be the part that 's just to show as well .  Another , I was thinking of was  taking some spectral slices , like  like we look at with the recognizer , and look at the spectrum or cepstrum that you get out of there , and the the  , the reverberation  does make it does change that . grad c: W w what d what do you mean ? professor a: Well ,   all the recognizers look at frames . professor a: So it 's , yeah , at one point in time or  twenty over twenty milliseconds or something , you have a spectrum or a cepstrum . And if you look at phd b: You could just you could just throw up , you know ,  the  some MFCC feature vectors . You know , one from one , one from the other , and then , you know , you can look and see how different the numbers are . Well , that 's why I saying either Well , either spectrum or cepstrum phd b: I 'm just kidding . professor a: but but I think the thing is you wanna  phd b: I don't mean a graph . Yeah , at first I had a remark why I am wondering why the PDA is always so far . Since the last meeting we 've we 've tried to put together  the clean low - pass  downsampling , upsampling ,  the new filter that 's replacing the LDA filters , and also the  delay issue so that We considered th the the delay issue on the for the on - line normalization . professor a: But it 's not wer worse and it 's better better latency , phd f: It 's not professor a: right ? phd f: Yeah . It seems better when we look at the mismatched case but I think we are like like cheated here by the th this problem that  in some cases when you modify slight slightly modify the initial condition you end up completely somewhere air somewhere else in the in the space , the parameters . For Italian is at seventy - eight percent recognition rate on the mismatch , and this new system has eighty - nine . I don't I don't think it means that the new system is more robust professor a:  - huh . professor a: Well , the test would be if you then tried it on one of the other test sets , if if it was phd f: Y professor a: Right . professor a: So then if you take your changes phd f: It 's similar for other test sets professor a: and then phd f: but  from this se seventy - eight  percent recognition rate system , I could change the transition probabilities for the the first  and it will end up to eighty - nine also . phd f: By using point five instead of point six , point four as in the the HTK script . phd b:  th the only difference is you change the self - loop transition probability by a tenth of a percent phd f: Yeah . From point phd f: Even tenth of a percent ? phd b: I I 'm sorry phd f: Well , we tried we tried point one , phd b: f for point from You change at point one phd f: yeah . professor a: Oh ! phd b: and n not tenth of a percent , one tenth , phd f:  . phd b: alright ?  so from point five so from point six to point five and you get ten percent better . phd b: And it 's I think it 's what you basically hypothesized in the last meeting about  it just being very phd f:   phd b: and I think you mentioned this in your email too it 's just very  phd f: Mmm , yeah . phd b: you know get stuck in some local minimum and this thing throws you out of it I guess . professor a: Well , what 's what are according to the rules what what are we supposed to do about the transition probabilities ? Are they supposed to be point five or point six ? phd b: I think you 're not allowed to Yeah . But changing it to point five I think is which gives you much better results , but that 's not allowed . phd f: Yeah , but even if you use point five , I 'm not sure it will always give you the better results phd b: Yeah . We only tested it on the the medium mismatch , phd f: on the other training set ,  . phd b: right ? You said on the other cases you didn't notice phd f: Yeah . I think the reason is , yeah , I not I it was in my mail I think also , is the fact that the mismatch is trained only on the far microphone . Well , in for the mismatched case everything is  using the far microphone training and testing , whereas for the highly mismatched , training is done on the close microphone so it 's it 's clean speech basically so you don't have this problem of local minima probably and for the well - match , it 's a mix of close microphone and distant microphone and Well . phd b: I did notice  something phd f: So th I think the mismatch is the more difficult for the training part . phd b: Somebody , I think it was Morgan , suggested at the last meeting that I actually count to see how many parameters and how many frames . phd b: And there are  almost one point eight million frames of training data and less than forty thousand parameters in the baseline system . phd b: I did one quick experiment just to make sure I had everything worked out and I just  f for most of the  For for all of the digit models , they end up at three mixtures per state . And so I just did a quick experiment , where I changed it so it went to four and  it it it didn't have a r any significant effect at the  medium mismatch and high mismatch cases and it had it was just barely significant for the well - matched better . And I think also just seeing what we saw  in terms of the expected duration of the silence model ? when we did this tweaking of the self - loop ? The silence model expected duration was really different . phd b: And so in the case where  it had a better score , the silence model expected duration was much longer . I think you know if we make a better silence model I think that will help a lot too  for a lot of these cases so but one one thing I I wanted to check out before I increased the  number of mixtures per state was  in their default training script they do an initial set of three re - estimations and then they built the silence model and then they do seven iterations then the add mixtures and they do another seven then they add mixtures then they do a final set of seven and they quit . Seven seems like a lot to me and it also makes the experiments go take a really long time  to do one turn - around of the well matched case takes like a day . phd b: And so you know in trying to run these experiments I notice , you know , it 's difficult to find machines , you know , compute the run on . And so one of the things I did was I compiled HTK for the Linux machines professor a:   phd b: cuz we have this one from IBM that 's got like five processors in it ? professor a: Right . phd b: and so now I 'm you can run stuff on that and that really helps a lot because now we 've got you know , extra machines that we can use for compute . And if I 'm do running an experiment right now where I 'm changing the number of iterations ? from seven to three ? phd d:   And so if we can get away with just doing three , we can do many more experiments more quickly . And if it 's not a a huge difference from running with seven iterations , you know , we should be able to get a lot more experiments done . But if we can you know , run all of these back - ends f with many fewer iterations and on Linux boxes we should be able to get a lot more experimenting done . So I wanted to experiment with cutting down the number of iterations before I increased the number of Gaussians . professor a: But they 're not making things worse and we have reduced latency , right ? phd f: Yeah . But actually  actually it seems to do a little bit worse for the well - matched case and we just noticed that Yeah , actually the way the final score is computed is quite funny . It 's not a weighted mean of word error rate , it 's a weighted mean of improvements . Which means that actually the weight on the well - matched is Well I well what what What happened is that if you have a small improvement or a small if on the well - matched case it will have  huge influence on the improvement compared to the reference because the reference system is is is quite good for for the well - ma well - matched case also . phd b: So it it weights the improvement on the well - matched case really heavily compared to the improvement on the other cases ? phd f: No , but it 's the weighting of the of the improvement not of the error rate . Yeah , and it 's hard to improve on the on the best case , cuz it 's already so good , right ? phd f: Yeah but what  is that you can have a huge improvement on the H HMK 's ,  like five percent  absolute , and this will not affect the final score almost  this will almost not affect the final score because this improvement because the improvement  relative to the the baseline is small professor a: So they do improvement in terms of  accuracy ? rather than word error rate ? phd f:  .  improvement ? professor a: So phd f: No , it 's compared to the word er it 's improvement on the word error rate , professor a: OK . professor a: So if you have  ten percent error and you get five percent absolute  improvement then that 's fifty percent . So what you 're saying then is that if it 's something that has a small word error rate , then  a even a relatively small improvement on it , in absolute terms , will show up as quite quite large in this . Sure , but when we think about the weighting , which is point five , point three , point two , it 's on absolute on on relative figures , professor a: Yeah . That 's why I 've been saying we should be looking at word error rate  and and not not at at accuracies . phd b: it 's not it 's not that different , right ?  , just subtract the accuracy . professor a: Yeah but you 're but when you look at the numbers , your sense of the relative size of things is quite different . professor a: If you had ninety percent  correct and five percent , five over ninety doesn't look like it 's a big difference , but five over ten is is big . professor a: So just when we were looking at a lot of numbers and getting sense of what was important . professor a: What 's a little bit ? Like phd f: Like , it 's difficult to say because again  I 'm not sure I have the  phd b: Hey Morgan ? Do you remember that Signif program that we used to use for testing signi ? Is that still valid ? I I 've been using that . I just use my old one from ninety - two or whatever professor a: Yeah , I 'm sure it 's not that different but but he he  he was a little more rigorous , as I recall . s phd f:  well we start from ninety - four point sixty - four , and we go to ninety - four point O four . For Finnish , we start to ninety - three point eight - four and we go to ninety - three point seventy - four . And for Spanish we are we were at ninety - five point O five and we go to ninety - three - s point sixty one . professor a: And is that wh what do you know what piece you 've done several changes here . Because nnn , well  we don't have complete result , but the filter So the filter with the shorter delay hurts on Italian well - matched , which And , yeah . And the other things , like  downsampling , upsampling , don't seem to hurt and the new on - line normalization , neither . If we saw that making a small change like , you know , a tenth , to the self - loop had a huge effect , can we really make any conclusions about differences in this stuff ? phd f:   phd f: I think we can be completely fooled by this thing , but I don't know . There is first this thing , and then the yeah , I computed the  like , the confidence level on the different test sets . professor a: But OK , so you these these degradations you were talking about were on the well - matched case phd f: So . Do the does the new filter make things  better or worse for the other cases ? phd f: Yeah . OK , so  I guess the argument one might make is that , " Yeah , if you looked at one of these cases and you jiggle something and it changes then  you 're not quite sure what to make of it . But when you look across a bunch of these and there 's some some pattern , so eh h here 's all the if if in all these different cases it never gets better , and there 's significant number of cases where it gets worse , then you 're probably hurting things , I would say . So   at the very least that would be a reasonably prediction of what would happen with with a different test set , that you 're not jiggling things with . What I was asking , though , is  are what 's what 's the level of communication with  the O G I gang now , about this and phd f: Well , we are exchanging mail as soon as we we have significant results . We are working on our side on other things like  also trying a sup spectral subtraction but of of our own , another spectral substraction . It 's going professor a: Is there any further discussion about this this idea of of having some sort of source code control ? phd f: Yeah . As soon as we have something that 's significant and that 's better than than what was submitted , we will fix fix the system and But we 've not discussed it it it this yet , yeah . Sounds like a great idea but but I think that that  he 's saying people are sort of scrambling for a Eurospeech deadline . Anybo - anybody in the in this group do doing anything for Eurospeech ? phd f: S professor a: Or , is that what is that phd f: Yeah we are We are trying to to do something with the Meeting Recorder digits , professor a: Right . phd f: and , well , some people from OGI are working on a paper for this , but there is also the  special session about th Aurora which is  which has an extended deadline . professor a: So the deadline When 's the deadline ? When 's the deadline ? phd f:  ? I think it 's the thirteenth of May . So , I I think that you could certainly start looking at at the issue  but but  I think it 's probably , on s from what Stephane is saying , it 's it 's unlikely to get sort of active participation from the two sides until after they 've phd b: Well I could at least Well , I 'm going to be out next week but I could try to look into like this  CVS over the web . That seems to be a very popular way of people distributing changes and over , you know , multiple sites and things professor a:   phd b: so maybe if I can figure out how do that easily and then pass the information on to everybody so that it 's you know , as easy to do as possible and and people don't it won't interfere with their regular work , then maybe that would be good . And if you 're interested in using CVS , I 've set it up here , phd b: Oh great . grad c:  j phd b: I used it a long time ago but it 's been a while so maybe I can ask you some questions . Maybe you and I can talk a little bit at some point about coming up with a better  demonstration of the effects of reverberation for our web page , cuz  the    , actually the the  It made a good good audio demonstration because when we could play that clip the the the really obvious difference is that you can hear two voices and in the second one and only hear phd b: Maybe we could just like , talk into a cup . professor a: No , it sound it sounds pretty reverberant , but  you can't when you play it back in a room with a you know a big room , nobody can hear that difference really . professor a: They hear that it 's lower amplitude and they hear there 's a second voice , grad c:  - huh . professor a:  but  that actually that makes for a perfectly good demo because that 's a real obvious thing , that you hear two voices . But for the the visual , just , you know , I 'd like to have   , you know , the spectrogram again , grad c: Yeah . professor a: because you 're you 're you 're visual  abilities as a human being are so good you can pick out you know , you you look at the good one , you look at the cru the screwed up one , and and you can see the features in it without trying to @ @ phd b: I noticed that in the pictures . phd b: I thought " hey , you know th " I My initial thought was " this is not too bad ! " professor a: Right . But you have to you know , if you look at it closely , you see " well , here 's a place where this one has a big formant   formant maj major formants here are are moving quite a bit . professor a: So  you could that 's why I was thinking , in a section like that , you could take a look look at just that part of the spectrogram and you could say " Oh yeah . The main thing that struck me in looking at those two spectrograms was the difference in the high frequencies . It looked like for the one that was farther away , you know , it really everything was attenuated professor a: Right . Since you 're getting all this indirect energy , then a lot of it does have have  reduced high frequencies . But  the other thing is the temporal courses of things really are changed , and and  we want to show that , in some obvious way . " but I I just  After after  they were put in there I didn't really look at them anymore , cuz I just they were different . So maybe we can just substitute one of these wave forms and  then do some kind of zoom in on the spectrogram on an interesting area . professor a: The other thing that we had in there that I didn't like was that  the most obvious characteristic of the difference  when you listen to it is that there 's a second voice , and the the the the the  cuts that we have there actually don't correspond to the full wave form . It 's just the first I think there was something where he was having some trouble getting so much in , or . But it it 's  it 's the first six seconds or something of it and it 's in the seventh or eighth second or something where @ @ the second voice comes in . So we we would like to actually see the voice coming in , too , I think , since that 's the most obvious thing when you listen to it . phd f: So the thing that we did is just to add spectral subtraction before this , the Wall  process , which contains LDA on - line normalization . phd f: And so we started to look at at  things like this , which is , well , it 's Yeah . And the sentence contain only one word , which is " Due " And it can't clearly be seen . phd f: Where is the word ? phd b: This is this is , grad e:  . phd f: This is a plot of C - zero ,  when we don't use spectral substraction , and when there is no on - line normalization . phd f:  then when we apply mean normalization it looks like the second figure , though it is not . phd f: and And then the third figure is what happens when we apply mean normalization and variance normalization . What we can clearly see is that on the speech portion the two channel come becomes very close , but also what happens on the noisy portion is that the variance of the noise is professor a:   phd b: Can I ask  what does variance normalization do ? w What is the effect of that ? professor a: Normalizes the variance . phd f: So it phd b: No , I understand that , phd f: You you get an estimate of the standard deviation . phd f:  phd b: No , I understand what it is , but  , what does it what 's what is phd f: Yeah but . professor a: Well , because everything  If you have a system based on Gaussians , everything is based on means and variances . professor a: So if there 's an overall reason You know , it 's like  if you were doing  image processing and in some of the pictures you were looking at ,  there was a lot of light  and and in some , there was low light , phd b:   professor a: And the variance is just sort of like the next moment , you know ? So  what if  one set of pictures was taken  so that throughout the course it was went through daylight and night    ten times , another time it went thr  i is , you know , how how much how much vari phd b: Oh , OK . I guess a better example would be how much of the light was coming in from outside rather than artificial light . So if it was a lot if more was coming from outside , then there 'd be the bigger effect of the of the of the change in the So every mean every all all of the the parameters that you have , especially the variances , are going to be affected by the overall variance . professor a: And so , in principle , you if you remove that source , then , you know , you can phd b: I see . So would the major effect is that you 're gonna get is by normalizing the means , professor a: That 's the first order but thing , phd b: but it may help First - order effects . professor a: but then the second order is is the variances phd b: And it may help to do the variance . professor a: because , again , if you if you 're trying to distinguish between E and B phd b: OK . professor a: if it just so happens that the E 's were a more you know , were recorded when when the energy was was was larger or something , phd b:    professor a: or the variation in it was larger ,  than with the B 's , then this will be give you some some bias . phd b:  professor a: So the it 's removing these sources of variability in the data that have nothing to do with the linguistic component . And it and this professor a: i is if If you have a good voice activity detector , isn't isn't it gonna pull that out ? phd f: Yeah . Well what it it shows is that , yeah , perhaps a good voice activity detector is is good before on - line normalization and that 's what  we 've already observed . phd b: I don't know , it seems like this would be a lot easier than this signal to work with . What I notice is that , while I prefer to look at the second figure than at the third one , well , because you clearly see where speech is . phd f: But the problem is that on the speech portion , channel zero and channel one are more different than when you use variance normalization where channel zero and channel one become closer . phd b: But for the purposes of finding the speech phd f: And Yeah , but here phd b: You 're more interested in the difference between the speech and the nonspeech , phd f: Yeah . For I th I think that it perhaps it shows that  the parameters that the voice activity detector should use  have to use should be different than the parameter that have to be used for speech recognition . phd f: Well , y professor a: So you can do that by doing the voi voice activity detection . You also could do it by spect  spectral subtraction before the variance normalization , right ? phd f: Yeah , but it 's not clear , yeah . phd f: the the number that at that are here are recognition experiments on Italian HM and  with these two kinds of parameters . phd b: Where 's th phd f: But the fact is that the voice activity detector doesn't work on channel one . phd b: Where at what stage is the voice activity detector applied ? Is it applied here or a after the variance normalization ? phd f:  ? professor a: Spectral subtraction , I guess . Is it applied all the way back here ? phd f: It 's applied the  on , yeah , something like this , phd b: Maybe that 's why it doesn't work for channel one . professor a: Can I phd f: So we could perhaps do just mean normalization before VAD . Can I ask a ,  a sort of top - level question , which is  " if if most of what the OGI folk are working with is trying to integrate this other other  spectral subtraction , why are we worrying about it ? " phd f:   phd f: It 's just  Well it 's another They are trying to u to use the  the Ericsson and we 're trying to use something something else . When we do spectral subtraction , actually , I think that this is the the two last figures . phd b: Speech is more what ? phd f: Well , the difference between the energy of the speech and the energy of the n spectral subtrac subtracted noise portion is is larger . phd f: Well , if you compare the first figure to this one Actually the scale is not the same , but if you look at the the numbers  you clearly see that the difference between the C - zero of the speech and C - zero of the noise portion is larger .  but what happens is that after spectral subtraction , you also increase the variance of this of C - zero . And what they did at OGI is just  they don't use on - line normalization , for the moment , on spectral subtraction and I think Yeah . So yeah , we 're working on the same thing but I think  with different different system and professor a: Right .  , i the Intellectually it 's interesting to work on things th  one way or the other phd f:   professor a: but I 'm I 'm just wondering if  on the list of things that there are to do , if there are things that we won't do because we 've got two groups doing the same thing . phd b: if  you know , if if if you work on something else and and you 're waiting for them to give you spectral subtraction  it 's hard to know whether the effects that you get from the other experiments you do will carry over once you then bring in their spectral subtraction module . I don't know if that 's true or not , but I could see how phd f: Mmm .  , we still evidently have a latency reduction plan which which isn't quite what you 'd like it to be . And then  weren't issues of of having a a second stream or something ? That was Was it There was this business that , you know , we we could use up the full forty - eight hundred bits , and phd f: Yeah . And they are t I think they want to work on the second stream also , but more with some kind of multi - band or , well , what they call TRAP or generalized TRAP . Do you remember when the next meeting is supposed to be ? the next  phd f: It 's  in June . Yeah , the other thing is that you saw that that mail about  the VAD V A Ds performing quite differently ? That that  So  . This there was this experiment of  " what if we just take the baseline ? " phd f: Mmm . professor a: set  of features , just mel cepstra , and you inc incorporate the different V A And it looks like the the French VAD is actually  better significantly better . If the use the small VAD I th I think it 's on I think it 's easy to do better because it doesn't work at all . He Actually , I think that he say with the good VAD of from OGI and with the Alcatel VAD . phd f: Yeah but I it 's  I think you were talking about the other mail that used VAD on the reference features . professor a:  it was enough better that that it would  account for a fair amount of the difference between our performance , actually . phd f: And perhaps we can easily improve if if we put like mean normalization before the before the VAD . professor a: H Hynek will be back in town  the week after next , back back in the country . phd d: Also is Stephane was thinking that maybe it was useful to f to think about  voiced - unvoiced phd f:   phd f: Yeah , my feeling is that  actually when we look at all the proposals , ev everybody is still using some kind of spectral envelope professor a: Right . phd f: Yeah , well , not pitch , but to look at the  fine at the at the high re high resolution spectrum . We don't necessarily want to find the the pitch of the of the sound but  Cuz I have a feeling that when we look when we look at the just at the envelope there is no way you can tell if it 's voiced and unvoiced , if there is some It 's it 's easy in clean speech because voiced sound are more low frequency and . phd f:  there is the first formant , which is the larger and then voiced sound are more high frequencies cuz it 's frication and professor a: Right . When you have noise there is no  if if you have a low frequency noise it could be taken for for voiced speech and . professor a: but but phd b: Isn't there some other phd f: S phd b:  d phd f: So I think that it it would be good Yeah , yeah , well , go go on . phd b: I was just gonna say isn't there aren't aren't there lots of ideas for doing voice activity , or speech - nonspeech rather ,  by looking at  , you know ,  I guess harmonics or looking across time professor a: Well , I think he was talking about the voiced - unvoiced , though , phd f: Mmm . phd b:  w ah you know ,  even with the voiced - non voiced - unvoiced phd f: Mmm . phd f:  yeah , so yeah , I think if we try to develop a second stream well , there would be one stream that is the envelope and the second , it could be interesting to have that 's something that 's more related to the fine structure of the spectrum . We were thinking about like using ideas from from Larry Saul , have a good voice detector , have a good , well , voiced - speech detector , that 's working on on the FFT and  professor a: U phd f: Larry Saul could be an idea . We were are thinking about just kind of  taking the spectrum and computing the variance of of the high resolution spectrum and things like this .  we had a guy here some years ago who did some work on  making use of voicing information  to help in reducing the noise . phd f: Yeah ? professor a: So what he was doing is basically y you you do estimate the pitch . professor a: And  you from that you you estimate or you estimate fine harmonic structure , whichev ei either way , it 's more or less the same . But  the thing is that  you then can get rid of things that are not i if there is strong harmonic structure , you can throw away stuff that 's that 's non - harmonic . professor a: And that that is another way of getting rid of part of the noise phd f: Yeah . It was kind of like RASTA was taking care of convolutional stuff phd f: Mmm . We 've actually back when Chuck was here we did some voiced - unvoiced  classification using a bunch of these , phd f: But professor a: and and  works OK . professor a: But the thing is that you can't given the constraints of this task , we can't , in a very nice way , feed forward to the recognizer the information the probabilistic information that you might get about whether it 's voiced or unvoiced , where w we can't you know affect the the  distributions or anything . phd b: Didn't the head dude send around that message ? Yeah , I think you sent us all a copy of the message , where he was saying that I I 'm not sure , exactly , what the gist of what he was saying , but something having to do with the voice activity detector and that it will that people shouldn't put their own in or something . professor a: I guess what you could do , maybe this would be w useful , if if you have if you view the second stream , yeah , before you before you do KLT 's and so forth , if you do view it as probabilities , and if it 's an independent So , if it 's if it 's  not so much envelope - based by fine - structure - based ,  looking at harmonicity or something like that ,  if you get a probability from that information and then multiply it by you know , multiply by all the voiced outputs and all the unvoiced outputs , you know , then use that as the phd f:   professor a:  take the log of that or  pre pre  pre - nonlinearity , phd f: Yeah . i if professor a:  and do the KLT on the on on that , phd f: Yeah . And then that would be phd f: Yeah , well , I was not thinking this yeah , this could be an yeah So you mean have some kind of probability for the v the voicing professor a: R Right . If you have a tandem system and then you have some kind of it can be pretty small net phd f:   professor a: and the and and you use the thing is to use information primarily that 's different as you say , it 's more fine - structure - based than than envelope - based phd f:   professor a:  so then it you you you can pretty much guarantee it 's stuff that you 're not looking at very well with the other one , and  then you only use for this one distinction . professor a: And and so now you 've got a probability of the cases , and you 've got  the probability of the finer  categories on the other side . professor a: if they really are from independent information sources then they should have different kinds of errors phd f:   professor a: and roughly independent errors , and it 's a good choice for phd f:   Because , yeah , well , spectral subtraction is good and we could u we could use the fine structure to to have a better estimate of the noise but still there is this issue with spectral subtraction that it seems to increase the variance of of of professor a: Yeah . phd f:  Well it 's this musical noise which is annoying if you d you do some kind of on - line normalization after . Spectral subtraction and on - line normalization don't seem to to go together very well . I professor a: Or if you do a spectral subtraction do some spectral subtraction first and then do some on - line normalization then do some more spectral subtraction  , maybe maybe you can do it layers or something so it doesn't doesn't hurt too much or something . professor a: But it but  , anyway I think I was sort of arguing against myself there by giving that example phd f: Yeah . professor a:   cuz I was already sort of suggesting that we should be careful about not spending too much time on exactly what they 're doing In fact if you get if you go into  a  harmonics - related thing it 's definitely going to be different than what they 're doing and   phd f:   I know that when have people have done  sort of the obvious thing of taking  your feature vector and adding in some variables which are pitch related or  that it hasn't my impression it hasn't particularly helped . professor a: But I think  that 's that 's a question for this  you know extending the feature vector versus having different streams . Was it nois noisy condition ? the example that you you just professor a: And and it may not have been noisy conditions . I I don't remember the example but it was it was on some DARPA data and some years ago and so it probably wasn't , actually phd f:   But we were thinking , we discussed with Barry about this , and perhaps thinking we were thinking about some kind of sheet cheating experiment where we would use TIMIT professor a:  - huh . phd f: and see if giving the d  , this voicing bit would help in in terms of  frame classification . professor a: Why don't you why don't you just do it with Aurora ? phd f: Mmm . professor a: Just any i in in each in each frame phd f: Yeah , but but B but we cannot do the cheating , this cheating thing . Cuz we don't have Well , for Italian perhaps we have , but we don't have this labeling for Aurora . professor a: But you could  you can you can align so that It 's not perfect , but if you if you know what was said and phd b: But the problem is that their models are all word level models . phd b: You So you could find out where the word boundaries are but that 's about it . grad e: S But we could use  the the noisy version that TIMIT , which you know , is similar to the the noises found in the TI - digits  portion of Aurora . phd f: Well , I guess I guess we can we can say that it will help , but I don't know . If this voicing bit doesn't help , I think we don't have to to work more about this because professor a:  . professor a: and different ta it was probably Resource Management or something , I think you were getting something like still eight or nine percent error on the voicing , as I recall . professor a: what that said is that , sort of , left to its own devices , like without the a strong language model and so forth , that you would you would make significant number of errors just with your  probabilistic machinery in deciding phd b: It also professor a: one oh phd b: Yeah , the though I think  there was one problem with that in that , you know , we used canonical mapping so our truth may not have really been true to the acoustics . Well back twenty years ago when I did this voiced - unvoiced stuff , we were getting more like ninety - seven or ninety - eight percent correct in voicing . professor a: And we did this complex feature selection thing where we looked at all the different possible features one could have for voicing and and and  and exhaustively searched all size subsets and and  for for that particular speaker and you 'd find you know the five or six features which really did well on them . professor a: And then doing doing all of that we could get down to two or three percent error . professor a: So I would I would believe that  it was quite likely that  looking at envelope only , that we 'd be significantly worse than that . phd f: And the all the the SpeechCorders ? what 's the idea behind ? Cuz they they have to Oh , they don't even have to detect voiced spe speech ? professor a: The modern ones don't do a a simple switch . phd f: They just work on the code book professor a: They work on the code book excitation . They try they they try every every possible excitation they have in their code book and find the one that matches best . One of the ideas that we had come up with last week for things to try to improve the system  . Actually I I s we didn't I guess I wrote this in after the meeting b but the thought I had was  looking at the language model that 's used in the HTK recognizer , which is basically just a big loop , grad e:   phd b: and then that can be either go to silence or go to another digit , which That model would allow for the production of infinitely long sequences of digits , right ? professor a: Right . I thought " well I 'm gonna just look at the what actual digit strings do occur in the training data . phd b: And the interesting thing was it turns out that there are no sequences of two - long or three - long digit strings in any of the Aurora training data . So it 's either one , four , five , six ,  up to eleven , and then it skips and then there 's some at sixteen . So I I just for the heck of it , I made a little grammar which  , you know , had it 's separate path for each length digit string you could get . So there was a one - long path and there was a four - long and a five - long professor a:   phd b: So it was you know , I I didn't have any weights of these paths or I didn't have anything like that . phd b: And I played with tweaking the word transition penalties a bunch , but I couldn't go anywhere . I thought " well if I only allow " Yeah , I guess I should have looked at to see how often there was a mistake where a two - long or a three - long path was actually put out as a hypothesis . phd b: So to do that right you 'd probably want to have allow for them all but then have weightings and things <doc-sep>We  we abandoned the lapel because they sort of were not too not too hot , not too cold , they were you know , they were  , far enough away that you got more background noise , and  and so forth phd a:  - huh . professor c: but they weren't so close that they got quite the you know , the really good No , th phd a: OK . They were not so far away that they were really good representative distant mikes , phd a:  - huh . professor c: but on the other hand they were not so close that they got rid of all the interference . On the other hand if you only had to have one mike in some ways you could argue the lapel was a good choice , precisely because it 's in the middle . professor c: There 's  , some kinds of junk that you get with these things that you don't get with the lapel  , little mouth clicks and breaths and so forth are worse with these than with the lapel , but given the choice we there seemed to be very strong opinions for  , getting rid of lapels . phd a: The mike number is professor c: So , phd f: your mike number 's written on the back of that unit there . phd f: It - it 's one less than what 's written on the back of your phd a: OK . professor c: And you should do a lot of talking so we get a lot more of your pronunciations . phd f: So what we usually do is  , we typically will have our meetings professor c: Yeah . One thing phd f: Sunil 's here for the summer ? professor c: Sunil 's here for the summer , right .  , so , one thing is to talk about a kick off meeting maybe  , and then just  , I guess  , progress reports individually , and then  , plans for where we go between now and then , pretty much . phd f: I could say a few words about  , some of the  , compute stuff that 's happening around here , so that people in the group know . professor c: Yeah ? phd f: We  So we just put in an order for about twelve new machines , to use as sort of a compute farm . And  , we ordered  , SUN - Blade - one - hundreds , and  , I 'm not sure exactly how long it 'll take for those to come in , but , in addition , we 're running So the plan for using these is , we 're running P - make and Customs here and Andreas has sort of gotten that all  , fixed up and up to speed . And he 's got a number of little utilities that make it very easy to  , run things using P - make and Customs . The simplest thing And I can send an email around or , maybe I should do an FAQ on the web site about it or something .  , professor c: How about an email that points to the FAQ , phd f: there 's a c professor c: you know what I 'm saying ? phd f: Yeah , yeah . And , if you say that and then some job that you want to execute , it will find the fastest currently available machine , and export your job to that machine , and  and run it there and it 'll duplicate your environment . So you can say " run dash command L S " , and , it 'll actually export that LS command to some machine in the institute , and  , do an LS on your current directory . So , substitute LS for whatever command you want to run , and  And that 's a simple way to get started using using this . And , so , soon , when we get all the new machines up , e then we 'll have lots more compute to use . Now th one of the nice things is that  , each machine that 's part of the P - make and Customs network has attributes associated with it .  , attributes like how much memory the machine has , what its speed is , what its operating system , and when you use something like " run command " , you can specify those attributes for your program . For example if you only want your thing to run under Linux , you can give it the Linux attribute , and then it will find the fastest available Linux machine and run it on that . You can control where your jobs go , to a certain extent , all the way down to an individual machine . If there 's already a job running , on some machine that you 're trying to select , your job will get queued up , and then when that resource , that machine becomes available , your job will get exported there . So , there 's a lot of nice features to it and it kinda helps to balance the load of the machines and  , right now Andreas and I have been the main ones using it and we 're  . professor c: So as I understand , you know , he 's using all the machines and you 're using all the machines , phd f: So . Yeah , you know , I I sort of got started using the recognizer just recently and  ,  I fired off a training job , and then I fired off a recognition job and I get this email about midnight from Andreas saying , "  , are you running two trainings simultaneously s my m my jobs are not getting run . grad g: I have I have a question about the  , parallelization ? phd f:   grad g: So , let 's say I have like , a thousand little little jobs to do ? phd f:   grad g: how do I do it with " run command " ?  do phd f: You could write a script  , which called run command on each sub - job grad g:  - huh . A thousand times ? phd f: right ? But you probably wanna be careful with that grad g: OK .  , so , you know , you should you should probably not run more than , say ten jobs yourself at any one time , just because then it would keep other people grad g: Oh , too much file transfer and stuff . phd f: Well it 's not that so much as that , you know , e with if everybody ran fifty jobs at once then it would just bring everything to a halt and , you know , people 's jobs would get delayed , so it 's sort of a sharing thing . phd f: so you should try to limit it to somet sometim some number around ten jobs at a time . So if you had a script for example that had a thousand things it needed to run , you 'd somehow need to put some logic in there if you were gonna use " run command " , to only have ten of those going at a time .  , professor c: I remember I I forget whether it was when the Rutgers or or Hopkins workshop , I remember one of the workshops I was at there were everybody was real excited cuz they got twenty - five machines and there was some kind of P - make like thing that sit sent things out . professor c: So all twenty - five people were sending things to all twenty - five machines phd f:   professor c: and and things were a lot less efficient than if you 'd just use your own machine . phd f: but  , you can also If you have that level of parallelization  , and you don't wanna have to worry about writing the logic in in a Perl script to take care of that , you can use  , P - make grad g: Just do P - make . phd f: and and you basically write a Make file that  , you know your final job depends on these one thousand things , grad g: s   phd f: and when you run P - make , on your Make file , you can give it the dash capital J and and then a number , grad g:   If you " Run command " , that I mentioned before , is doesn't know about other things that you might be running . phd f: So , it would be possible to run a hundred run jobs at once , phd d: Right . But if you use P - make , then , it knows about all the jobs that it has to run phd d:   professor c: So " run command " doesn't use P - make , or ? phd f: It uses " export " underlyingly . But , if you i It 's meant to be run one job at a time ? So you could fire off a thousand of those , and it doesn't know any one of those doesn't know about the other ones that are running . professor c: So why would one use that rather than P - make ? phd f: Well , if you have ,  Like , for example ,  if you didn't wanna write a P - make script and you just had a ,  an HTK training job that you know is gonna take  , six hours to run , and somebody 's using , the machine you typically use , you can say " run command " and your HTK thing and it 'll find another machine , the fastest currently available machine and and run your job there . professor c: Now , does it have the same sort of behavior as P - make , which is that , you know , if you run something on somebody 's machine and they come in and hit a key then it phd f: Yes . And if you specify that , in in one of your attribute lines , then it 'll go to a machine which your job won't be evicted from . phd f: But , the machines that don't have that attribute , if a job gets fired up on that , which could be somebody 's desktop machine , and and they were at lunch , professor c:   phd f: they come back from lunch and they start typing on the console , then your machine will get evicted your job will get evicted from their machine and be restarted on another machine . So which can cause you to lose time , right ? If you had a two hour job , and it got halfway through and then somebody came back to their machine and it got evicted . If you don't want your job to run on a machine where it could be evicted , then you give it the minus the attribute , you know , " no evict " , and it 'll pick a machine that it can't be evicted from . professor c: what what about I remember always used to be an issue , maybe it 's not anymore , that if you if something required if your machine required somebody hitting a key in order to evict things that are on it so you could work , but if you were logged into it from home ? phd f:   professor c: and you weren't hitting any keys ? cuz you were , home ? phd f: Yeah , I I 'm not sure how that works . I don't know whether it monitors the keyboard or actually looks at the console TTY , so maybe if you echoed something to the you know , dev dev console or something . phd f:  ? professor c:  you sort of you 're at home and you 're trying to log in , and it takes forever to even log you in , and you probably go , " screw this " , phd f: Yeah , yeah . phd a: I need a little orientation about this environment and  scr s how to run some jobs here because I never d did anything so far with this X emissions phd f: OK . Yeah , and and also  , Stephane 's a a really good resource for that if you can't find me . Well , why don't we  , Sunil since you 're haven't haven't been at one of these yet , why don't yo you tell us what 's what 's up with you ? Wh - what you 've been up to , hopefully . phd a: yeah , after the submission the what I 've been working on mainly was to take take other s submissions and then over their system , what they submitted , because we didn't have any speech enhancement system in in ours . And then I found that  , if if I combine it with LDA , it gives @ @ improvement over theirs . I just plug in I just take the cepstral coefficients coming from their system and then plug in LDA on top of that . phd a: What I did was I took the LDA filter 's design using clean speech , mainly because the speech is already cleaned up after the enhancement so , instead of using this , narrow narrow band LDA filter that we submitted  , I got new filters . And  , so then after after that I I added  , on - line normalization also on top of that . And that there there also I n I found that I have to make some changes to their time constant that I used because th it has a a mean and variance update time constant and which is not suitable for the enhanced speech , and whatever we try it on with proposal - one . But  , I didn't I didn't play with that time constant a lot , I just t g I just found that I have to reduce the value  , I have to increase the time constant , or reduce the value of the update value . And  , the other other thing what I tried was , I just  , took the baseline and then ran it with the endpoint inf  th information , just the Aurora baseline , to see that how much the baseline itself improves by just supplying the information of the  the w speech and nonspeech . And  , I found that the baseline itself improves by twenty - two percent by just giving the wuh . professor c: can you back up a second , I I I missed something , I guess my mind wandered . Ad - ad When you added the on - line normalization and so forth ,  things got better again ? phd a: Yeah . phd a: With the different time constant I found that  , I didn't get an improvement over not using on - line normalization , professor c: Oh . phd a: But I didn't play it with play play quite a bit to make it better than . The the other thing what I tried was the adding the  , endpoint information to the baseline and that itself gives like twenty - two percent because the the second the new phase is going to be with the endpointed speech . And just to get a feel of how much the baseline itself is going to change by adding this endpoint information , I just , use professor c:  . phd f: So people won't even have to worry about , doing speech - nonspeech then . professor c: G I guess the issue is that people do that anyway , phd f: I see . professor c: and they wanted to see , given that you 're doing that , what what are the best features that you should use . professor c: But but it might be  In some ways it might be better t to rather than giving the endpoints , to have a standard that everybody uses and then interacts with . phd f: So , are people supposed to assume that there is  Are are people not supposed to use any speech outside of those endpoints ? phd a:  phd f: Or can you then use speech outside of it for estimating background noise and things ? phd a: No . Like y you will you will You 'll be given the information about the beginning and the end of speech but the whole speech is available to you . professor c: So it should make the spectral subtraction style things work even better , phd a: Yeah . I found that in s one of the SpeechDat - Car cases , that like , the Spanish one improves by just fifty percent by just putting the endpoint . phd a: Yeah , so professor c: But but phd a: so that is when  , the the qualification criteria was reduced from fifty percent to something like twenty - five percent for well - matched . And  , Yeah , I guess after that , I just went home f I just had a vacation fo for four weeks . phd a: Ye Yeah , and I I came back and I started working on  , some other speech enhancement algorithm .  , so I from the submission what I found that people have tried spectral subtraction and Wiener filtering . phd a: so just to just to fill the space with some f few more speech enhancement algorithms to see whether it improves a lot , I I 've been working on this  , signal subspace approach for speech enhancement where you take the noisy signal and then decomposing the signal s and the noise subspace and then try to estimate the clean speech from the signal plus noise subspace . phd a: So , I 've been actually running some s So far I 've been trying it only on Matlab . phd a: and then I 'll p port it to C and I 'll update it with the repository once I find it it giving any some positive result . professor c: S So you s you So you said one thing I want to jump on for a second . So so now you 're you 're getting tuned into the repository thing that he has here phd a: Yeah . Cuz you did some stuff that you talked about last week , I guess ? phd d:   professor c: where you were also combining something both of you I guess were both combining something from the  , French Telecom system with the u  phd d: Right . professor c: I I don't know whether it was system one or system two , or ? phd d:   phd d: we The main thing that we did is just to take the spectral subtraction from the France Telecom , which provide us some speech samples that are  , with noise removed . So then , one distinction is that  , you were taking the actual France Telecom features and then applying something to phd a: no there is a slight different .   , which are extracted at the handset because they had another back - end blind equalization professor c: Yeah . professor c: What I meant was you had something like cepstra or something , right ? phd a: Yeah , yeah , yeah , yeah . But I guess it 's the s exactly the same thing because on the heads  , handset they just applied this Wiener filter and then compute cepstral features , phd a: Yeah , the cepstral f The difference is like There may be a slight difference in the way phd d: right ? or ? phd a: because they use exactly the baseline system for converting the cepstrum once you have the speech . But  , I th phd d: Yeah , well I think we should  , have a table with all the result because I don't know I  , I don't exactly know what are your results ? But , phd a: OK . Yeah , but so we did this , and another difference I guess is that we just applied  , proposal - one system after this without well , with our modification to reduce the delay of the the LDA filters , phd a:  - huh . phd d: and phd b: And the filter phd d: Well there are slight modifications , but it was the full proposal - one . In your case , if you tried just putting LDA , then maybe on - line normalization ? phd a: Only LDA . So we just tried directly to to just , keep the system as it was and , when we plug the spectral subtraction it improves  , signif significantly .  , but , what seems clear also is that we have to retune the time constants of the on - line normalization . phd d: Because if we keep the value that was submitted  , it doesn't help at all . But , you can still find some kind of optimum somewhere , and we don't know where exactly phd a: Yeah . professor c: So it sounds like you should look at some tables of results or something phd d: Right . professor c: and see where i where the where they were different and what we can learn from it . phd d: because we change it the system to have phd a: Oh yeah ,  the the new LDA filters . There are other things that we finally were shown to improve also like , the sixty - four hertz cut - off . phd d: And , right now if we look at the results , it 's , always better than it seems always better than France Telecom for mismatch and high - mismatch . But , the problem is that it 's not significant , but if you put this in the , mmm , spreadsheet , it 's still worse . I don't think it 's importa important because when they will change their metric , mainly because of  , when you p you plug the  , frame dropping in the baseline system , it will improve a lot HM , and  , phd a: Yeah . phd a: Because the your improvement on HM and  will also go down significantly in the spreadsheet so . phd a:  the well - matched may be the one which is least affected by adding the endpoint information . But they d the everything  is like , but there that 's how they reduce why they reduce the qualification to twenty - five percent or some something on . professor c: But are they changing the weighting ? phd a: no , I guess they are going ahead with the same weighting . professor c: I guess I I haven't been part of the discussion , so , it seems to me that the well - matched condition is gonna be unusual , phd a: Usual . professor c: Because , you don't actually have good matches ordinarily for what any @ @ particular person 's car is like , or phd a: Mmm . phd a: Yeah , but actually the well well the well - matched  ,  the the well - matched condition is not like , the one in TI - digits where  , you have all the training , conditions exactly like replicated in the testing condition also . The well - matched has also some some mismatch in that which is other than the professor c: The well wa matched has mismatch ? phd a: has has also some slight mismatches , unlike the TI - digits where it 's like prefectly matched phd f: Perfect to match . So remind me of what well - matched meant ? phd a: The the well - matched is like professor c: You 've told me many times . phd a: the the well - matched is defined like it 's seventy percent of the whole database is used for training and thirty percent for testing . phd a: It 's it 's phd d: Because it phd a: OK , it 's professor c: Yeah . So , yeah , unless they deliberately chose it to be different , which they didn't because they want it to be well - matched , it is pretty much You know , so it 's so it 's sort of saying if you phd f: It 's it 's not guaranteed though . phd a: the main mismatch is coming from the amount of noise and the silence frames and all those present in the database actually . professor c: Again , if you have enough if you have enough phd a: No yeah , yeah . professor c: So it 's sort of i i it 's sort of saying OK , so you much as you train your dictation machine for talking into your computer , you you have a car , and so you drive it around a bunch and and record noise conditions , or something , and then I don't think that 's very realistic ,  I th phd a:   professor c: I I you know , so I I I you know , I guess they 're saying that if you were a company that was selling the stuff commercially , that you would have a bunch of people driving around in a bunch of cars , and and you would have something that was roughly similar and maybe that 's the argument , but I 'm not sure I buy it , so . We are playing we are also playing , trying to put other spectral subtraction mmm , in the code .  , it would be a very simple spectral subtraction , on the  , mel energies which I already tested but without the  frame dropping actually , and I think it 's important to have frame dropping if you use spectral subtraction . phd f: Is it is spectral subtraction typically done on the after the mel , scaling or is it done on the FFT bins ? phd d: phd f: Does it matter , or ? phd d: I d I don't know . So - some of the proposal , we 're doing this on the bin on the FFT bins , phd f:  . You can do both , but I cannot tell you what 's which one might be better or I phd f:  . phd a: I guess if you want to reconstruct the speech , it may be a good idea to do it on FFT bins .  it may not be very different if you do it on mel warped or whether you do it on FFT . phd d: Well , it gives something different , but I don't know what are the , pros and cons of both . phd a: The other thing is like when you 're putting in a speech enhancement technique , is it like one stage speech enhancement ? Because everybody seems to have a mod two stages of speech enhancement in all the proposals , which is really giving them some improvement . phd a: And So , there 's something that is good about doing it  , to cleaning it up once more . phd a: so we can phd d: So maybe in my implementation I should also try to inspire me from this kind of thing phd a: Yeah . That 's what professor c: Well , the other thing would be to combine what you 're doing . professor c:  maybe one or one or the other of the things that you 're doing would benefit from the other happening first . So , professor c: Right , so he 's doing a signal subspace thing , maybe it would work better if you 'd already done some simple spectral subtraction , or maybe vi maybe the other way around , phd d: Yeah ,   professor c: you know ? phd a: So I 've been thinking about combining the Wiener filtering with signal subspace , phd d:   phd a:  just to see all some some such permutation combination to see whether it really helps or not . professor c: How is it I I guess I 'm ignorant about this , how does  , since Wiener filter also assumes that you 're that you 're adding together the two signals , how is how is that differ from signal subspace ? phd a: The signal subspace ? The professor c: Yeah . phd a: So , the the different the c the the advantage of combining two things is mainly coming from the signal subspace approach doesn't work very well if the SNR is very bad . So essentially you could do simple spectral subtraction , followed by a KL transform , followed by a phd a: Wiener filtering . Yeah , in general , you don't that 's right you don't wanna othorg orthogonalize if the things are noisy .  , that was something that  , Herve and I were talking about with  , the multi - band stuff , that if you 're converting things to from  , bands , groups of bands into cepstral coef you know , local sort of local cepstral coefficients that it 's not that great to do it if it 's noisy . phd a: So that that 's one reason maybe we could combine s some something to improve SNR a little bit , first stage , professor c: Yeah . phd d: What was your point about about colored noise there ? phd a: Oh , the colored noise  phd d: Yeah . phd a: the colored noise the the v the signal subspace approach has  , it it actually depends on inverting the matrices . phd a:  it has a it 's It doesn't behave very well if it is not positive definite ak It works very well with white noise because we know for sure that it has a positive definite . phd a: So the way they get around is like they do an inverse filtering , first of the colo colored noise professor c: Yeah . phd a: and then finally when you reconstruct the speech back , you do this filtering again .  if you sort of you do the s spectral subtraction , that also gets rid phd a: Yeah . professor c: and then you then then add a little bit l noise noise addition  , that sort of what J JRASTA does , in a way . professor c: If you look at what JRASTA doing essentially i i it 's equivalent to sort of adding a little adding a little noise , phd a: Huh ?  - huh . And maybe we well we find some people so that  , agree to maybe work with us , and they have implementation of VTS techniques so it 's  , Vector Taylor Series that are used to mmm ,  f to model the transformation between clean cepstra and noisy cepstra . Well , if you take the standard model of channel plus noise , it 's it 's a nonlinear eh  , transformation in the cepstral domain . phd d: And  , there is a way to approximate this using  , first - order or second - order Taylor Series and it can be used for  , getting rid of the noise and the channel effect . professor c: Who is doing this ? phd d:  w working in the cepstral domain ? So there is one guy in Grenada , phd b: Yeah , in Grenada one of my friend . professor c: Who 's the guy in Grenada ? phd d: phd b: Jose Carlos Segura . professor c: Yeah , so at any rate , you 're looking general , standing back from it , looking at ways to combine one form or another of  , noise removal , with with these other things we have , phd d:   But for sure there 's required to that requires to re - check everything else , and re - optimize the other things professor c: Oh yeah .  , professor c: Well one of the seems like one of the things to go through next week when Hari 's here , phd d: I professor c: cuz Hari 'll have his own ideas too or I guess not next week , phd d:  - huh . professor c: week and a half , will be sort of go through these alternatives , what we 've seen so far , and come up with some game plans .  one would be , you look at a few things very quickly , you pick on something that looks like it 's promising and then everybody works really hard on the same different aspects of the same thing . Another thing would be to have t to to pick two pol two plausible things , and and you know , have t sort of two working things for a while until we figure out what 's better , phd d:   professor c: and then , you know , but , w  , he 'll have some ideas on that too . phd a: The other thing is to ,  Most of the speech enhancement techniques have reported results on small vocabulary tasks . But we we going to address this Wall Street Journal in our next stage , which is also going to be a noisy task so s very few people have reported something on using some continuous speech at all . So , there are some  , I was looking at some literature on speech enhancement applied to large vocabulary tasks and spectral subtraction doesn't seems to be the thing to do for large vocabulary tasks . And it 's Always people have shown improvement with Wiener filtering and maybe subspace approach over spectral subtraction everywhere . But if we if we have to use simple spectral subtraction , we may have to do some optimization to make it work @ @ . professor c: So they 're making there Somebody 's generating Wall Street Journal with additive artificially added noise or something ? phd a: Yeah , yeah . professor c: Sort of a sort of like what they did with TI - digits , and ? phd a: Yeah . professor c: And then they 're they 're  , generating HTK scripts to phd a: Yeah . There are they have there is no I don't know if they are converging on HTK or are using some Mississippi State , professor c: Mis - Mississippi State maybe , phd a: yeah . professor c: well we 've Yeah , it 's true for the additive noise , y artificially added noise we 've always used small vocabulary too . But for n there 's been noisy speech this larv large vocabulary that we 've worked with in Broadcast News . professor c: and some of the focus conditions were noisy and and phd a: It had additive n professor c: But we but we didn't do spectral subtraction . We were doing our funny stuff , right ? We were doing multi multi  , multi - stream and and so forth . professor c: and and  , that we have  , for the  , the quote - unquote noisy data there is just noisy and reverberant actually . And that 's what most o again , most of our work has been done with that , with with  , connected digits . professor c: but  , we have recognition now with some of the continuous speech , large vocabulary continuous speech , using Switchboard  , Switchboard recognizer , phd a: Yeah . You just take the Switchboard trained ? Yeah , professor c: That 's that 's what we 're doing , phd a: yeah . professor c: but we 're hop  , actually  , Dave and I were just talking earlier today about maybe at some point not that distant future , trying some of the techniques that we 've talked about on , some of the large vocabulary data .  , I guess no one had done yet done test one on the distant mike using  , the SRI recognizer and , phd f: I don't not that I know of . professor c: You 'll see a little smoke coming up from the the CPU or something trying to trying to do it , phd f: That 's right professor c: but  , yeah . But , you 're right that that that 's a real good point , that  , we we don't know yeah , what if any of these ta I guess that 's why they 're pushing that in the  in the evaluation . Anything else going on ? at you guys ' end , phd b: I don't have good result , with the inc including the new parameters , professor c: or ? phd b: I don't have good result . phd a: With what what other new p new parameter ? grad g: You 're talking about your voicing ? professor c: Yeah . professor c: seeing as how Sunil , phd b: I tried to include another new parameter to the traditional parameter , professor c: yeah . phd b: that , like , the auto - correlation , the R - zero and R - one over R - zero phd a:   phd b: and another estimation of the var the variance of the difference for of the spec si  , spectrum of the signal and and the spectrum of time after filt mel filter bank . phd b: and you have the on the other side you have the output of the mel filter bank . phd b: You can extend the coefficient of the mel filter bank and obtain an approximation of the spectrum of the signal . phd b: because , suppose we we think that if the variance is high , maybe you have n  , noise . phd b: To to To The idea is to found another feature for discriminate between voice sound and unvoice sound . And I did experiment I need to change to obtain this new feature I need to change the size the window size size . phd b: And I do I did two type of experiment to include this feature directly with the with the other feature and to train a neural network to select it voice - unvoice - silence silence phd a: Unvoiced . It 's neve e e sometime it 's worse , sometime it 's a little bit better , but not significantly . phd a: is it with TI - digits , or with ? phd b: And No , I work with eh , Italian and Spanish basically . phd b: And if I don't y use the neural network , and use directly the feature the results are worse . professor c:  we 've had these discussions before , and and one of the things that struck me was that  , about this line of thought that was particularly interesting to me was that we  whenever you condense things , in an irreversible way , you throw away some information . And , that 's mostly viewed on as a good thing , in the way we use it , because we wanna suppress things that will cause variability for  particular , phonetic units . And so the question is , can we figure out if there 's something we 've thrown away that we shouldn't have . So , when they were looking at the difference between the filter bank and the FFT that was going into the filter bank , I was thinking " oh , OK , so they 're picking on something they 're looking on it to figure out noise , or voice voiced property whatever . But for me sort of the interesting thing was , " well , but is there just something in that difference which is useful ? " So another way of doing it , maybe , would be just to take the FFT  , power spectrum , and feed it into a neural network , phd b: To know professor c: and then use it , you know , in combination , or alone , or or whatever phd f: Wi - with what targets ? phd a: Voiced , unvoiced is like professor c: no . professor c: No the just the same same way we 're using  , the same way that we 're using the filter bank . And , you know , maybe if it 's used in combination , it will get at something that we 're missing . And maybe , you know , using , orth you know , KLT , or  , adding probabilities , all th all the different ways that we 've been playing with , that we would let the essentially let the neural network determine what is it that 's useful , that we 're missing here . professor c: Well , that 's probably why y i it would be unlikely to work as well by itself , but it might help in combination . professor c: But I I I have to tell you , I can't remember the conference , but , I think it 's about ten years ago , I remember going to one of the speech conferences and and  , I saw within very short distance of one another a couple different posters that showed about the wonders of some auditory inspired front - end or something , and a couple posters away it was somebody who compared one to  , just putting in the FFT and the FFT did slightly better . professor c: but again we have these wonderful statistical mechanisms for quantifying that a that variability , and you know , doing something reasonable with it . professor c: So , It - it 's same , you know , argument that 's gone both ways about  , you know , we have these data driven filters , in LDA , and on the other hand , if it 's data driven it means it 's driven by things that have lots of variability , and that are necessarily not necessarily gonna be the same in training and test , so , in some ways it 's good to have data driven things , and in some ways it 's bad to have data driven things . So , phd a: Yeah , d professor c: part of what we 're discovering , is ways to combine things that are data driven than are not . professor c: so anyway , it 's just a thought , that that if we if we had that maybe it 's just a baseline  , which would show us " well , what are we really getting out of the filters " , or maybe i i probably not by itself , but in combination , phd d:   professor c: you know , maybe there 's something to be gained from it , and let the But , you know , y you 've only worked with us for a short time , maybe in a year or two you w you will actually come up with the right set of things to extract from this information . phd a: What one one  p one thing is like what before we started using this VAD in this Aurora , the th what we did was like , I I guess most of you know about this , adding this additional speech - silence bit to the cepstrum and training the  on that . phd a: That is just a binary feature and that seems to be improving a lot on the SpeechDat - Car where there is a lot of noise but not much on the TI - digits . So , a adding an additional feature to distin to discriminate between speech and nonspeech was helping . phd d: Wait I I 'm sorry ? phd a: Yeah , we actually added an additional binary feature to the cepstrum , just the baseline . Well , in in the case of TI - digits it didn't actually give us anything , because there wasn't any f anything to discriminate between speech , phd d: Yeah . But anyway the question is even more , is within speech , can we get some features ? Are we drop dropping information that can might be useful within speech , phd a: OK . professor c: And it 's particularly more relevant now since we 're gonna be given the endpoints . phd a: There was a paper in ICASSP this ICASSP over the  extracting some higher - order  , information from the cepstral coefficients and I forgot the name . Some is some harmonics I don't know , I can I can pull that paper out from ICASSP . phd a: It wa it was taking the ,  It was about finding the higher - order moments of Yeah . professor c: Yeah , phd a: And I 'm not sure about whether it is the higher - order moments , or professor c: cumulants , yeah . professor c: Yeah , but again You could argue that th that 's exactly what the neural network does . professor c: So n neural network  , is in some sense equivalent to computing , you know , higher - order moments of what you phd a: trying to f to Moments , yeah . I can I can just  , share a little bit Sunil hasn't hasn't heard about  , what I 've been doing . grad g: so , I told you I was I was I was getting prepared to take this qualifier exam . So basically that 's just , trying to propose  , your next your your following years of of your PHD work , trying trying to find a project to to define and and to work on . So , I 've been , looking into , doing something about r  , speech recognition using acoustic events . So , the idea is you have all these these different events , for example voicing , nasality , R - coloring , you know burst or noise , frication , that kinda stuff , building robust  , primary detectors for these acoustic events , and using the outputs of these robust detectors to do speech recognition .  , and , these these primary detectors , will be , inspired by , you know , multi - band techniques , doing things , similar to Larry Saul 's work on , graphical models to to detect these these , acoustic events . And , so I I been I been thinking about that and some of the issues that I 've been running into are , exactly what what kind of acoustic events I need , what  , what acoustic events will provide a a good enough coverage to in order to do the later recognition steps . And , also , once I decide a set of acoustic events , h how do I how do I get labels ? Training data for for these acoustic events . And , then later on down the line , I can start playing with the the models themselves , the the primary detectors .  , so , I kinda see like , after after building the primary detectors I see  , myself taking the outputs and feeding them in , sorta tandem style into into a  , Gaussian mixtures  back - end , and doing recognition . grad g: professor c: By by the way , the voiced - unvoiced version of that for instance could tie right in to what Carmen was looking at . professor c: you know , if you if a multi - band approach was helpful as as I think it is , it seems to be helpful for determining voiced - unvoiced , grad g:   And so , this this past week  , I 've been  , looking a little bit into  , TRAPS  , and doing doing TRAPS on on these e events too , just , seeing seeing if that 's possible .  , and  , other than that , I was kicked out of I - house for living there for four years . professor c: or , no ? grad g: well , s s som something like that . professor c: Suni - i d ' you v did  did you find a place ? phd a: no professor c: Is that out of the way ? phd a: not yet .  , yesterday I called up a lady who ha who will have a vacant room from May thirtieth and she said she 's interviewing two more people . And then , you 're coming back  phd a: i  , I I p I plan to be here on thirty - first . professor c: Thirty - first , phd a: Yeah , well if there 's a house available or place to professor c: OK . professor c: They 're available , and they 'll be able to get you something , so worst comes to worst we 'll put you up in a hotel for for for a while phd a: Yeah . grad e: You know , if you 're in a desperate situation and you need a place to stay , you could stay with me for a while . professor c: Do y wanna say anything about You you actually been  , last week you were doing this stuff with Pierre , you were you were mentioning . Is that that something worth talking about , or ? grad e: it 's Well , it I don't think it directly relates .  , well , so , I was helping a speech researcher named Pierre Divenyi and he 's int He wanted to  , look at  , how people respond to formant changes , I think . So he he created a lot of synthetic audio files of vowel - to - vowel transitions , and then he wanted a psycho - acoustic  , spectrum . And he wanted to look at  , how the energy is moving over time in that spectrum and compare that to the to the listener tests . And to  he he t wanted to track the peaks so he could look at how they 're moving . I found the roots of the  , LPC polynomial to , track the peaks in the , PLP LPC spectra . phd a: well there is aligned spectral pairs , is like the the Is that the aligned s professor c: It 's a r root LPC , of some sort . phd a: Oh , it 's like line sp professor c: Except I think what they call line spectral pairs they push it towards the unit circle , don't they , phd a: Yeah , yeah , yeah , yeah . But what we 'd used to do w when I did synthesis at National Semiconductor twenty years ago , the technique we were playing with initially was was taking the LPC polynomial and and  , finding the roots . It wasn't PLP cuz Hynek hadn't invented it yet , but it was just LPC , and  , we found the roots of the polynomial , And th When you do that , sometimes they 're f they 're what most people call formants , sometimes they 're not . professor c:  Formant tracking with it can be a little tricky cuz you get these funny values in in real speech , phd f: So you just You typically just get a few roots ? professor c: but . phd f: something like that ? professor c: And it depends on the order that you 're doing , but . So , if @ @ Every root that 's Since it 's a real signal , the LPC polynomial 's gonna have real coefficients . So I think that means that every root that is not a real root is gonna be a c complex pair , phd f:   So for each And if you look at that on the unit circle , one of these one of the members of the pair will be a positive frequency , one will be a negative frequency , I think . So I just So , f for the I 'm using an eighth - order polynomial and I 'll get three or four of these pairs professor c: Yeah .  for real speech in real then what you end up having is , like I say , funny little things that are don't exactly fit your notion of formants all that well . grad e: Mmm , professor c: And and what  in in what we were doing , which was not so much looking at things , it was OK phd d: I professor c: because it was just a question of quantization .  , we were just you know , storing It was We were doing , stored speech , quantization . professor c: But but  , in your case  , you know phd d: Actually you have peaks that are not at the formant 's positions , but they are lower in energy grad e: But there 's some of that , yes . phd f: If this is synthetic speech can't you just get the formants directly ?  h how is the speech created ? grad e: It was created from a synthesizer , and  phd f: Wasn't a formant synthesizer was it ? professor c: I bet it it might have may have been grad e: I d d this professor c: but maybe he didn't have control over it or something ? grad e: In in fact w we we could get , formant frequencies out of the synthesizer , as well . And , w one thing that the , LPC approach will hopefully give me in addition , is that I I might be able to find the b the bandwidths of these humps as well . grad e: but I don't think there 's a g a really good reason not to  , get the formant frequencies from the synthesizer instead . professor c: Yeah , so the actual So you 're not getting the actual formants per se . professor c: You 're getting something that is is  , af strongly affected by the PLP model . So it 's a little It 's It 's It 's sort of sort of a different thing . i Ordinarily , in a formant synthesizer , the bandwidths as well as the ban  , formant centers are phd f: Yeah . professor c: that 's Somewhere in the synthesizer that was put in , as as what you grad e:   professor c: But but yeah , you view each complex pair as essentially a second - order section , which has , band center and band width , and  ,  But . So , yeah , you 're going back today and then back in a week I guess , phd a: Yeah 
The team explored different approaches for spectral subtraction, including both scaling and GGT bins. Each approach had its pros and cons. The professor suggested a simple spectral subtraction followed by a KL transformation. Additionally, he recommended using JRASTRA's technique, which involved adding noise to eliminate noise. The team also discussed VTS techniques. Initially, the professor did not see the advantage of combining the Wiener filter and the subspace, but later realized that a simple spectral subtraction, followed by a KL transform and a Wiener filter, would suffice. The log magnitude spectral subtraction utilized twelve seconds from the past and future to calculate the mean. The use of a short window did not significantly impact the model's performance, which was desirable. The team also removed silence to enhance the model's effectiveness, drawing a parallel to caching utterances. PhD F shared that the preliminary results from spectral subtraction were unsatisfactory, but mean normalization helped reduce noise. However, mean and variance normalization only improved speech, not noise variance.
Summarize the discussion on spectral subtraction and the opinions of the professor and PhD F.