 , I can say about just q just quickly to get through it , that Dave and I submitted this ASRU .  , basically we 're dealing with rever reverberation , and , when we deal with pure reverberation , the technique he 's using works really , really well .  , and when they had the reverberation here , we 'll measure the signal - to - noise ratio and it 's , about nine DB . professor b: phd a: You mean , from the actual , recordings ? professor b: a fair amount of phd d: k phd a: It 's nine DB ? professor b: Yeah .  , I know that when you figured out the filters that we 're using for the Mel scale , there was some experimentation that went on at at ,  at OGI .  , but one of the differences that we found between the two systems that we were using , the the Aurora HTK system baseline system and the system that we were the the  , other system we were using , the  , the SRI system , was that the SRI system had maybe a , hundred hertz high - pass . professor b: For some reason , Dave thought it was twenty , phd d:  So the , center would be somewhere around like hundred professor b: but . professor b: But do you know , for instance , h how far down it would be at twenty hertz ? What the how much rejection would there be at twenty hertz , let 's say ? phd d: At twenty hertz . professor b: Yeah , any idea what the curve looks like ? phd d: Twenty hertz frequency Oh , it 's it 's zero at twenty hertz , right ? The filter ? phd c: Yea - actually , the left edge of the first filter is at sixty - four . professor b: It 's actually set to zero ? What kind of filter is that ? phd c: Yeah . professor b: Is this oh , from the from phd c: It This is the filter bank in the frequency domain that starts at sixty - four . professor b: Oh , so you ,  so you really set it to zero , the FFT ? phd d: Yeah , phd c: Yeah . So I wonder , is it @ @ Was there their experimentation with , say , throwing away that filter or something ? And ,  phd d: throwing away the first ? professor b: Yeah . professor b: Right , but the question is , whether sixty - four hertz is is , too , low . phd d: I t I think I 've tried a hundred and it was more or less the same , or slightly worse . professor b: On what test set ? phd d: On the same , SpeechDat - Car , Aurora . professor b: phd d: So it was professor b: and on and on the , TI - digits also ? phd d: No , no , no . That 'd be something to look at sometime because what , eh , he was looking at was performance in this room . professor b: Would that be more like Well , you 'd think that 'd be more like SpeechDat - Car , I guess , in terms of the noise .  , the signal - to - noise ratio , you know , looks a fair amount better if you if you high - pass filter it from this room . professor b: So , the main the the phd a: So that 's on th that 's on the f the far field ones though , right ? Yeah . phd a: So wha what is ,  what 's causing that ? professor b: Well , we got a a video projector in here , and ,  which we keep on during every every session we record , phd a: Yeah . professor b: which , you know , I I w we were aware of phd a:  - huh . professor b: But but ,  So , those are those are major components , I think , phd a: I see . professor b: but , it ,  I guess , I maybe I said this last week too but it it it really became apparent to us that we need to to take account of noise . And , so I think when when he gets done with his prelim study I think one of the next things we 'd want to do is to take this , noise , processing stuff and and , synthesize some speech from it . phd a: When are his prelims ? professor b: And then  , I think in about , a little less than two weeks .  , let 's see , this is the sixteenth , seventeenth ? Yeah , I don't know if he 's before It might even be in a week . I I guessed that they were gonna do it some time during the semester professor b: week and a half . phd a: but they 'll do it any time , huh ? professor b: They seem to be Well , the semester actually is starting up . phd a: Is it already ? professor b: Yeah , the semester 's late late August they start here .  , that that was sort of one  , the overall results seemed to be first place in in in the case of either , artificial reverberation or a modest sized training set . And But if you had a a really big training set , a recognizer , system that was capable of taking advantage of a really large training set I thought that One thing with the HTK is that is has the as we 're using the configuration we 're using is w s is being bound by the terms of Aurora , we have all those parameters just set as they are . So even if we had a hundred times as much data , we wouldn't go out to , you know , ten or t or a hundred times as many Gaussians or anything .  , so , that ,  that seemed to be So , if you have that that better recognizer that can that can build up more parameters , and if you , have the natural room , which in this case has a p a pretty bad signal - to - noise ratio , then in that case , the right thing to do is just do u use speaker adaptation . But I think that that would not be true if we did some explicit noise - processing as well as , the convolutional kind of things we were doing . phd d: That sample was released only yesterday or the day before , right ? phd a: No Well , I haven't grabbed that one yet . phd d: Oh , there is another short sample set phd a: There was another short one , yeah . phd a: And so I haven't grabbed the latest one that he just , put out yet . professor b: Is there any word yet about the issues about , adjustments for different feature sets or anything ? phd a: No , I I d You asked me to write to him and I think I forgot to ask him about that . phd d: Cuz they have it phd a: Maybe I 'll send it to the list . phd d: Cuz they have , already frozen those in i insertion penalties and all those stuff is what I feel . phd d: And , on that , they have run some experiments using various insertion penalties and all those phd a: And so they 've picked the values . professor b: For r w what test set ? phd d: p the one that they have reported is a NIST evaluation , Wall Street Journal . professor b: But that has nothing to do with what we 're testing on , right ? phd c:   So they 're , like  So they are actually trying to , fix that those values using the clean , training part of the Wall Street Journal . phd d: they want to train it and then this they 're going to run some evaluations . professor b: So they 're set they 're setting it based on that ? phd d: Yeah . So now , we may come back to the situation where we may be looking for a modification of the features to account for the fact that we can't modify these parameters . professor b:  but it 's still worth , I think , just since you know , just chatting with Joe about the issue . Do you think that 's something I should just send to him professor b:  phd a: or do you think I should send it to this there 's an a m a mailing list .  , we 're , you know , certainly willing to talk about it with everybody , but I think I think that , it 's probably best to start talking with him just to phd a: OK . professor b:  @ @ you know , it 's a dialogue between two of you about what you know , what does he think about this and what what you know what could be done about it . professor b: if you get ten people in involved in it there 'll be a lot of perspectives based on , you know , how phd a: Yeah . professor b: but if if if there is any , way to move in a way that would that would , you know , be more open to different kinds of features . But if if ,  if there isn't , and it 's just kind of shut down and and then also there 's probably not worthwhile bringing it into a larger forum where where political issues will come in . phd d: Because he there was some mail r saying that it 's may not be stable for Linux and all those . phd d: SUSI phd a: Yeah , SUSI or whatever it was phd d: yeah . professor b: but , I noticed , just glancing at the , Hopkins workshop , web site that ,  one of the thing I don't know Well , we 'll see how much they accomplish , but one of the things that they were trying to do in the graphical models thing was to put together a a , tool kit for doing ,  r  , arbitrary graphical models for , speech recognition . professor b: So And Jeff ,  the two Jeffs were phd a: Who 's the second Jeff ? professor b:  Oh , do you know Geoff Zweig ? phd a: No . professor b: so he did he did his PHD on dynamic Bayes - nets , for for speech recognition . He had some continuity built into the model , presumably to handle some , inertia in the in the production system , and ,  phd a:  .  , so it 's exactly the same approach , but the features that the VAD neural network use are , MFCC after noise compensation . professor b: What was it using before ? phd c: Before it was just P L phd d:  phd c: So . phd c: Yeah , phd d: Yeah , yeah , yeah , phd c: noisy noisy features . phd c:  This is what we get after This So , actually , we , yeah , here the features are noise compensated and there is also the LDA filter .  , and then it 's a pretty small neural network which use , nine frames of of six features from C - zero to C - fives , plus the first derivatives . phd a: Is that nine frames u s  , centered around the current frame ? Or phd c: Yeah . professor b: S so , I 'm I 'm sorry , there 's there 's there 's how many how many inputs ? phd c: So it 's twelve times nine . So I guess about eleven thousand parameters , which actually shouldn't be a problem , even in in small phones . phd a: So , I 'm I 'm s so what is different between this and and what you phd c: It should be OK . So the previous syst It 's based on the system that has a fifty - three point sixty - six percent improvement . The only thing that changed is the n a p eh a es the estimation of the silence probabilities . phd c:  So it 's it 's not bad , but the problem is still that the latency is too large . professor b: What 's the latency ? phd c: Because  the the latency of the VAD is two hundred and twenty milliseconds . And , the VAD is used  , i for on - line normalization , and it 's used before the delta computation . So if you add these components it goes t to a hundred and seventy , right ? professor b: I I 'm confused . You started off with two - twenty and you ended up with one - seventy ? phd c: With two an two hundred and seventy . phd c: If Yeah , if you add the c delta comp delta computation professor b: Oh . I the is this are these twenty - millisecond frames ? Is that why ? Is it after downsampling ? or phd c: The two - twenty is one hundred milliseconds for the  No , it 's forty milliseconds for t for the , cleaning of the speech . phd c: Yeah , and there is the delta at the input which is , professor b: One hundred milliseconds for smoothing . phd c:  phd d: It 's like forty plus forty plus professor b: And then forty phd c: Mmm . phd c: Yeah , there are twenty that comes from There is ten that comes from the LDA filters also . phd d: If you are using professor b:  phd c: Plus the frame , phd d: t If you are using three frames phd c: so it 's two - twenty . phd d: If you are phrasing f using three frames , it is thirty here for delta . So it 's like s five , six cepstrum plus delta at nine nine frames of professor b: And then ten milliseconds for phd d: Fi - There 's an LDA filter . professor b: ten milliseconds for LDA filter , and t and ten another ten milliseconds you said for the frame ? phd c: For the frame I guess . I computed two - twenty Yeah , well , it 's I guess it 's for the fr the professor b: OK . And then there 's delta besides that ? phd c: So this is the features that are used by our network and then afterwards , you have to compute the delta on the , main feature stream , professor b: OK . phd c: which is  , delta and double - deltas , which is fifty milliseconds . No , the after the noise part , the forty the the other hundred and eighty Well , Wait a minute . Some of this is ,  is ,  is in parallel , isn't it ?  , the LDA Oh , you have the LDA as part of the V D -  , VAD ? Or phd c: The VAD use , LDA filtered features also . professor b: so the delta at the end is how much ? phd c: It 's fifty . So phd c: But well , we could probably put the delta , before on - line normalization . It should not that make a big difference , phd a: What if you used a smaller window for the delta ? phd c: because phd a: Could that help a little bit ?  , I guess there 's a lot of things you could do to phd c: Yeah . phd c: Yeah , professor b: So phd c: but , nnn professor b: Yeah . So if you if you put the delta before the , ana on - line If Yeah phd c:   phd c: Cuz i professor b: And then y then you don't have that additive phd c: Yeah , phd d: Yep . phd c: cuz the time constant of the on - line normalization is pretty long compared to the delta window , professor b: OK . And you ought to be able to shove tw ,  sh  pull off twenty milliseconds from somewhere else to get it under two hundred , right ?  phd a: Is two hundred the d professor b: The hundred milla phd c:   It could be eighty and and probably do @ @ phd c: Yeah , phd a: i a hun phd c: yeah . phd a:  Wh - what 's the baseline you need to be under ? Two hundred ? professor b: Well , we don't know . professor b: if it 's two if if it 's ,  if it 's two - fifty , then we could keep the delta where it is if we shaved off twenty . If it 's two hundred , if we shaved off twenty , we could we could , meet it by moving the delta back . phd a: So , how do you know that what you have is too much if they 're still deciding ? professor b: we don't , but it 's just  , the main thing is that since that we got burned last time , and you know , by not worrying about it very much , we 're just staying conscious of it . professor b: And so , th  , if if if a week before we have to be done someone says , " Well , you have to have fifty milliseconds less than you have now " , it would be pretty frantic around here . professor b:  phd a: But still , that 's that 's a pretty big , win . And it doesn't seem like you 're in terms of your delay , you 're , that professor b: He added a bit on , I guess , because before we were we were had were able to have the noise , stuff , and the LVA be in parallel . phd c: Well , but I think the main thing , maybe , is the cleaning of the speech , which takes forty milliseconds or so . phd d: Yeah , the LDA LDA we don't know , is , like is it very crucial for the features , right ? phd c: No . professor b: so you could start pulling back , phd d: S s h professor b: but phd d: Yeah , professor b: But I think you have phd d: l professor b: you have twenty for delta computation which y now you 're sort of doing twice , right ? But yo w were you doing that before ? phd c: Mmm . Well , in the proposal , the input of the VAD network were just three frames , I think . professor b: So , what you have now is fort  , forty for the the noise , twenty for the delta , and ten for the LDA . That 's seventy milliseconds of stuff which was formerly in parallel , phd c:  professor b: right ? So I think , phd c:   professor b: you know , that 's that 's the difference as far as the timing , right ? phd c: Yeah . professor b: and you could experiment with cutting various pieces of these back a bit , but  , we 're s we 're not we 're not in terrible shape . professor b: It 's it 's not like it 's adding up to four hundred milliseconds or something . phd a: Where where is this where is this fifty - seven point O two in in comparison to the last evaluation ? professor b: Well , it 's I think it 's better than anything , anybody got . Yeah , and r and phd c: It would phd d: Yeah , so this is this is like the first proposal .  , I 'm sure other people working on this are not sitting still either , but phd a: Yeah . professor b: but but , the important thing is that we learn how to do this better , and , you know . So , our ,  Yeah , you can see the kind of kind of numbers that we 're having , say , on SpeechDat - Car which is a hard task , cuz it 's really ,  I think it 's just sort of sort of reasonable numbers , starting to be . Yeah , even for a well - matched case it 's sixty percent error rate reduction , professor b: Yeah . So actually , this is in between what we had with the previous VAD and what Sunil did with an IDL VAD . Which gave sixty - two percent improvement , right ? phd d: Yeah , it 's almost that . phd a: What was that ? Say that last part again ? phd c: So , if you use , like , an IDL VAD , for dropping the frames , phd d: o o Or the best we can get . phd c: the best that we can get i That means that we estimate the silence probability on the clean version of the utterances . phd a: So that would be even That wouldn't change this number down here to sixty - two ? phd c: Yeah . So you you were get phd c: If you add a g good v very good VAD , that works as well as a VAD working on clean speech , phd a: Yeah . phd c: then you wou you would go phd a: So that 's sort of the best you could hope for . professor b: And ,  and sixty - two with the the , you know , quote , unquote , cheating VAD . phd c: yeah , the next thing is , I started to play Well , I don't want to worry too much about the delay , no . And the other stream is the output of a neural network , using as input , also , these , cleaned MFCC .  phd a: Those are th those are th what is going into the tandem net ? phd c: I don't have the comp Mmm ? phd a: Those two ? phd c: So there is just this feature stream , the fifteen MFCC plus delta and double - delta . phd a: Yeah ? phd c: so it 's makes forty - five features that are used as input to the HTK . professor b: cuz then it has one part that 's discriminative , phd c: Yeah . Right now it seems that i I just tested on SpeechDat - Car while the experiment are running on your on TI - digits . Well , it improves on the well - matched and the mismatched conditions , but it get worse on the highly mismatched .  , professor b: y phd c: like , on the well - match and medium mismatch , the gain is around five percent relative , but it goes down a lot more , like fifteen percent on the HM case . professor b: You 're just using the full ninety features ? phd c:  The professor b: Y you have ninety features ? phd c: i I have ,  From the networks , it 's twenty - eight . It 's i i i It 's because it 's what we did for the first proposal . phd c: But we have to for sure , we have to go down , because the limit is now sixty features .  phd a: So , it seems funny that I don't know , maybe I don't u quite understand everything , but that adding features I guess I guess if you 're keeping the back - end fixed . But I guess if you 're keeping the number of Gaussians fixed in the recognizer , then professor b: Well , yeah . professor b: But , just in general , adding information Suppose the information you added , well , was a really terrible feature and all it brought in was noise . professor b: Right ? So so ,  Or or suppose it wasn't completely terrible , but it was completely equivalent to another one feature that you had , except it was noisier . professor b: Right ? In that case you wouldn't necessarily expect it to be better at all . I 'm just surprised that you 're getting fifteen percent relative worse on the wel professor b:  - huh . professor b: So , " highly mismatched condition " means that in fact your training is a bad estimate of your test . professor b: So having having , a g a l a greater number of features , if they aren't maybe the right features that you use , certainly can e can easily , make things worse . If you have if you have , lots and lots of data , and you have and your your your training is representative of your test , then getting more sources of information should just help . professor b: So I wonder , Well , what 's your what 's your thought about what to do next with it ? phd c: I don't know . I 'm surprised , because I expected the neural net to help more when there is more mismatch , as it was the case for the professor b:   phd c: Yeah , it 's the same training set , so it 's TIMIT with the TI - digits ' , noises , added . phd c:  professor b: Well , we might  , we might have to experiment with ,  better training sets . professor b: I The other thing is , before you found that was the best configuration , but you might have to retest those things now that we have different The rest of it is different , right ? So , For instance , what 's the effect of just putting the neural net on without the o other other path ? phd c:   professor b: You don't necessarily know what phd a: What if you did the Would it make sense to do the KLT on the full set of combined features ? Instead of just on the phd c: Yeah . The reason I did it this ways is that in February , it we we tested different things like that , so , having two KLT , having just a KLT for a network , or having a global KLT . phd c: And phd a: So you tried the global KLT before phd c: Well phd a: and it didn't really phd c: Yeah . phd c: The differences between these configurations were not huge , but it was marginally better with this configuration . professor b: And I guess if the These are all so all of these seventy - three features are going into , the ,  the  . professor b: And is are i i are are any deltas being computed of tha of them ? phd c: Of the straight features , yeah . phd c: So , yeah , maybe we can add some context from these features also as Dan did in in his last work . i Yeah , but the other thing I was thinking was , now I lost track of what I was thinking . phd a: What is the You said there was a limit of sixty features or something ? phd c:   phd a: What 's the relation between that limit and the , forty - eight  , forty eight hundred bits per second ? professor b: Oh , I know what I was gonna say . phd a: So I I I don't understand , phd c: The f the forty - eight hundred bits is for transmission of some features . phd a: because i  , if you 're only using h phd c: And generally , i it s allows you to transmit like , fifteen , cepstrum . professor b: The issue was that , this is supposed to be a standard that 's then gonna be fed to somebody 's recognizer somewhere which might be , you know , it it might be a concern how many parameters are use u used and so forth . What I was going to say is that , maybe maybe with the noise removal , these things are now more correlated . So you have two sets of things that are kind of uncorrelated , within themselves , but they 're pretty correlated with one another . professor b: And , they 're being fed into these , variants , only Gaussians and so forth , and and , phd c:   professor b: so maybe it would be a better idea now than it was before to , have , one KLT over everything , to de - correlate it . phd d: What are the S N Rs in the training set , TIMIT ? phd c: It 's , ranging from zero to clean ? Yeah . So we found this this ,  this Macrophone data , and so forth , that we were using for these other experiments , to be pretty good . professor b: So that 's i after you explore these other alternatives , that might be another way to start looking , is is just improving the training set . professor b: we were getting , lots better recognition using that , than Of course , you do have the problem that , u i we are not able to increase the number of Gaussians , or anything to , to match anything . So we 're only improving the training of our feature set , but that 's still probably something . phd a: So you 're saying , add the Macrophone data to the training of the neural net ? The tandem net ? professor b: Yeah , that 's the only place that we can train . professor b: We can't train the other stuff with anything other than the standard amount , phd a: Right .  ,  phd a: What what was it trained on again ? The one that you used ? phd c: It 's TIMIT with noise . phd c: So , yeah , it 's rather a small professor b: How big is the net , by the way ? phd c: it 's , five hundred hidden units . And professor b: And again , you did experiments back then where you made it bigger and it and that was that was sort of the threshold point . @ @ ? phd d: So is it is it though the performance , big relation in the high ma high mismatch has something to do with the , cleaning up that you that is done on the TIMIT after adding noise ? phd c:  phd d: So it 's i All the noises are from the TI - digits , phd c: Yeah . phd d: right ? So you i phd c:  They k  phd d: Well , it it 's like the high mismatch of the SpeechDat - Car after cleaning up , maybe having more noise than the the training set of TIMIT after clean s after you do the noise clean - up . phd d: So it had like all these different conditions of S N Rs , actually in their training set of neural net . phd d: But after cleaning up you have now a different set of S N Rs , right ? phd c: Yeah . phd d: And is it something to do with the mismatch that that 's created after the cleaning up , like the high mismatch phd c: You mean the the most noisy occurrences on SpeechDat - Car might be a lot more noisy than phd d:   professor b: but , you 're saying Yeah , the noisier ones are still going to be , even after our noise compensation , are still gonna be pretty noisy . phd d: Yeah , so now the after - noise compensation the neural net is seeing a different set of S N Rs than that was originally there in the training set . phd d: And that SNR may not be , like , com covering the whole set of S N Rs that you 're getting in the SpeechDat - Car . professor b: Right , but the SpeechDat - Car data that you 're seeing is also reduced in noise by the noise compensation . phd c: Well , if the initial range of SNR is different , we the problem was already there before . phd c: Because Mmm professor b: Yeah , it depends on whether you believe that the noise compensation is equally reducing the noise on the test set and the training set .  professor b: Right ?  , you 're saying there 's a mismatch in noise that wasn't there before , phd d:  . professor b: but if they were both the same before , then if they were both reduic reduced equally , then , there would not be a mismatch . professor b: So , this may be Heaven forbid , this noise compensation process may be imperfect , but . I I just that could be seen from the TI - digits , testing condition because , the noises are from the TI - digits , right ? Noise phd c: Yeah . So phd d: So cleaning up the TI - digits and if the performance goes down in the TI - digits mismatch high mismatch like this phd c: Clean training , yeah . professor b: the Macrophone data , I think , you know , it was recorded over many different telephones . I don't think there 's anybody recording over a car from a car , but I think it 's it 's varied enough that if if doing this adjustments , and playing around with it doesn't , make it better , the most  , it seems like the most obvious thing to do is to improve the training set .   , what we were  the condition It it gave us an enormous amount of improvement in what we were doing with Meeting Recorder digits , even though there , again , these m Macrophone digits were very , very different from , what we were going on here . But it was just I think just having a a nice variation in acoustic conditions was just a good thing . phd c: Yeah , actually to s eh , what I observed in the HM case is that the number of deletion dramatically increases . phd c: When I added the num the neural network it doubles the number of deletions . Yeah , so I don't you know how to interpret that , but , mmm professor b: Yeah . phd c: t phd a: And and did an other numbers stay the same ? Insertion substitutions stay the same ? phd c: They p stayed the same , phd a: Roughly ? phd c: they maybe they are a little bit  , lower . But professor b: Did they increase the number of deletions even for the cases that got better ? phd c:   professor b: And it Remind me again , the " highly mismatched " means that the phd c: Clean training and professor b: sorry ? phd c: It 's clean training Well , close microphone training and distant microphone , high speed , I think . professor b: Close mike training phd c: Well The most noisy cases are the distant microphone for testing . phd c: The feature are the same except that professor b: that 's right , that 's right .  phd a: Well that that says that , you know , the ,  the models in in , the recognizer are really paying attention to the neural net features . professor b: But , yeah , actually the TIMIT noises are sort of a range of noises and they 're not so much the stationary driving kind of noises , right ? It 's it 's pretty different . " phd c: " Subway " , right ? and phd d: " Street " or " Airport " or something . phd c: So it 's mostly Well , " Car " is stationary , professor b:   phd c: " Babble " , it 's a stationary background plus some voices , professor b:   professor b: Well , I I think that if you run it Actually , you maybe you remember this . When you in in the old experiments when you ran with the neural net only , and didn't have this side path , with the the pure features as well , did it make things better to have the neural net ? phd c:   professor b: Was it about the same ?  , w i phd c: It was b a little bit worse . professor b: So , until you put the second path in with the pure features , the neural net wasn't helping at all . as soon as we added LDA on - line normalization , and all these things , then professor b: They were doing similar enough things . Well , I still think it would be k sort of interesting to see what would happen if you just had the neural net without the side thing . phd c: Yeah , professor b: And and the thing I I have in mind is , maybe you 'll see that the results are not just a little bit worse . You know ? And ,  But if on the ha other hand , it 's , say , somewhere in between what you 're seeing now and and and , what you 'd have with just the pure features , then maybe there is some problem of a of a , combination of these things , or correlation between them somehow . professor b: If it really is that the net is hurting you at the moment , then I think the issue is to focus on on , improving the the net . professor b: So what 's the overall effe  , you haven't done all the experiments but you said it was i somewhat better , say , five percent better , for the first two conditions , and fifteen percent worse for the other one ? But it 's but of course that one 's weighted lower , phd c: Y yeah , oh . That 's not that bad , but it was l like two percent relative worse on SpeechDat - Car . phd d: Well , it will overall it will be still better even if it is fifteen percent worse , because the fifteen percent worse is given like f w twenty - five point two five eight . So the so the worst it could be , if the others were exactly the same , is four , phd d: Is it like professor b: and and , in fact since the others are somewhat better phd d: Yeah , so it 's four . Is i So either it 'll get cancelled out , or you 'll get , like , almost the same . phd a: In the ,  a lot of the ,  the Hub - five systems , recently have been using LDA . So there 's the the LDA is is right there before the H M phd d: Yeah . phd a: So , you guys are using LDA but it seems like it 's pretty far back in the process . The LDA that you saying is , like , you take a block of features , like nine frames or something , and then do an LDA on it , phd a: Yeah . phd a: it 's you know , you 're just basically i phd d: Yeah , so this is like a two d two dimensional tile . And the LDA that we are f applying is only in time , not in frequency high cost frequency . So it 's like more like a filtering in time , rather than doing a r phd a: Ah . So what i what about ,  i u what i w  , I don't know if this is a good idea or not , but what if you put ran the other kind of LDA , on your features right before they go into the  ? phd d: it phd c:   phd a: But  , w but the other features that you have , th the non - tandem ones , phd c:  . Well , in the proposal , they were transformed u using PCA , but phd a:  - huh . professor b: The a the argument i is kind of i in and it 's not like we really know , but the argument anyway is that , we always have the prob  , discriminative things are good . professor b: they 're good because you you you learn to distinguish between these categories that you want to be good at distinguishing between . It PAC - PCA low - order PCA throws away pieces that are  , maybe not not gonna be helpful just because they 're small , basically . professor b: But , the problem is , training sets aren't perfect and testing sets are different . So you f you you face the potential problem with discriminative stuff , be it LDA or neural nets , that you are training to discriminate between categories in one space but what you 're really gonna be g getting is is something else . professor b: And so , Stephane 's idea was , let 's feed , both this discriminatively trained thing and something that 's not . So you have a good set of features that everybody 's worked really hard to make , phd a: Yeah . professor b: and then , you you discriminately train it , but you also take the path that that doesn't have that , phd a:  - huh . And that that seem So it 's kind of like a combination of the  , what , Dan has been calling , you know , a feature  , you know , a feature combination versus posterior combination or something . It 's it 's , you know , you have the posterior combination but then you get the features from that and use them as a feature combination with these these other things . And that seemed , at least in the last one , as he was just saying , he he when he only did discriminative stuff , i it actually was was it didn't help at all in this particular case . But by having them both there The fact is some of the time , the discriminative stuff is gonna help you . professor b: And some of the time it 's going to hurt you , phd a: Right . professor b: and by combining two information sources if , you know if if phd a: So you wouldn't necessarily then want to do LDA on the non - tandem features because now you 're doing something to them that professor b: That i i I think that 's counter to that idea . But if that 's the hypothesis , at least it would be counter to that hypothesis to do that . professor b: and in principle you would think that the neural net would do better at the discriminant part than LDA .  , we ,  we were getting ready to do the tandem , stuff for the Hub - five system , and , Andreas and I talked about it , and the idea w the thought was , " Well , yeah , that i you know th the neural net should be better , but we should at least have  , a number , you know , to show that we did try the LDA in place of the neural net , so that we can you know , show a clear path . phd a: You know , that you have it without it , then you have the LDA , then you have the neural net , and you can see , theoretically . professor b: or tha that 's a phd a: That 's what that 's what we 're gonna do next as soon as I finish this other thing . phd a: it everybody believes it , professor b: Oh , no it 's a g phd a: but you know , we just professor b: No , no , but it might not not even be true . professor b: it 's it 's it 's it 's it 's a great idea .  , one of the things that always disturbed me , in the the resurgence of neural nets that happened in the eighties was that , a lot of people Because neural nets were pretty easy to to use a lot of people were just using them for all sorts of things without , looking at all into the linear , versions of them . professor b: And , people were doing recurrent nets but not looking at IIR filters , and You know , so I think , yeah , it 's definitely a good idea to try it . phd a: Yeah , and everybody 's putting that on their systems now , and so , I that 's what made me wonder about this , professor b: Well , they 've been putting them in their systems off and on for ten years , phd a: but . professor b: but but but , phd a: Yeah , what  is it 's it 's like in the Hub - five evaluations , you know , and you read the system descriptions and everybody 's got , you know , LDA on their features . phd c: It 's the transformation they 're estimating on Well , they are trained on the same data as the final  are . Cuz they don't have these , you know , mismatches that that you guys have . phd a: So that 's why I was wondering if maybe it 's not even a good idea . professor b: part of why I I think part of why you were getting into the KLT Y you were describing to me at one point that you wanted to see if , you know , getting good orthogonal features was and combining the the different temporal ranges was the key thing that was happening or whether it was this discriminant thing , right ? So you were just trying I think you r  , this is it doesn't have the LDA aspect but th as far as the orthogonalizing transformation , you were trying that at one point , right ? phd c:   phd d: So , yeah , I 've been exploring a parallel VAD without neural network with , like , less latency using SNR and energy , after the cleaning up . So what I 'd been trying was ,  After the b after the noise compensation , n I was trying t to f find a f feature based on the ratio of the energies , that is , cl after clean and before clean . So that if if they are , like , pretty c close to one , which means it 's speech . And if it is n if it is close to zero , which is So it 's like a scale @ @ probability value . So I was trying , with full band and multiple bands , m ps  separating them to different frequency bands and deriving separate decisions on each bands , and trying to combine them .  , the advantage being like it doesn't have the latency of the neural net if it if it can professor b:   phd d: just like Which means that it 's it 's doing a slightly better job than the previous VAD , professor b:   phd d: so ,  professor b: But i d I 'm sorry , phd d: so u professor b: does it still have the median filter stuff ? phd d: It still has the median filter . professor b: So it still has most of the delay , phd d: So professor b: it just doesn't phd d: Yeah , so d with the delay , that 's gone is the input , which is the sixty millisecond . professor b: Well , w i phd d: At the input of the neural net you have this , f nine frames of context plus the delta . phd d: so the delay is only the forty millisecond of the noise cleaning , plus the hundred millisecond smoothing at the output . So the the di the biggest The problem f for me was to find a consistent threshold that works well across the different databases , because I t I try to make it work on tr SpeechDat - Car professor b:   phd d: and it fails on TI - digits , or if I try to make it work on that it 's just the Italian or something , it doesn't work on the Finnish . So there are there was , like , some problem in balancing the deletions and insertions when I try different thresholds . phd d: So The I 'm still trying to make it better by using some other features from the after the p clean up maybe , some , correlation auto - correlation or some s additional features of to mainly the improvement of the VAD . professor b: Now this this this , " before and after clean " , it sounds like you think that 's a good feature . That that , it you th think that the ,  the i it appears to be a good feature , right ? phd d:   phd c: Yeah , eventually we could could just phd d: Yeah , so Yeah , so that 's the Yeah . phd d: Because they did that itself phd c: Then you don't have to worry about the thresholds and phd d: There 's a threshold and Yeah . So if we if we can live with the latency or cut the latencies elsewhere , then then that would be a , good thing . professor b: anybody has anybody you guys or or Naren , somebody , tried the , second th second stream thing ?  . phd d: Oh , I just I just h put the second stream in place and ,  ran one experiment , but just like just to know that everything is fine . phd d: So it was like , forty - five cepstrum plus twenty - three mel log mel . phd d: And and , just , like , it gave me the baseline performance of the Aurora , which is like zero improvement . phd d: So I just tried it on Italian just to know that everything is But I I didn't export anything out of it because it was , like , a weird feature set . Well , what I think , you know , would be more what you 'd want to do is is is , put it into another neural net . phd d: The  , other thing I was wondering was , if the neural net , has any because of the different noise con unseen noise conditions for the neural net , where , like , you train it on those four noise conditions , while you are feeding it with , like , a additional some four plus some f few more conditions which it hasn't seen , actually , phd c:   phd d:  instead of just h having c  , those cleaned up t cepstrum , sh should we feed some additional information , like The the We have the VAD flag .  , should we f feed the VAD flag , also , at the input so that it it has some additional discriminating information at the input ? phd c:  -  !  professor b: Wh -  , the the VAD what ? phd d: We have the VAD information also available at the back - end . phd d: So if it is something the neural net is not able to discriminate the classes professor b: Yeah . phd d:  Because most of it is sil  , we have dropped some silence f We have dropped so silence frames ? professor b:   So , by having an additional , feature which says " this is speech and this is nonspeech " , it certainly helps in some unseen noise conditions for the neural net . phd a: What Do y do you have that feature available for the test data ? phd d: Well , we have we are transferring the VAD to the back - end feature to the back - end . phd d: Which is which is certainly giving a phd a: So you 're saying , feed that , also , into the neural net . The other thing you could do is just , p modify the , output probabilities of the of the , neural net , tandem neural net , based on the fact that you have a silence probability . professor b: So you have an independent estimator of what the silence probability is , and you could multiply the two things , and renormalize . professor b: But but ,  phd c: Yeah , so maybe , yeah , when phd a: But in principle wouldn't it be better to feed it in ? And let the net do that ? professor b: Well , u Not sure . " Or you can say , " It 's silence ! Go away ! "  , i Doesn't ? I think I think the second one sounds a lot more direct . So , what if you then ,  since you know this , what if you only use the neural net on the speech portions ? professor b: Well , phd c: That 's what phd a: Well , I guess that 's the same . professor b: Yeah , y you 'd have to actually run it continuously , phd a: But   , train the net only on professor b: but it 's @ @ Well , no , you want to train on on the nonspeech also , because that 's part of what you 're learning in it , to to to generate , that it 's it has to distinguish between . phd a: But  , if you 're gonna if you 're going to multiply the output of the net by this other decision , would then you don't care about whether the net makes that distinction , right ? professor b: Well , yeah . Now the only thing that that bothers me about all this is that I I I The the fact i i It 's sort of bothersome that you 're getting more deletions . But So I might maybe look at , is it due to the fact that  , the probability of the silence at the output of the network , is , professor b: Is too high . So maybe So phd c: If it 's the case , then multiplying it again by i by something ? phd d: It may not be it professor b: Yeah . phd d: Yeah , it it may be too it 's too high in a sense , like , everything is more like a , flat probability . phd d: So , like , it 's not really doing any distinction between speech and nonspeech phd c: yeah . phd a: Be interesting to look at the Yeah , for the I wonder if you could do this . But if you look at the , highly mism high mismat the output of the net on the high mismatch case and just look at , you know , the distribution versus the the other ones , do you do you see more peaks or something ? phd c: Yeah . phd c: or professor b: But I bu phd c: It it seems that the VAD network doesn't Well , it doesn't drop , too many frames because the dele the number of deletion is reasonable . But it 's just when we add the tandem , the final MLP , and then professor b: Yeah . Now the only problem is you don't want to ta I guess wait for the output of the VAD before you can put something into the other system , phd c: u professor b: cuz that 'll shoot up the latency a lot , right ? Am I missing something here ? phd c: But phd d:   But but I I guess phd a: But if you were gonna put it in as a feature it means you already have it by the time you get to the tandem net , right ? phd d: well . professor b: It 's kind of done in  , some of the things are , not in parallel , but certainly , it would be in parallel with the with a tandem net . So maybe , if that doesn't work ,  But it would be interesting to see if that was the problem , anyway . And and and then I guess another alternative would be to take the feature that you 're feeding into the VAD , and feeding it into the other one as well . professor b:  But that 's Yeah , that 's an interesting thing to try to see , if what 's going on is that in the highly mismatched condition , it 's , causing deletions by having this silence probability up up too high , phd c:   phd c: So , m professor b: Cuz Well , the V A if the VAD said since the VAD is is is right a lot ,  phd c: Yeah . Yeah , and the other thing Well , there are other issues maybe for the tandem , like , well , do we want to , w  n Do we want to work on the targets ? Or , like , instead of using phonemes , using more context dependent units ? phd a: For the tandem net you mean ? phd c: Well , I 'm Yeah . phd c: I 'm thinking , also , a w about Dan 's work where he he trained a network , not on phoneme targets but on the  state targets . professor b: Problem is , if you are going to run this on different m test sets , including large vocabulary , phd c: Yeah . I was just thinking maybe about , like , generalized diphones , and come up with a a reasonable , not too large , set of context dependent units , and and Yeah . But I d I d it it i it 's all worth looking at , phd c:   professor b: but it sounds to me like , looking at the relationship between this and the speech noise stuff is is is probably a key thing . phd a: So if ,  if the , high mismatch case had been more like the , the other two cases in terms of giving you just a better performance , how would this number have changed ? phd c:   If if i phd a: y Like sixty ? professor b: Well , we don't know what 's it 's gonna be the TI - digits yet . If you extrapolate the SpeechDat - Car well - matched and medium - mismatch , it 's around , yeah , maybe five . phd c: Well , it 's around five percent , because it 's s Right ? If everything is five percent . phd c: I d I d I just have the SpeechDat - Car right now , so phd a: Yeah . phd c: It 's running it shou we should have the results today during the afternoon , phd a:  . Well  So I won't be here for phd a: When When do you leave ? professor b: I 'm leaving next Wednesday .  , phd a: But you 're professor b: so I phd a: are you you 're not gonna be around this afternoon ? professor b: Yeah . professor b: This afternoon  Oh , right , for the Meeting meeting ? Yeah , that 's just cuz of something on campus . But , yeah , so next week I won't , and the week after I won't , cuz I 'll be in Finland .  , phd a: What 's September sixth ? professor b: and  , that 's during Eurospeech .  Right ? So it 'll be a few weeks , really , before we have a meeting of the same cast of characters . And and then  , we 'll start up again with Dave and Dave and Barry and Stephane and us on the , twentieth . Thirteenth ? About a month ? phd a: So , you 're gonna be gone for the next three weeks or something ? professor b: I 'm gone for two and a half weeks starting starting next Wed - late next Wednesday . Is that right ? professor b: I won't it 's probably four because of is it three ? Let 's see , twenty - third , thirtieth , sixth . And the the third one won't probably won't be a meeting , cuz cuz , Su - Sunil , Stephane , and I will all not be here . So it 's just , the next two where there will be there , you know , may as well be meetings , phd a: OK . And then starting up on the thirteenth , we 'll have meetings again but we 'll have to do without Sunil here somehow . phd a: When is the evaluation ? November , or something ? professor b: Yeah , it was supposed to be November fifteenth <doc-sep>phd a: And what was that , Morgan ? What project ? professor b: the , TORRENT chip . professor b: we went through it Jim and I went through old emails at one point and and for two years there was this thing saying , yeah , we 're we 're two months away from being done . phd a: So , should we just do the same kind of deal where we go around and do , status report kind of things ? OK . phd a: Any objection ? Do y OK , M professor b: All in favor phd a: Do you want to start , Morgan ? Do you have anything , or ? professor b: I don't do anything . I No , I I 'm involved in discussions with with people about what they 're doing , but I think they 're since they 're here , they can talk about it themselves . Why don't you go ahead , Barry ? grad f: you 're gonna talk about Aurora stuff , per se ? phd a: OK . Well , this past week I 've just been , getting down and dirty into writing my my proposal . I just finished a section on ,  on talking about these intermediate categories that I want to classify , as a as a middle step . And , I hope to hope to get this ,  a full rough draft done by , Monday so I can give it to Morgan . phd a: So , is the idea you 're going to do this paper and then you pass it out to everybody ahead of time and ? grad f: Right , right . So , y you write up a proposal , and give it to people ahead of time , and you have a short presentation . phd a: Have you d ? I was just gonna ask , do you want to say any a little bit about it , grad f: Y s phd a: or ? Mmm .  , a little bit about ? phd a: Wh - what you 're what you 're gonna You said you were talking about the , particular features that you were looking at , grad f: Oh , the the phd a: or grad f: Right . Well , I was , I think one of the perplexing problems is , for a while I was thinking that I had to come up with a complete set of intermediate features in intermediate categories to to classify right away .  , build a system that , classifies these ,  these feat  , these intermediate categories using , multi - band techniques . Look at then I would look at the errors produced in the phoneme recognition and say , OK , well , I could probably reduce the errors if I included this extra feature or this extra intermediate category . phd a: So you 're gonna use TIMIT ? grad f: for that for that part of the the process , yeah , I would use TIMIT . grad f: that 's that 's ,  that 's just the ph the phone recognition task . So , I would mov I would then shift the focus to , something like Schw - Switchboard , where I 'd I would be able to ,  to model , intermediate categories that span across phonemes , phd a:   grad f: not just within the phonemes , themselves , and then do the same process there , on on a large vocabulary task like Switchboard .  , and for that for that part I would I 'd use the SRI recognizer since it 's already set up for for Switchboard . phd a: It 's about a month from now ? grad f: It 's a it 's a month and and a week . phd a: So , you want to go next , Dave ? And we 'll do grad e: Oh . And , we we got an improvement , in word error rate , training on the TI - digits data set and testing on Meeting Recorder digits of , six percent to four point five percent , on the n on the far - mike data using PZM F , but , the near - mike performance worsened , from one point two percent to two point four percent . And , wh why would that be , considering that we actually got an improvement in near - mike performance using HTK ? And so , with some input from , Andreas , I have a theory in two parts .  , first of all HTK sorry , SR - the SRI system is doing channel adaptation , and so HTK wasn't .  , so this ,  This mean subtraction approach will do a kind of channel normalization and so that might have given the HTK use of it a boost that wouldn't have been applied in the SRI case . So those finer - grained acoustic models could be more sensitive to the artifacts in the re - synthesized audio . And me and Barry were listening to the re - synthesized audio and sometimes it seems like you get of a bit of an echo of speech in the background . And so that seems like it could be difficult for training , cuz you could have different phones lined up with a different foreground phone , depending on the timing of the echo . So , I 'm gonna try training on a larger data set , and then , eh , the system will have seen more examples o of these artifacts and hopefully will be more robust to them . professor b: I had another thought just now , which is , remember we were talking before about we were talking in our meeting about , this stuff that some of the other stuff that Avendano did , where they were , getting rid of low - energy sections ?  , if you if you did a high - pass filtering , as Hirsch did in late eighties to reduce some of the effects of reverberation , Avendano and Hermansky were arguing that , perhaps one of the reasons for that working was ma may not have even been the filtering so much but the fact that when you filter a an all - positive power spectrum you get some negative values , and you gotta figure out what to do with them if you 're gonna continue treating this as a power spectrum . So if you imagine a a waveform that 's all positive , which is the time trajectory of energy , and , shifting it downwards , and then getting rid of the negative parts , that 's essentially throwing away the low - energy things . And it 's the low - energy parts of the speech where the reverberation is most audible . You know , you have the reverberation from higher - energy things showing up in So in this case you have some artificially imposed reverberation - like thing .  , you 're getting rid of some of the other effects of reverberation , but because you have these non - causal windows , you 're getting these funny things coming in , at n And , what if you did ?  , there 's nothing to say that the the processing for this re - synthesis has to be restricted to trying to get it back to the original , according to some equation . professor b: And one of the things you could do is , you could do some sort of VAD - like thing grad e:   professor b: and you actually could take very low - energy sections and set them to some some , very low or or near zero value .  , I 'm just saying if in fact it turns out that that these echoes that you 're hearing are ,  grad e:  - huh . professor b: or pre - echoes , whichever they are are are , part of what 's causing the problem , you actually could get rid of them . professor b: so that if you made a mistake you were more likely to keep in an echo than to throw out speech . phd g: what is the reverberation time like there ? grad e: In thi in this room ?  phd g: On , the the one what the s in the speech that you are you are using like ? grad e: Y Yeah . professor b: so it 's these are just microphone this micro close microphone and a distant microphone , he 's doing these different tests on . I think it 's I would guess , point seven , point eight seconds f  , R T grad f:  ! professor b: something like that ? But it 's you know , it 's this room . But the other thing is , he 's putting in w I was using the word " reverberation " in two ways . He 's also putting in , a he 's taking out some reverberation , but he 's putting in something , because he has averages over multiple windows stretching out to twelve seconds , which are then being subtracted from the speech . And since , you know , what you subtract , sometimes you 'll be you 'll be subtracting from some larger number and sometimes you won't . professor b: So you can end up with some components in it that are affected by things that are seconds away .  , and if it 's a low energy compo portion , you might actually hear some funny things . grad e: O o one thing , I noticed is that , the mean subtraction seems to make the PZM signals louder after they 've been re - synthesized . So I was wondering , is it possible that one reason it helped with the Aurora baseline system is just as a kind of gain control ? Cuz some of the PZM signals sound pretty quiet if you don't amplify them . I don't see why why your signal is louder after processing , because yo grad e: Yeah . professor b: I don't think just multiplying the signal by two would have any effect .  , I think if you really have louder signals , what you mean is that you have better signal - to - noise ratio . phd c: Well , well professor b: So if what you 're doing is improving the signal - to - noise ratio , then it would be better . professor b: But just it being bigger if with the same signal - to - noise ratio grad e: It w i i it wouldn't affect things . phd c: Well , the system is use the absolute energy , so it 's a little bit dependent on on the signal level . professor b: So if the if the if you change in both training and test , the absolute level by a factor of two , it will n have no effect . phd a: Did you add this data to the training set , for the Aurora ? Or you just tested on this ? grad e:   . Did I w what ? phd a: Well , Morgan was just saying that , as long as you do it in both training and testing , it shouldn't have any effect . But I didn't I 'm not sure if it made the clean ti TI - digits any louder . If it 's if it 's like , if it 's trying to find a a reverberation filter , it could be that this reverberation filter is making things quieter .  , there 's there 's nothing inherent about removing if you 're really removing , grad e: Nuh - huh . phd a: I wonder if there could be something like ,  for s for the PZM data , phd c: Eh phd a: you know , if occasionally , somebody hits the table or something , you could get a spike . I 'm just wondering if there 's something about the ,  you know , doing the mean normalization where , it it could cause you to have better signal - to - noise ratio . It it i maybe i If ,  Subtracting the the mean log spectrum is is is like dividing by the spectrum . So , depending what you divide by , if your if s your estimate is off and sometimes you 're you 're you 're getting a small number , you could make it bigger . professor b: So , it 's it 's just a a question of there 's It it could be that there 's some normalization that 's missing , or something to make it grad e:   professor b: y you 'd think it shouldn't be larger , but maybe in practice it is . So , you trained it on TI - digits ? But except this , it 's exactly the same system as the one that was tested before and that was trained on Macrophone . Right ? So on TI - digits it gives you one point two percent error rate and on Macrophone it 's still O point eight . grad e: If you 're talking about the Macrophone results that Andreas had about , a week and a half ago , I think it 's the same system . So you use VTL -  , vocal tract length normalization and , like MLLR transformations also , grad e:   phd c: and professor b: I 'm sorry , was his point eight percent , er , a a result on testing on Macrophone or or training ? phd c: all that stuff . grad e: That 's phd c: It was training on Macrophone and testing yeah , on on meeting digits . I I 've just been text testing the new Aurora front - end with well , Aurora system actually so front - end and HTK , acoustic models on the meeting digits and it 's a little bit better than the previous system . professor b: So , what w ? phd c: And phd g: With the with the HTK back - end ? What we have for Aurora ? phd c: Yeah . phd g: I know in the meeting , like phd c: On the meeting we have two point seven . phd c: we have the new LDA filters , and I think , maybe I didn't look , but one thing that makes a difference is this DC offset compensation .  , eh Do y did you have a look at at the meet  , meeting digits , if they have a DC component , or ? grad e: I I didn't .  , any all of the mikes have the DC removal some capacitor sitting right in that bias it . professor b: typi you know , unless Actually , there are instrumentation mikes that that do pass go down to DC . And you can get , I think it was I think it was in the Wall Street Journal data that that I can't remember , one of the DARPA things . professor b: we didn't we didn't know about for a while , while we were messing with it . And , the interesting thing that I tried was , Adam and Morgan had this idea , since my original attempts to , take the mean of the phase spectra over time and normalize using that , by subtracting that off , didn't work .  , so , well , that we thought that might be due to , problems with , the arithmetic of phases . They they add in this modulo two pi way and , there 's reason to believe that that approach of taking the mean of the phase spectrum wasn't really mathematically correct . So , what I did instead is I took the mean of the FFT spectrum without taking the log or anything , and then I took the phase of that , and I subtracted that phase off to normalize . professor b: You see , all he has to do is go back and reverse what he did before , and he 's really got something . phd a: Well , could you take what was left over and then subtract that ? professor b: Ex - exactly . phd g: Oh , it 's professor b: Just listen very carefully to what I say and do the opposite . I 'm more interested in trying to figure out what 's still the difference between the SRI system and the Aurora system . So , I think I will maybe train , like , gender - dependent models , because this is also one big difference between the two systems .  , the other differences were the fact that maybe the acoustic models of the SRI are more SRI system are more complex . But , Chuck , you did some experiments with this and phd a: It didn't seem to help in the HTK system . professor b: Well , it sounds like they also have he he 's saying they have all these , different kinds of adaptation . phd a: Like they do ,  I 'm not sure how they would do it when they 're working with the digits , phd c: The vocal tr phd a: but , like , in the Switchboard data , there 's ,  conversation - side normalization for the non - C - zero components , phd c: Yeah . phd c: So , it might be it might be better with it might be worse if the channel is constant , phd a: Yeah . phd g: And the acoustic models are like - k triphone models or or is it the whole word ? phd c: SRI it 's it 's tr grad f: SRI . So there 's there 's these kind of , pooled models and and they can go out to all sorts of dependencies . professor b: They have tied states and I think I I I don't real I 'm talk I 'm just guessing here . And maybe see with Andreas if Well , I I don't know how much it helps , what 's the model . phd a: So so the n stuff on the numbers you got , the two point seven , is that using the same training data that the SRI system used and got one point two ? phd c: That 's right . grad e: You know , the the Aurora baseline is set up with these ,  this version of the clean training set that 's been filtered with this G - seven - one - two filter , and , to train the SRI system on digits S - Andreas used the original TI - digits , under U doctor - speech data TI - digits , which don't have this filter . professor b: So is that ?  , are are these results comparable ? So you you were getting with the , Aurora baseline something like two point four percent on clean TI - digits , when , training the SRI system with clean TR digits TI - digits . And , so , is your two point seven comparable , where you 're , using , the submitted system ? phd c: Yeah . grad e: W w it was one one point two phd c: Ye grad e: with the SRI system , professor b: I 'm sorry . professor b: OK , so the comparable number then ,  for what you were talking about then , since it was HTK , would be the  , two point f phd c: It was four point something . Right ? The HTK system with , b grad e: D d professor b: Oh , right , right , right , right . phd c: MFCC features grad e: Do you mean the b ? The baseline Aurora - two system , trained on TI - digits , tested on Meeting Recorder near , I think we saw in it today , and it was about six point six percent . And another thing I I maybe would like to do is to just test the SRI system that 's trained on Macrophone test it on , the noisy TI - digits , professor b: Yeah . But I wonder if it 's just because maybe Macrophone is acoustically closer to the meeting digits than than TI - digit is , which is TI - digits are very clean recorded digits professor b:   phd c: and phd a: You know , it would also be interesting to see ,  to do the regular Aurora test , phd c: f s phd a: but use the SRI system instead of HTK . phd c: So we don't professor b: You 'd have to train the SRI system with with all the different languages . phd a: So , like , comple professor b: It 'd be a lot of work . phd a: I guess the work would be into getting the the files in the right formats , or something . phd a: Because when you train up the Aurora system , you 're ,  you 're also training on all the data . professor b: That 's true , but I think that also when we 've had these meetings week after week , oftentimes people have not done the full arrange of things phd a:   professor b: because on on whatever it is they 're trying , because it 's a lot of work , even just with the HTK . professor b: So , it 's it 's a good idea , but it seems like it makes sense to do some pruning phd a:   professor b: first with a a test or two that makes sense for you , phd a: Yeah . But , just testing on TI - digits would already give us some information about what 's going on .  , the next thing is this this VAD problem that ,  So , I 'm just talking about the the curves that I I sent I sent you so , whi that shows that when the SNR decrease , the current VAD approach doesn't drop much frames for some particular noises , which might be then noises that are closer to speech , acoustically . I They were supp Supposedly , in the next evaluation , they 're going to be supplying us with boundaries . First of all , the boundaries might be ,  like we would have t two hundred milliseconds or before and after speech . professor b: Do we ?  , is there some reason that we think that 's the case ? phd c: And No . professor b: But maybe we 'll get some insight on that when when , the gang gets back from Crete . professor b: And then the thing is if if they really are going to have some means of giving us fairly tight , boundaries , then that won't be so much the issue . phd g: Because w we were wondering whether that VAD is going to be , like , a realistic one or is it going to be some manual segmentation . And then , like , if if that VAD is going to be a realistic one , then we can actually use their markers to shift the point around , the way we want professor b:   phd g: to find a  , rather than keeping the twenty frames , we can actually move the marker to a point which we find more suitable for us . phd g: But if that is going to be something like a manual , segmenter , then we can't use that information anymore , phd c:   phd g: because that 's not going to be the one that is used in the final evaluation . There 's an  , I think it 's still for even for the evaluation , it might still be interesting to work on this because the boundaries apparently that they would provide is just , starting of speech and end of speech  , at the utterance level . phd c: So phd g: with some pauses in the center , provided they meet that whatever the hang - over time which they are talking . professor b: So if you could get at some of that ,  phd c: So professor b: although that 'd be hard . It might be useful for , like , noise estimation , and a lot of other things that we want to work on . So I did I just started to test putting together two VAD which was was not much work actually .  , I im re - implemented a VAD that 's very close to the , energy - based VAD that , the other Aurora guys use . phd c: and , detect detecting the first group of four frames that have a energy that 's above this threshold , and , from this point , tagging the frames there as speech . And it really removes it , still o on the noises where our MLP VAD doesn't work a lot . phd c: professor b: Cuz I would have thought that having some kind of spectral information , phd c: and professor b: you know , in the old days people would use energy and zero crossings , for instance  , would give you some better performance . Right ? Cuz you might have low - energy fricatives or or ,  stop consonants , or something like that . So , your point is will be to u use whatever professor b: Oh , that if you d if you use purely energy and don't look at anything spectral , then you don't have a good way of distinguishing between low - energy speech components and nonspeech . professor b: just as a gross generalization , most nonsp many nonspeech noises have a low - pass kind of characteristic , some sort of slope . And and most , low - energy speech components that are unvoiced have a a high - pass kind of characteristic phd c:   professor b: you know , at the beginning of a of a of an S sound for instance , just starting in , it might be pretty low - energy , phd c:   Whereas , a a lot of rumble , and background noises , and so forth will be predominantly low - frequency .  , you know , by itself it 's not enough to tell you , but it plus energy is sort of phd c: Yeah . professor b: if you look up in Rabiner and Schafer from like twenty - five years ago or something , that 's sort of what they were using then . It it might be that what I did is so , removes like low ,  low - energy , speech frames . Because the way I do it is I just I just combine the two decisions so , the one from the MLP and the one from the energy - based with the with the and operator . So if the energy - based dropped dropped low - energy speech , mmm , they they are they are lost . phd c: But s still , the way it 's done right now it it helps on on the noises where it seems to help on the noises where our VAD was not very good . But but , I guess what you 're saying is that the the MLP - based one has the spectral information . professor b: Well , you can imagine phd c: The way I use a an a " AND " operator is So , it I ,  professor b: Is ? phd c: The frames that are dropped by the energy - based system are are , dropped , even if the , MLP decides to keep them . phd a: No professor b: but  , I guess in principle what you 'd want to do is have a  , a probability estimated by each one and and put them together . phd a: Something that that I 've used in the past is ,  when just looking at the energy , is to look at the derivative . phd a: But , I 'm I 'm trying to remember if that requires that you keep some amount of speech in a buffer . phd c: Well , actually if I don't maybe don't want to work too much of on it right now . I just wanted to to see if it 's what I observed was the re was caused by this this VAD problem .  , which I 've just started yesterday to launch a bunch of , twenty - five experiments , with different , values for the parameters that are used . So , it 's the Makhoul - type spectral subtraction which use an over - estimation factor . So , we substr I subtract more , noise than the noise spectra that is estimated on the noise portion of the s  , the utterances . And after subtraction , I also add a constant noise , and I also try different , noise , values and we 'll see what happen . But st still when we look at the ,  Well , it depends on the parameters that you use , but for moderate over - estimation factors and moderate noise level that you add , you st have a lot of musical noise . On the other hand , when you subtract more and when you add more noise , you get rid of this musical noise but maybe you distort a lot of speech . So the next thing , maybe I what I will try to to do is just to try to smooth mmm , the ,  to smooth the d the result of the subtraction , to get rid of the musical noise , using some kind of filter , or phd g: Can smooth the SNR estimate , also . So , to get something that 's would be closer to what you tried to do with Wiener filtering . phd c: It phd g: And it 's phd c: Maybe you can phd g: go ahead . And there are there were some bugs in the program , so I was p initially trying to clear them up . Because one of the bug was I was assuming that always the VAD  , the initial frames were silence . So the it wasn't estimating the noise initially , and then it never estimated , because I assumed that it was always silence . phd c: So , in some cases s there are also phd g: SpeechDat - Car Italian . And , so once it was cleared , I ran a few experiments with different ways of smoothing the estimated clean speech and how t estimated the noise and , eh , smoothing the SNR also . And so the the trend seems to be like , smoothing the current estimate of the clean speech for deriving the SNR , which is like deriving the Wiener filter , seems to be helping . So we 'll have , like , a few results where the estimating the the More smoothing is helping . And , so I 'm I 'm trying a few more experiments with different time constants for smoothing the noise spectrum , and smoothing the clean speech , and smoothing SNR . So , one is fixed in the line , like Smoothing the clean speech is is helping , so I 'm not going to change it that much . But , the way I 'm estimating the noise and the way I 'm estimating the SNR , I 'm just trying trying a little bit . So , that h And the other thing is , like , putting a floor on the , SNR , because that if some In some cases the clean speech is , like when it 's estimated , it goes to very low values , so the SNR is , like , very low . So , I 'm thinking of , like , putting a floor also for the SNR so that it doesn't vary a lot in the low - energy regions . The results are , like So far I 've been testing only with the baseline , which is which doesn't have any LDA filtering and on - line normalization . So it 's just VAD , plus the Wiener filter , plus the baseline system , which is , just the spectral  , the mel sp mel , frequency coefficients . And the other thing that I tried was but I just took of those , Carlos filters , which Hynek had , to see whether it really h helps or not . And it 's it seems to be like it 's not hurting a lot by just blindly picking up one filter which is nothing but a four hertz a band - pass m m filter on the cubic root of the power spectrum . So there must be something that I can that can be done with that type of noise compensation also , which I guess I would ask Carlos about that .  , how how he derived those filters and and where d if he has any filters which are derived on OGI stories , added with some type of noise which what we are using currently , or something like that . So maybe I 'll professor b: This is cubic root of power spectra ? phd g: Yeah . professor b: So , if you have this band - pass filter , you probably get n you get negative values . phd g: So it has , like the spectrogram has , like  , it actually , enhances the onset and offset of  , the the begin and the end of the speech . So it 's there seems to be , like , deep valleys in the begin and the end of , like , high - energy regions , professor b:   phd g: So , those are the regions where there are , like when I look at the spectrogram , there are those deep valleys on the begin and the end of the speech . There are a few very not a lot of because the filter doesn't have a really a deep negative portion , so that it 's not really creating a lot of negative values in the cubic root . So , I 'll I 'll s may continue with that for some w I 'll I 'll Maybe I 'll ask Carlos a little more about how to play with those filters , and but while making this Wiener filter better . professor b: last week you were also talking about building up the subspace stuff ? phd g: Yeah . I I I would actually m m didn't get enough time to work on the subspace last week . phd a: How about you , Carmen ? phd d: Well , I am still working with , eh , VTS . And , one of the things that last week , eh , say here is that maybe the problem was with the diff because the signal have different level of energy . professor b:  ? phd d: And , maybe , talking with Stephane and with Sunil , we decide that maybe it was interesting to to apply on - line normalization before applying VTS . But then we decided that that 's it doesn't work absolutely , because we modified also the noise . I don't hav I don't this is I didn't do the experiment yet to apply VTS in cepstral domain . professor b: The other thing is So so , in i i and Not and C - zero would be a different So you could do a different normalization for C - zero than for other things anyway .  , the other thing I was gonna suggest is that you could have two kinds of normalization with with , different time constants . So , you could do some normalization s  , before the VTS , and then do some other normalization after . phd d: Well , we s decide to m to to obtain the new expression if we work in the cepstral domain . It 's k it 's k It 's quite a lot It 's a lot of work . phd d: And I want to know if if we have some feeling that the result I I would like to know if I don't have any feeling if this will work better than apply VTS aft in cepstral domain will work better than apply in m mel in filter bank domain . Well , you 're I think you 're the first one here to work with VTS , so , maybe we could call someone else up who has , ask them their opinion . phd c: Actually , the VTS that you tested before was in the log domain and so the codebook is e e kind of dependent on the level of the speech signal . phd d: Yeah ? phd c: And So I expect it If if you have something that 's independent of this , I expect it to it to , be a better model of speech . And then you have one number which is very dependent on the level cuz it is the level , phd d:   phd d: Ye phd c: Because it 's like first doing general normalization phd d: Yea phd c: and then noise removal , which is phd d: Yeah . We I was thinking to to to estimate the noise with the first frames and then apply the VAD , professor b:   phd d: We we see Well , I am thinking about that and working about that , professor b: Yeah .  , one of the things we 've talked about maybe it might be star time to start thinking about pretty soon , is as we look at the pros and cons of these different methods , how do they fit in with one another ? Because we 've talked about potentially doing some combination of a couple of them . Maybe maybe pretty soon we 'll have some sense of what their characteristics are , phd d:  -  
The professor expressed his frustration with the constant delays in the TORRENT chip schedule. Grad F presented his proposal, focusing on the section on intermediate categories and suggesting the inclusion of features from these categories as a means of reducing error. The professor was particularly interested in understanding the impact of the neural net on older models that lacked pure features. While acknowledging the potential for improvement, he also cautioned against overusing nets.
Summarize the discussion on the TORRENT schedule, intermediate categorization, and the professor's opinion on the neural net.