professor b: ok , so we we had a meeting with , with hynek , in which , sunil and stephane , summarized where they were and , talked about where we were gon na go . so that happened mid - week . phd a: what was the update ? so there is th then the all the new features that go in . the , noise suppression , the re - synthesis of speech after suppression . these are the phd e: is the , the cvs mechanism working ? are are people , up at ogi grabbing code , via that ? phd d: , i do n't think anybody up there is like working on it right now . professor b: it more likely that what it means is that when sunil is up there he will grab it . professor b: but what 'll happen is he 'll go back up there and , pratibha will come back from , the east coast . and , and i actually , after eurospeech for a little bit , he 'll go up there too . so , actually everybody who 's working on it will be up there for at least a little while . so they 'll remotely access it from there . phd a: actually i tried wh while when i installed the repository , i tried from belgium . i logged in there and i tried to import phd a: i do n't s i did n't set up you can also set up a cvs server on a new port . it 's like , a main server , or d you can do a cvs server . phd a: but i did n't do that because i was not about security problems . i would have to phd e: so w when you came in from belgian belgium , using ssh , was it asking you for your own password into icsi ? so if yo you can only do that if you have an account at icsi ? phd a: , you ha in this way you ca you have to set up a cvs server but then , you can access it . phd a: you can access them and mostly if you if y the set the server is set up like this . phd e: because a lot of the open source works with anonymous cvs and i ' m just wondering , for our transcripts we may want to do that . professor b: , for this i do n't think we 're quite up to that . , we 're still so much in development . phd e: , . , i was n't suggesting for this . i ' m thinking of the meeting recorder but . professor b: , maybe the thing to me might be i me i ' m you ' ve just been working on , details of that since the meeting , right ? and so phd a: mmm , since the meeting , i ' ve been train training a new vad and a new feature net . professor b: but i maybe the thing since you were n't yo you guys were n't at that meeting , might be just to , recap , the conclusions of the meeting . professor b: cuz that was , we 'd been working up to that , he would come here this week and we would since he 's going out of town like now , and i ' m going out town in a couple weeks , and time is marching , given all the mu many wonderful things we could be working on , what will we actually focus on ? and , and what do we freeze ? and , what do we ? so , . , this software that these guys created was certainly a key part . so then there 's something central and there are n't at least a bunch of different versions going off in ways that differ trivially . , and , professor b: and then within that , i the idea was to freeze a certain set of options for now , to run it , a particular way , and decide on what things are gon na be experimented with , as opposed to just experimenting with everything . so keep a certain set of things constant . , maybe describe roughly what we are keeping constant for now , or ? phd a: . so we ' ve been working like six weeks on the noise compensation and we end up with something that seems reasonable . phd a: so finally it 's , wiener filtering on fft bins . and it uses , two steps , smoothing of the transfer function , the first step , that 's along time , which use recursion . and after this step there is a further smoothing along frequency , which use a sliding window of twenty fft bins . mmm . and , professor b: this this smoothing is done on the estimate , of what you 're going to subtract ? or on the thing that has already had something subtracted ? phd a: , so we tried different configuration within this idea . we tried u applying this on mel bands , having spectral subtraction instead of wiener filtering . , finally we end up with this configuration that works , quite . so we are going to fix this for the moment and work on the other aspects of the whole system . professor b: actually , let me int , dave is n't here to talk about it , but let me just interject . this module , in principle , i , you would know whether it 's true , is somewhat independent from the rest of it . , because you re - synthesize speech , you do n't i you do n't re - synthesize speech , but you could phd a: , we do , but we do n't re - synthesize . in in the program we do n't re - synthesize and then re - analyze once again . we just use the clean fft bins . professor b: but you have a re - synthesized thing that you that 's an option here . professor b: , i gu i my point is that , i in some of the work he 's doing in reverberation , one of the things that we 're finding is that , it 's for the for an artificial situation , we can just deal with the reverberation and his techniques work really . but for the real situation , problem is , is that you do n't just have reverberation , you have reverberation in noise . and if you do n't include that in the model , it does n't work very . so it might be a very thing to do , to just take the noise removal part of it and put that in front of what he 's looking at . and , generate new files or whatever , and then do the reverberation part . so it 's professor b: , but , that 'll , it 's clear that we , we are not with the real case that we 're looking at , we ca n't just look at reverberation in isolation because the interaction between that and noise is considerable . and that 's , in the past we ' ve looked at , and this is hard enough , the interaction between channel effects and , and additive noise , so convolutional effects and additive effects . and that 's hard enough . , i do n't think we really , we 're trying to deal with that . in a sense that 's what we 're trying to deal with in this aurora task . and we have , the , lda that in principle is doing something about convolutional effects . and we have the noise suppression that 's doing something about noise . , even that 's hard enough . and and the on - line normalization as , in that s category . i there 's all these interactions between these two and that 's part of why these guys had to work so hard on juggling everything around . but now when you throw in the reverberation , it 's even worse , because not only do you have these effects , but you also have some long time effects . and , so dave has something which , is doing some things under some conditions with long time effects but when it 's when there 's noise there too , it 's pretty hard . so we have to start since any almost any real situation is gon na have , where you have the microphone distant , is going to have both things , we actually have to think about both at the same time . so there 's this noise suppression thing , which is worked out and then , maybe you should just continue telling what else is in the form we have . phd a: , the other parts of the system are the blocks that were already present before and that we did not modify a lot . professor b: so that 's again , that 's the wiener filtering , followed by , that 's done at the fft level . then professor b: the the filtering is done in the frequency domain ? , and then the mel and then the log , and then the phd a: followed by upsampling . then finally , we compute delta and we put the neural network also . professor b: right , and then in parallel with an a neural net . and then following neural net , some probably some orthogonalization . phd a: and finally frame dropping , which , would be a neural network also , used for estimated silence probabilities . and the input of this neural network would be somewhere between log mel bands or one of the earlier stages of the processing . professor b: so that 's most of this is , is operating parallel with this other . so the things that we , , i we , there 's there 's some , neat ideas for v a so , in there 's like there 's a bunch of tuning things to improve . there 's questions about various places where there 's an exponent , if it 's the right exponent , or ways that we 're estimating noise , that we can improve estimating noise . and there 's gon na be a host of those . but structurally it seemed like the things the main things that we brought up that , are gon na need to get worked on are , a significantly better vad , putting the neural net on , which , we have n't been doing anything with , the , neural net at the end there , and , the , opening up the second front . professor b: and , so the initial thing which came from , the meeting that we had down south was , that , we 'll initially just put in a mel spectrum as the second one . it 's , cheap , easy . there 's a question about exactly how we do it . we probably will go to something better later , but the initial thing is that cepstra and spectra behave differently , so . , tony robinson used to do i was saying this before . he used to do mel , spectra and mel cepstra . he used them as alternate features . put them together . phd e: so if you took the system the way it is now , the way it 's fro you 're gon na freeze it , and it ran it on the last evaluation , where it would it be ? professor b: although , you have n't tested it actually on the german and danish , have you ? phd e: so on the ones that you did test it on it would have been second ? professor b: would it but when you 're saying second , you 're comparing to the numbers that the , that the best system before got on , also without german and danish ? phd d: and th the ranking actually did n't change after the german and danish . so , . professor b: ranking did n't before , but i ' m just asking where this is to where theirs was without the german and danish , professor b: , we were also esp essentially second , although there were , we had a couple systems and they had a couple systems . and so , i by that we were third , but , there were two systems that were pretty close , that came from the same place . , so institutionally we were second , with , the third system . phd e: we 're so this second that you 're saying now is system - wide second ? phd d: their their first system is fifty - four point something . and , we are fifty - three point something . phd d: and their second system is also fifty - three point something . , one percent . professor b: and and , , in some sense we 're all doing fairly similar things . , one could argue about the lda and but , in a lot of ways we 're doing very similar things . but what phd e: so how did they fill up this all these bits ? , if we 're u professor b: , so , you guys are closer to it than me , so correct me if i ' m wrong , but that what 's going on is that in both cases , some normalization is done to deal with convola convolutional effects . , they have some cepstral modification , in our case we have a couple things . we have the on - line normalization and then we have the lda rasta . and they seem to comple complement each other enough and be different enough that they both seem to help us . but in any event , they 're both doing the same thing . but there 's one difference . the lda rasta , throws away high modulation frequencies . and they 're not doing that . phd e: so what if you did n't so do you explicitly downsample then ? do we explicitly downsample ? and what if we did n't do that ? would we get worse performance ? professor b: it does n't affect it , does it ? so , since we 're not evidently throwing away useful information , let 's try to put in some useful information . and , so i , we ' ve found in a lot of ways for quite a while that having a second stream , helps a lot . so that 's put in , and , it may even end up with mel spectrum even though i ' m saying we could do much better , just because it 's simple . and , in the long run having something everybody will look at and say , " , i understand " , is very helpful . phd e: so you would you 're you 're thinking to put the , mel spectrum in before any of the noise removal ? or after ? professor b: , that 's a question . , we were talking about that . it looks like it 'd be straightforward to , remove the noise , and , professor b: so , to do it after the mel conversion , after the noise removal , after the mel conversion . there 's even a question in my mind anyhow of whether th you should take the log or not . i think you should , but i . professor b: right . , but normalizing spectra instead of cepstra ? , probably . some kind would be good . ? i would think . phd d: , it so it actually makes it dependent on the overall energy of the , the frame . professor b: yes , so , one would think that you would want to normalize . but i w my thought is , particularly if you take the log , try it . and then if normalization helps , then y you have something to compare against , and say , " ok , this much effect " , you do n't want to change six things and then see what happens . you want to change them one at a time . professor b: so adding this other stream in , that 's simple in some way . and then saying , particularly because we ' ve found in the past there 's all these different results you get with slight modifications of how you do normalization . normalization 's a very tricky , sensitive thing and you learn a lot . so , i would think you would wanna have some baseline that says , " ok , we do n't normalize , this is what we get " , when we do this normalization , when we do that normalization . but but the other question is so ultimately we 'll wind up doing some normalization . i agree . professor b: we 're ta we 're pretty far out . so it 's just in terms of what data it 's depending on . it 's depending on the same data as the other . grad c: so with this , new stream would you train up a vad on both features , somehow ? phd d: , which could be this one of these streams , or it can be something derived from these streams . phd a: and there is also the idea of using traps , maybe , for the vad , which , phd a: , pratibha showed , when , she was at ibm , that it 's a good idea . professor b: , there 's the delays and the storage , but i do n't think the storage is so big for that . professor b: th the biggest we ' ve run into for storage is the neural net . and so i the issue there is , are we using neural - net - based traps , and how big are they ? so that 'll be , an issue . grad c: cuz she also does the , the correlation - based , traps , with without the neural net , just looking at the correlation between professor b: and maybe for vad they would be ok . that 's true . or a simple neural net , , if you 're doing correlation , you 're just doing a simple , dot product , with some weights which you happened to learn from this learn from the data . professor b: and so , putting a nonlinearity on it is , not that big a deal . it certainly does n't take much space . so , the question is , how complex a function do you need ? do you need to have an added layer ? in which case , potentially , it could be big . so , so what 's next ? maybe s remind us . phd e: so the meeting with hynek that you guys just had was to decide exactly what you were gon na freeze in this system ? is that ? or was there ? were you talking about what t new , professor b: what to freeze and then what to do after we froze . and like i was saying , the basic directions are , there 's lots of little things , such as improve the noise estimator but the bigger things are adding on the neural net and , the second stream . and then , improving the vad . phd d: so , i 'll actually after the meeting i 'll add the second stream to the vad and maybe i 'll start with the feature net in that case . it 's like , you 're looking at the vad , phd d: , you already have it ? ok , so just figure how to take the features from the final ok . phd a: but , there are plenty of issues to work on for the feature net @ . phd e: what about the , the new part of the evaluation , the , wall street journal part ? professor b: have you ever ? very good question . have you ever worked with the mississippi state h , software ? professor b: you may be called upon to help , on account of , all the work in this here has been , with small vocabulary . phd e: so what how is the , interaction supposed to happen ? , i remember the last time we talked about this , it was up in the air whether they were going to be taking , people 's features and then running them or they were gon na give the system out or , so they 're gon na just deliver a system . phd d: , th i it 's almost ready . so that 's what so they have released their , document , describing the system . professor b: cuz one of the things that might be helpful , if you ' ve got time in all of this is , is if these guys are really focusing on improving , all the digit , maybe and you got the front - end from them , maybe you could do the runs for the and , iron out hassles that you have to , tweak joe about or whatever , because you 're more experienced with running the large vocabulary . s phd d: so i 'll point you to the web site and the mails corresponding . so i phd d: , they are still , tuning something on that . so they 're like , d they 're varying different parameters like the insertion penalty and other , and then seeing what 's the performance . phd d: , w i there is , time during which people are gon na make suggestions . phd d: , i . so these sugges these this , period during which people are gon na make suggestions is to know whether it is actually biased towards any set of features or professor b: , so i th certainly the thing that i would want to know about is whether we get really hurt , on in insertion penalty , language model , scaling , sorts of things . professor b: , in which case , h hari or hynek will need to , push the case more about this . phd e: and we may be able to revisit this idea about , somehow modifying our features to work with professor b: yes . in this case , that 's right . that 's right . , some of that may be , a last minute rush thing because if the if our features are changing . but , . , the other thing is that even though it 's months away , it 's starting to seem to me now like november fifteenth is right around the corner . and , if they have n't decided things like this , like what the parameters are gon na be for this , when " deciding " is not just somebody deciding . , there should be some understanding behind the , deciding , which means some experiments and . it it seems pretty tight to me . professor b: that 's when the evaluation is . so , so after but , they may even decide in the end to push it off . it would n't , entirely surprise me . but , due to other reasons , like some people are going away , i ' m hoping it 's not pushed off for a l a long while . that would be , put us in an awkward position . but great . , that 'll be helpful . there 's there 's not anybody ogi currently who 's , working with this and phd e: is is this part of the evaluation just a small part , or ho how important is this to the overall ? professor b: , it 's conceptually , it my impression , again , you guys correct me if i ' m wrong , but my impression is that , they want it as a double check . that you have n't come across you have n't invented features which are actually gon na do badly for a significantly different task , particularly one with larger vocabulary . and , but it 's not the main emphasis . , the truth is , most of the applications they 're looking at are pretty small vocabulary . so it 's a double check . so they 'll probably assign it some low weight . phd e: seems to me that if it 's a double check , they should give you a one or a zero . y you passed the threshold or you did n't pass the threshold , and they should n't even care about what the score is . professor b: but , we 'll see what they come up with . , but in the current thing , where you have this - matched , moderately - matched , and mis highly - mismatched , the emphasis is somewhat on the - matched , but it 's only a marginal , it 's like forty , thirty - five , twenty - five , like that . so you still if you were way , way off on the highly - mismatched , it would have a big effect . and , it would n't surprise me if they did something like that with this . so again , if you 're if you get if it does n't help you much , for noisy versions of this of large vocabulary data , then , , it may not hurt you that much . but if it if you do n't if it does n't help you much , or to put it another way , if it helps some people a lot more than it helps other people , if their strategies do , then phd e: so is this , ? , guenter was putting a bunch of wall street journal data on our disks . phd e: so this test may take quite a while to run , then . may - judging by the amount of data that he was putting on . professor b: no , if it 's like the other things , there 's data for training the h m ms and data for testing it . so i would n't professor b: so , training the recognizer , but , but it 's trained on clean and is it trained on clean and test on ? phd a: , no . it 's training on a range between ten and twenty db , and testing between five and fifteen . that 's what i got on phd a: and so the noise is there is a range of different noises also which are selected randomly and added randomly , to the files . and there are noises that are different from the noises used on ti - digits . professor b: . , i would n't imagine that the amount of testing data was that huge . they probably put training , almost certain they put training data there too . maybe not . that 's that . anybody have anything else ? phd e: , one last question on that . when did they estimate that they would have that system available for download ? phd d: so i the f final system will be frozen by middle of , like , one more week maybe . professor b: , their they have a lot of options in their recognizer and the svm is one of the things they ' ve done with it , but it 's not their more standard thing . professor b: , the svm thing was an hmm also . it was just a it was like a hybrid , like phd e: so , just so that i understand , they 're providing scripts and everything so that , you push a button and it does training , and then it does test , and everything ? is that the idea ? phd d: i , i something like that . it 's like as painless as possible , is what do they provide all the scripts , everything , and then just , professor b: , . , if you look into it a little bit , it might be reasonable joe , just to ask him about the issue of , different features having different kinds of , scaling characteristics and so on . so that , w possibly having entirely different optimal values for the usual twiddle factors and what 's the plan about that ? phd d: so sh shall we , like , add chuck also to the mailing lists ? it may be better , in that case if he 's going to because there 's a mailing list for this . phd d: , i maybe hari or hynek , one of them , has to send a mail to joe . or maybe if you professor b: , so maybe just cc hari and say that you ' ve just been asked to handle the large vocabulary part here , and , , professor b: why do n't you just ask joe but cc hari , and then in the note say , " hari , hopefully this is ok with you " . and then if joe feels like he needs a confirmation , hari can answer it . that way you can get started asking joe quickly while he 's maybe still , putting in nails and screws and phd d: and there is an , archive of all the mails that has been gon that has gone , between these people among these people . so just you can see all this mails in the isip web site mississippi web site . professor b: have you thought about how long would be , most useful for you to go up to ogi ? phd a: i , we can for september , we can set up a work schedule and we can maybe work independently . and then at some point it maybe be better to work together again . professor b: , so you 're imagining more that you would come back here first for a while and then go up there ? professor b: i ju you guys are , y anyway , you do n't have to decide this second but thi think about it about what you would think would be the best way to work it . i 'll grad c: , i ' ve been reading some literature about clustering of data . just , i , let me put it in context . ok , so we 're talking about discovering intermediate categories to , to classify . and , i was looking at some of the work that , sangita was doing on these traps things . so she has , she has temporal patterns for , a certain set of phonemes , from timit , right ? the most common phonemes . and each one of them has a pattern over time , a one second window . and it has these patterns . , so she has , a trap for each one of the phonemes , times fifteen , for each of the fifteen critical bands . and , she does this agglomerative hierarchical clustering which , is a clustering algorithm that , starts with many , many different points many different clusters , corresponding to the number of data , patterns that you have in the data . and then you have this distance mej metric which , measures how closely related they are . and you start , by merging the patterns that are most closely related . phd e: and then you can pick , values anywhere along that tree to fix your set of clusters . grad c: right , usually it 's when , when the sol similarity measures , do n't go down as much . and so , so you stop at that point . and what she found was , sh , was there were five broad , broad categories , corresponding to , things like , fricatives and , vocalic , and , stops . and , one for silence and another one for schwa sounds . , and , i was thinking about ways to generalize this because w you 're it 's like a it 's not a completely automatic way of clustering , because yo beforehand you have these traps and you 're saying that these frames correspond to this particular phoneme . , and that 's constraining your clustering to the set of phonemes that you already have . , whereas maybe we want to just take a look at , arbitrary windows in time , of varying length , and cluster those . and i ' m thinking if we do that , then we would probably , at some point in the clustering algorithm find that we ' ve clustered things like , ok , thi this is a transition , this is a relatively stable point . , and i ' m hoping to find other things of similarity and maybe use these things as the intermediate , intermediate categories that , , i 'll later classify . grad c: , i have n't exactly figured out , the exact details for that but , the representation of the data that i was thinking of , was using , critical band , energies , over different lengths of time . so professor b: , it seems somehow that needs th , there 's a couple things that i wonder about with this . , so one is , again , looking at the same representation , if you 're going for this thing where you have , little detectors that are looking at narrow bands , then what you 're going to be looking for should be some category that you can find with the narrow bands . that that seems to be fundamental to it . , and then the other thing , is that i wonder about with it , and do n't take this in the wrong way , like i ' m doing or anything , but , . , just wondering really . , the standard answer about this thing is that if you 're trying to find the right system in some sense , whether you 're trying by categories or parameters , and your goal is discrimination , then having choices based on discrimination as opposed to , unsupervised nearness of things , is actually better . , and i if that , since you 're dealing with issues of robustness , maybe this is n't right , but it 'd be something i 'd be concerned about . because , you can imagine , i if you remember from , from your quals , john ohala saying that , " buh " and " puh " differed , not really cuz of voicing but because of aspiration . , in as far as wha what 's really there in the acoustics . so , if you looked if you were doing some coarse clustering , you probably would put those two sounds together . and yet , i would gue i would that many of your recognition errors were coming from , , pfft , screwing up on this distinction . so , it 's a little hard because recognizers , to first order , work . and the reasons we 're doing the things we 're doing is because they do n't work as we 'd like . and since they work , it means that they are already doing if you go and take any recognizer that 's already out there and you say , " how is it distinguishing between schwas and stops ? " boy , i bet they 're all doing nearly perfectly on this , so these big categories that differ in huge obvious ways , we already know how to do . so , what are we bringing to the party ? , what we wanna do is have something that , particularly in the presence of noise , is better at distinguishing between , categories that are actually close to one another , and hence , would probably be clustered together . professor b: so that 's th that 's the hard thing . , i understand that there 's this other constraint that you 're considering , is that you wanna have categories that , that would be straightforward for , say , a human being to mark if you had manual annotation . and it 's something that you really think you can pick up . but it 's also essential that you wanna look at what are the confusions that you 're making and how can you come up with , categories that , can clarify these confusions . so , the standard way of doing that is take a look at the algorithms you 're looking at , but then throw in some discriminative aspect to it . y y this is more like , how does lda differ from pca ? , they 're the same thing . they 're both orthogonalizing . but , and , this is a little harder because you 're not just trying to find parameters . you 're actually trying to find the categories themselves . , so a little more like brain surgery , on yourself . anyway . that 's my thought . you ' ve been thinking about this problem for a long time actually . , w actually , you stopped thinking about it for a long time , but you used to think about it a lot . and you ' ve been thinking about it more now , these categories . phd e: i do n't , it 's not clear to me how to reconcile , what you 're saying , which is right , with the way i ' ve been looking at it . that it 's all not very clear to me . but it seems to me that the desire the desirable feature to have is something that , is bottom - up . , however we do that . and and so i what i do n't understand is how to do that and still be discriminative , because to be discriminative you have to have categories and the only categories that we know of to use are these human sig significant categories that are significant to humans , like phonemes , things like that . but that 's what you want to avoid . and so that feels i how to get out of this . professor b: , here 's a , here 's a generic and possibly useless thought , which is , what do you really , in a sense the only s systems that make sense , are ones that have something from top - down in th in them . because if e even the smallest organism that 's trying to learn to do anything , if it does n't have any reward for doing or penal penalty for doing anything , then it 's just going to behave randomly . so whether you 're talking about something being learned through evolution or being learned through experience , it 's got ta have something come down to it that gives its reward or , at least some reinforcement learning , phd e: we could stop at words , but we do n't , we go all the way down to phonemes . professor b: right , but i me that maybe in some ways part of the difficulty is trying to deal with the with these phonemes . , and i it 's almost like you want categories if our , , metric of goodness , i if our correction if our metric of badness is word error rate then , maybe we should be looking at words . , for very , reasons we ' ve looked for a while at syllables , and they have a lot of good properties , but i if you go all the way to words , that 's really , d w in many applications you wanna go further . you wanna go to concepts , or have concepts , actions , this thing . phd e: , so the common right , the common wisdom is you ca n't do words because there 's too many of them , so you have to have some smaller set that you can use , and so everybody goes to phonemes . but the problem is that we build models of words in terms of phonemes and these models are really cartoon - ish , so when you look at conversational speech , you do n't see the phonemes that you have in your word models . professor b: but but we 're not trying for models of words here . see , so her here 's maybe where if the issue is that we 're trying to come up with , some intermediate categories which will then be useful for later , then maybe it does n't matter that we ca n't have enough , what you wanna do is build up these categories that are best for word recognition . and and somehow if that 's built into the loop of what the categories , we do this every day in this very gross way of running o a thousand experiments because we have fast computers and picking the thing that has the best word error rate . in some way , we derive that all the time . in some ways it 's really not a bad thing to do because it tells you how your adjustments at the very low level affect the final goal . , so maybe there 's a way to even put that in a much more automatic way , where you take , something about the error at the level of the word or some other it could be syllable but in some large unit , and , you may not have word models , you have phone models , whatever , but you do n't worry about that , and just somehow feed it back through . , so that 's , wh what i called a useless comments because i ' m not really telling you how to do it . but , it 's a it 's , it phd e: no , but the important part in there is that , if you want to be discriminative , you have to have , categories . and this the important categories are the words , and not the phones . maybe . and so if you can put the words in to the loop somehow for determining goodness of your sets of clusters professor b: now , that being said , that if you have something that is , i once you start dealing with spontaneous speech , all the things you 're saying are really true . if you have read speech that 's been manually annotated , like timit , then , i you the phones are gon na be right , actually , for the most part . so so , it does n't really hurt them to do that , to put in discrimination at that level . , if you go to spontaneous speech then it 's trickier and , the phones are , it 's gon na be based on bad pronunciation models that you have of and , and it wo n't allow for the overlapping phenomenon phd e: so it 's almost like there 's this mechanism that we have that , when we 're hearing read speech and all the phonemes are there , we deal with that , but when we go to conversational , and then all of a sudden not all the phonemes are there , it does n't really matter that much to us as humans because we have some mechanism that allows for these word models , whatever those models are , to be munged , and it does n't really hurt , and i ' m not how to build that in . professor b: , i the other thing i is to think of a little bit , we when y when you start looking at these results it usually is pretty intuitive , but start looking at , what are the kinds of confusions that you do make , , between words if you want or , even phones in read speech , say , when there is noise . , so is it more across place or more across manner ? or is it cor , is it ? , i know one thing that happens is that you , you lose the , , low energy phones . , if there 's added noise then low energy phones sometimes do n't get heard . and if that is if it , if that turns it into another word or different , or another pair of words , then it 's more likely to happen . but , i w i would that you 'd w i . anyway , that 's phd e: part of the difficulty is that a l a lot of the robustness that we have is probably coming from a much higher level . , we understand the context of the situation when we 're having a conversation . and so if there 's noise in there , our brain fills in and imagines what should be there . professor b: , that 's really big . , but , even if you do , diagnostic rhyme test things , where there really is n't an any information like that , people are still better in noise than they are in , than the machines are . so , that 's i right . we ca n't we ca n't get it without any language models . language models are there and important if we 're not working on that then we should work on something else and improve it , but especially if it looks like the potential is there . should we do some digits ? since we 're here ?
icsi's meeting recorder group at berkeley met mainly to discuss work on their main project , the aurora task , but also talked about the work of one of their student members. some members of the group met recently with research partners to settle on the current state of their software , and decide on the future work they would investigate , and these decisions were relayed to the rest of the group. of the three areas for the future , they touched mostly upon the use of a second , parallel , data stream. the group also discussed a new part to the evaluation , the use of a chunk of the wall street journal. speaker me006 is working on data clustering , and discussion of related issues led to more general acoustic matters. after the meeting , mn052 volunteered to get the second data stream up and running in the current software. since he has been asked to assist in a particular aspect of the work , me018 has to contact the relevant persons in order to be added to the according mailing list. there is a new system coming from ogi for dealing with the wall street journal data , and me013 wanted everyone to pay attention to areas where the groups might get hurt because of their features. the group's code has been updated , and in it's frozen state , runs at the second best level in the project. the cvs allowing access to the code is up and running , but now one at ogi is actually working on it yet. speaker me006 has been reading literature on data clustering , with particular attention to a previous work , and is contemplating how to generalise it.
###dialogue: professor b: ok , so we we had a meeting with , with hynek , in which , sunil and stephane , summarized where they were and , talked about where we were gon na go . so that happened mid - week . phd a: what was the update ? so there is th then the all the new features that go in . the , noise suppression , the re - synthesis of speech after suppression . these are the phd e: is the , the cvs mechanism working ? are are people , up at ogi grabbing code , via that ? phd d: , i do n't think anybody up there is like working on it right now . professor b: it more likely that what it means is that when sunil is up there he will grab it . professor b: but what 'll happen is he 'll go back up there and , pratibha will come back from , the east coast . and , and i actually , after eurospeech for a little bit , he 'll go up there too . so , actually everybody who 's working on it will be up there for at least a little while . so they 'll remotely access it from there . phd a: actually i tried wh while when i installed the repository , i tried from belgium . i logged in there and i tried to import phd a: i do n't s i did n't set up you can also set up a cvs server on a new port . it 's like , a main server , or d you can do a cvs server . phd a: but i did n't do that because i was not about security problems . i would have to phd e: so w when you came in from belgian belgium , using ssh , was it asking you for your own password into icsi ? so if yo you can only do that if you have an account at icsi ? phd a: , you ha in this way you ca you have to set up a cvs server but then , you can access it . phd a: you can access them and mostly if you if y the set the server is set up like this . phd e: because a lot of the open source works with anonymous cvs and i ' m just wondering , for our transcripts we may want to do that . professor b: , for this i do n't think we 're quite up to that . , we 're still so much in development . phd e: , . , i was n't suggesting for this . i ' m thinking of the meeting recorder but . professor b: , maybe the thing to me might be i me i ' m you ' ve just been working on , details of that since the meeting , right ? and so phd a: mmm , since the meeting , i ' ve been train training a new vad and a new feature net . professor b: but i maybe the thing since you were n't yo you guys were n't at that meeting , might be just to , recap , the conclusions of the meeting . professor b: cuz that was , we 'd been working up to that , he would come here this week and we would since he 's going out of town like now , and i ' m going out town in a couple weeks , and time is marching , given all the mu many wonderful things we could be working on , what will we actually focus on ? and , and what do we freeze ? and , what do we ? so , . , this software that these guys created was certainly a key part . so then there 's something central and there are n't at least a bunch of different versions going off in ways that differ trivially . , and , professor b: and then within that , i the idea was to freeze a certain set of options for now , to run it , a particular way , and decide on what things are gon na be experimented with , as opposed to just experimenting with everything . so keep a certain set of things constant . , maybe describe roughly what we are keeping constant for now , or ? phd a: . so we ' ve been working like six weeks on the noise compensation and we end up with something that seems reasonable . phd a: so finally it 's , wiener filtering on fft bins . and it uses , two steps , smoothing of the transfer function , the first step , that 's along time , which use recursion . and after this step there is a further smoothing along frequency , which use a sliding window of twenty fft bins . mmm . and , professor b: this this smoothing is done on the estimate , of what you 're going to subtract ? or on the thing that has already had something subtracted ? phd a: , so we tried different configuration within this idea . we tried u applying this on mel bands , having spectral subtraction instead of wiener filtering . , finally we end up with this configuration that works , quite . so we are going to fix this for the moment and work on the other aspects of the whole system . professor b: actually , let me int , dave is n't here to talk about it , but let me just interject . this module , in principle , i , you would know whether it 's true , is somewhat independent from the rest of it . , because you re - synthesize speech , you do n't i you do n't re - synthesize speech , but you could phd a: , we do , but we do n't re - synthesize . in in the program we do n't re - synthesize and then re - analyze once again . we just use the clean fft bins . professor b: but you have a re - synthesized thing that you that 's an option here . professor b: , i gu i my point is that , i in some of the work he 's doing in reverberation , one of the things that we 're finding is that , it 's for the for an artificial situation , we can just deal with the reverberation and his techniques work really . but for the real situation , problem is , is that you do n't just have reverberation , you have reverberation in noise . and if you do n't include that in the model , it does n't work very . so it might be a very thing to do , to just take the noise removal part of it and put that in front of what he 's looking at . and , generate new files or whatever , and then do the reverberation part . so it 's professor b: , but , that 'll , it 's clear that we , we are not with the real case that we 're looking at , we ca n't just look at reverberation in isolation because the interaction between that and noise is considerable . and that 's , in the past we ' ve looked at , and this is hard enough , the interaction between channel effects and , and additive noise , so convolutional effects and additive effects . and that 's hard enough . , i do n't think we really , we 're trying to deal with that . in a sense that 's what we 're trying to deal with in this aurora task . and we have , the , lda that in principle is doing something about convolutional effects . and we have the noise suppression that 's doing something about noise . , even that 's hard enough . and and the on - line normalization as , in that s category . i there 's all these interactions between these two and that 's part of why these guys had to work so hard on juggling everything around . but now when you throw in the reverberation , it 's even worse , because not only do you have these effects , but you also have some long time effects . and , so dave has something which , is doing some things under some conditions with long time effects but when it 's when there 's noise there too , it 's pretty hard . so we have to start since any almost any real situation is gon na have , where you have the microphone distant , is going to have both things , we actually have to think about both at the same time . so there 's this noise suppression thing , which is worked out and then , maybe you should just continue telling what else is in the form we have . phd a: , the other parts of the system are the blocks that were already present before and that we did not modify a lot . professor b: so that 's again , that 's the wiener filtering , followed by , that 's done at the fft level . then professor b: the the filtering is done in the frequency domain ? , and then the mel and then the log , and then the phd a: followed by upsampling . then finally , we compute delta and we put the neural network also . professor b: right , and then in parallel with an a neural net . and then following neural net , some probably some orthogonalization . phd a: and finally frame dropping , which , would be a neural network also , used for estimated silence probabilities . and the input of this neural network would be somewhere between log mel bands or one of the earlier stages of the processing . professor b: so that 's most of this is , is operating parallel with this other . so the things that we , , i we , there 's there 's some , neat ideas for v a so , in there 's like there 's a bunch of tuning things to improve . there 's questions about various places where there 's an exponent , if it 's the right exponent , or ways that we 're estimating noise , that we can improve estimating noise . and there 's gon na be a host of those . but structurally it seemed like the things the main things that we brought up that , are gon na need to get worked on are , a significantly better vad , putting the neural net on , which , we have n't been doing anything with , the , neural net at the end there , and , the , opening up the second front . professor b: and , so the initial thing which came from , the meeting that we had down south was , that , we 'll initially just put in a mel spectrum as the second one . it 's , cheap , easy . there 's a question about exactly how we do it . we probably will go to something better later , but the initial thing is that cepstra and spectra behave differently , so . , tony robinson used to do i was saying this before . he used to do mel , spectra and mel cepstra . he used them as alternate features . put them together . phd e: so if you took the system the way it is now , the way it 's fro you 're gon na freeze it , and it ran it on the last evaluation , where it would it be ? professor b: although , you have n't tested it actually on the german and danish , have you ? phd e: so on the ones that you did test it on it would have been second ? professor b: would it but when you 're saying second , you 're comparing to the numbers that the , that the best system before got on , also without german and danish ? phd d: and th the ranking actually did n't change after the german and danish . so , . professor b: ranking did n't before , but i ' m just asking where this is to where theirs was without the german and danish , professor b: , we were also esp essentially second , although there were , we had a couple systems and they had a couple systems . and so , i by that we were third , but , there were two systems that were pretty close , that came from the same place . , so institutionally we were second , with , the third system . phd e: we 're so this second that you 're saying now is system - wide second ? phd d: their their first system is fifty - four point something . and , we are fifty - three point something . phd d: and their second system is also fifty - three point something . , one percent . professor b: and and , , in some sense we 're all doing fairly similar things . , one could argue about the lda and but , in a lot of ways we 're doing very similar things . but what phd e: so how did they fill up this all these bits ? , if we 're u professor b: , so , you guys are closer to it than me , so correct me if i ' m wrong , but that what 's going on is that in both cases , some normalization is done to deal with convola convolutional effects . , they have some cepstral modification , in our case we have a couple things . we have the on - line normalization and then we have the lda rasta . and they seem to comple complement each other enough and be different enough that they both seem to help us . but in any event , they 're both doing the same thing . but there 's one difference . the lda rasta , throws away high modulation frequencies . and they 're not doing that . phd e: so what if you did n't so do you explicitly downsample then ? do we explicitly downsample ? and what if we did n't do that ? would we get worse performance ? professor b: it does n't affect it , does it ? so , since we 're not evidently throwing away useful information , let 's try to put in some useful information . and , so i , we ' ve found in a lot of ways for quite a while that having a second stream , helps a lot . so that 's put in , and , it may even end up with mel spectrum even though i ' m saying we could do much better , just because it 's simple . and , in the long run having something everybody will look at and say , " , i understand " , is very helpful . phd e: so you would you 're you 're thinking to put the , mel spectrum in before any of the noise removal ? or after ? professor b: , that 's a question . , we were talking about that . it looks like it 'd be straightforward to , remove the noise , and , professor b: so , to do it after the mel conversion , after the noise removal , after the mel conversion . there 's even a question in my mind anyhow of whether th you should take the log or not . i think you should , but i . professor b: right . , but normalizing spectra instead of cepstra ? , probably . some kind would be good . ? i would think . phd d: , it so it actually makes it dependent on the overall energy of the , the frame . professor b: yes , so , one would think that you would want to normalize . but i w my thought is , particularly if you take the log , try it . and then if normalization helps , then y you have something to compare against , and say , " ok , this much effect " , you do n't want to change six things and then see what happens . you want to change them one at a time . professor b: so adding this other stream in , that 's simple in some way . and then saying , particularly because we ' ve found in the past there 's all these different results you get with slight modifications of how you do normalization . normalization 's a very tricky , sensitive thing and you learn a lot . so , i would think you would wanna have some baseline that says , " ok , we do n't normalize , this is what we get " , when we do this normalization , when we do that normalization . but but the other question is so ultimately we 'll wind up doing some normalization . i agree . professor b: we 're ta we 're pretty far out . so it 's just in terms of what data it 's depending on . it 's depending on the same data as the other . grad c: so with this , new stream would you train up a vad on both features , somehow ? phd d: , which could be this one of these streams , or it can be something derived from these streams . phd a: and there is also the idea of using traps , maybe , for the vad , which , phd a: , pratibha showed , when , she was at ibm , that it 's a good idea . professor b: , there 's the delays and the storage , but i do n't think the storage is so big for that . professor b: th the biggest we ' ve run into for storage is the neural net . and so i the issue there is , are we using neural - net - based traps , and how big are they ? so that 'll be , an issue . grad c: cuz she also does the , the correlation - based , traps , with without the neural net , just looking at the correlation between professor b: and maybe for vad they would be ok . that 's true . or a simple neural net , , if you 're doing correlation , you 're just doing a simple , dot product , with some weights which you happened to learn from this learn from the data . professor b: and so , putting a nonlinearity on it is , not that big a deal . it certainly does n't take much space . so , the question is , how complex a function do you need ? do you need to have an added layer ? in which case , potentially , it could be big . so , so what 's next ? maybe s remind us . phd e: so the meeting with hynek that you guys just had was to decide exactly what you were gon na freeze in this system ? is that ? or was there ? were you talking about what t new , professor b: what to freeze and then what to do after we froze . and like i was saying , the basic directions are , there 's lots of little things , such as improve the noise estimator but the bigger things are adding on the neural net and , the second stream . and then , improving the vad . phd d: so , i 'll actually after the meeting i 'll add the second stream to the vad and maybe i 'll start with the feature net in that case . it 's like , you 're looking at the vad , phd d: , you already have it ? ok , so just figure how to take the features from the final ok . phd a: but , there are plenty of issues to work on for the feature net @ . phd e: what about the , the new part of the evaluation , the , wall street journal part ? professor b: have you ever ? very good question . have you ever worked with the mississippi state h , software ? professor b: you may be called upon to help , on account of , all the work in this here has been , with small vocabulary . phd e: so what how is the , interaction supposed to happen ? , i remember the last time we talked about this , it was up in the air whether they were going to be taking , people 's features and then running them or they were gon na give the system out or , so they 're gon na just deliver a system . phd d: , th i it 's almost ready . so that 's what so they have released their , document , describing the system . professor b: cuz one of the things that might be helpful , if you ' ve got time in all of this is , is if these guys are really focusing on improving , all the digit , maybe and you got the front - end from them , maybe you could do the runs for the and , iron out hassles that you have to , tweak joe about or whatever , because you 're more experienced with running the large vocabulary . s phd d: so i 'll point you to the web site and the mails corresponding . so i phd d: , they are still , tuning something on that . so they 're like , d they 're varying different parameters like the insertion penalty and other , and then seeing what 's the performance . phd d: , w i there is , time during which people are gon na make suggestions . phd d: , i . so these sugges these this , period during which people are gon na make suggestions is to know whether it is actually biased towards any set of features or professor b: , so i th certainly the thing that i would want to know about is whether we get really hurt , on in insertion penalty , language model , scaling , sorts of things . professor b: , in which case , h hari or hynek will need to , push the case more about this . phd e: and we may be able to revisit this idea about , somehow modifying our features to work with professor b: yes . in this case , that 's right . that 's right . , some of that may be , a last minute rush thing because if the if our features are changing . but , . , the other thing is that even though it 's months away , it 's starting to seem to me now like november fifteenth is right around the corner . and , if they have n't decided things like this , like what the parameters are gon na be for this , when " deciding " is not just somebody deciding . , there should be some understanding behind the , deciding , which means some experiments and . it it seems pretty tight to me . professor b: that 's when the evaluation is . so , so after but , they may even decide in the end to push it off . it would n't , entirely surprise me . but , due to other reasons , like some people are going away , i ' m hoping it 's not pushed off for a l a long while . that would be , put us in an awkward position . but great . , that 'll be helpful . there 's there 's not anybody ogi currently who 's , working with this and phd e: is is this part of the evaluation just a small part , or ho how important is this to the overall ? professor b: , it 's conceptually , it my impression , again , you guys correct me if i ' m wrong , but my impression is that , they want it as a double check . that you have n't come across you have n't invented features which are actually gon na do badly for a significantly different task , particularly one with larger vocabulary . and , but it 's not the main emphasis . , the truth is , most of the applications they 're looking at are pretty small vocabulary . so it 's a double check . so they 'll probably assign it some low weight . phd e: seems to me that if it 's a double check , they should give you a one or a zero . y you passed the threshold or you did n't pass the threshold , and they should n't even care about what the score is . professor b: but , we 'll see what they come up with . , but in the current thing , where you have this - matched , moderately - matched , and mis highly - mismatched , the emphasis is somewhat on the - matched , but it 's only a marginal , it 's like forty , thirty - five , twenty - five , like that . so you still if you were way , way off on the highly - mismatched , it would have a big effect . and , it would n't surprise me if they did something like that with this . so again , if you 're if you get if it does n't help you much , for noisy versions of this of large vocabulary data , then , , it may not hurt you that much . but if it if you do n't if it does n't help you much , or to put it another way , if it helps some people a lot more than it helps other people , if their strategies do , then phd e: so is this , ? , guenter was putting a bunch of wall street journal data on our disks . phd e: so this test may take quite a while to run , then . may - judging by the amount of data that he was putting on . professor b: no , if it 's like the other things , there 's data for training the h m ms and data for testing it . so i would n't professor b: so , training the recognizer , but , but it 's trained on clean and is it trained on clean and test on ? phd a: , no . it 's training on a range between ten and twenty db , and testing between five and fifteen . that 's what i got on phd a: and so the noise is there is a range of different noises also which are selected randomly and added randomly , to the files . and there are noises that are different from the noises used on ti - digits . professor b: . , i would n't imagine that the amount of testing data was that huge . they probably put training , almost certain they put training data there too . maybe not . that 's that . anybody have anything else ? phd e: , one last question on that . when did they estimate that they would have that system available for download ? phd d: so i the f final system will be frozen by middle of , like , one more week maybe . professor b: , their they have a lot of options in their recognizer and the svm is one of the things they ' ve done with it , but it 's not their more standard thing . professor b: , the svm thing was an hmm also . it was just a it was like a hybrid , like phd e: so , just so that i understand , they 're providing scripts and everything so that , you push a button and it does training , and then it does test , and everything ? is that the idea ? phd d: i , i something like that . it 's like as painless as possible , is what do they provide all the scripts , everything , and then just , professor b: , . , if you look into it a little bit , it might be reasonable joe , just to ask him about the issue of , different features having different kinds of , scaling characteristics and so on . so that , w possibly having entirely different optimal values for the usual twiddle factors and what 's the plan about that ? phd d: so sh shall we , like , add chuck also to the mailing lists ? it may be better , in that case if he 's going to because there 's a mailing list for this . phd d: , i maybe hari or hynek , one of them , has to send a mail to joe . or maybe if you professor b: , so maybe just cc hari and say that you ' ve just been asked to handle the large vocabulary part here , and , , professor b: why do n't you just ask joe but cc hari , and then in the note say , " hari , hopefully this is ok with you " . and then if joe feels like he needs a confirmation , hari can answer it . that way you can get started asking joe quickly while he 's maybe still , putting in nails and screws and phd d: and there is an , archive of all the mails that has been gon that has gone , between these people among these people . so just you can see all this mails in the isip web site mississippi web site . professor b: have you thought about how long would be , most useful for you to go up to ogi ? phd a: i , we can for september , we can set up a work schedule and we can maybe work independently . and then at some point it maybe be better to work together again . professor b: , so you 're imagining more that you would come back here first for a while and then go up there ? professor b: i ju you guys are , y anyway , you do n't have to decide this second but thi think about it about what you would think would be the best way to work it . i 'll grad c: , i ' ve been reading some literature about clustering of data . just , i , let me put it in context . ok , so we 're talking about discovering intermediate categories to , to classify . and , i was looking at some of the work that , sangita was doing on these traps things . so she has , she has temporal patterns for , a certain set of phonemes , from timit , right ? the most common phonemes . and each one of them has a pattern over time , a one second window . and it has these patterns . , so she has , a trap for each one of the phonemes , times fifteen , for each of the fifteen critical bands . and , she does this agglomerative hierarchical clustering which , is a clustering algorithm that , starts with many , many different points many different clusters , corresponding to the number of data , patterns that you have in the data . and then you have this distance mej metric which , measures how closely related they are . and you start , by merging the patterns that are most closely related . phd e: and then you can pick , values anywhere along that tree to fix your set of clusters . grad c: right , usually it 's when , when the sol similarity measures , do n't go down as much . and so , so you stop at that point . and what she found was , sh , was there were five broad , broad categories , corresponding to , things like , fricatives and , vocalic , and , stops . and , one for silence and another one for schwa sounds . , and , i was thinking about ways to generalize this because w you 're it 's like a it 's not a completely automatic way of clustering , because yo beforehand you have these traps and you 're saying that these frames correspond to this particular phoneme . , and that 's constraining your clustering to the set of phonemes that you already have . , whereas maybe we want to just take a look at , arbitrary windows in time , of varying length , and cluster those . and i ' m thinking if we do that , then we would probably , at some point in the clustering algorithm find that we ' ve clustered things like , ok , thi this is a transition , this is a relatively stable point . , and i ' m hoping to find other things of similarity and maybe use these things as the intermediate , intermediate categories that , , i 'll later classify . grad c: , i have n't exactly figured out , the exact details for that but , the representation of the data that i was thinking of , was using , critical band , energies , over different lengths of time . so professor b: , it seems somehow that needs th , there 's a couple things that i wonder about with this . , so one is , again , looking at the same representation , if you 're going for this thing where you have , little detectors that are looking at narrow bands , then what you 're going to be looking for should be some category that you can find with the narrow bands . that that seems to be fundamental to it . , and then the other thing , is that i wonder about with it , and do n't take this in the wrong way , like i ' m doing or anything , but , . , just wondering really . , the standard answer about this thing is that if you 're trying to find the right system in some sense , whether you 're trying by categories or parameters , and your goal is discrimination , then having choices based on discrimination as opposed to , unsupervised nearness of things , is actually better . , and i if that , since you 're dealing with issues of robustness , maybe this is n't right , but it 'd be something i 'd be concerned about . because , you can imagine , i if you remember from , from your quals , john ohala saying that , " buh " and " puh " differed , not really cuz of voicing but because of aspiration . , in as far as wha what 's really there in the acoustics . so , if you looked if you were doing some coarse clustering , you probably would put those two sounds together . and yet , i would gue i would that many of your recognition errors were coming from , , pfft , screwing up on this distinction . so , it 's a little hard because recognizers , to first order , work . and the reasons we 're doing the things we 're doing is because they do n't work as we 'd like . and since they work , it means that they are already doing if you go and take any recognizer that 's already out there and you say , " how is it distinguishing between schwas and stops ? " boy , i bet they 're all doing nearly perfectly on this , so these big categories that differ in huge obvious ways , we already know how to do . so , what are we bringing to the party ? , what we wanna do is have something that , particularly in the presence of noise , is better at distinguishing between , categories that are actually close to one another , and hence , would probably be clustered together . professor b: so that 's th that 's the hard thing . , i understand that there 's this other constraint that you 're considering , is that you wanna have categories that , that would be straightforward for , say , a human being to mark if you had manual annotation . and it 's something that you really think you can pick up . but it 's also essential that you wanna look at what are the confusions that you 're making and how can you come up with , categories that , can clarify these confusions . so , the standard way of doing that is take a look at the algorithms you 're looking at , but then throw in some discriminative aspect to it . y y this is more like , how does lda differ from pca ? , they 're the same thing . they 're both orthogonalizing . but , and , this is a little harder because you 're not just trying to find parameters . you 're actually trying to find the categories themselves . , so a little more like brain surgery , on yourself . anyway . that 's my thought . you ' ve been thinking about this problem for a long time actually . , w actually , you stopped thinking about it for a long time , but you used to think about it a lot . and you ' ve been thinking about it more now , these categories . phd e: i do n't , it 's not clear to me how to reconcile , what you 're saying , which is right , with the way i ' ve been looking at it . that it 's all not very clear to me . but it seems to me that the desire the desirable feature to have is something that , is bottom - up . , however we do that . and and so i what i do n't understand is how to do that and still be discriminative , because to be discriminative you have to have categories and the only categories that we know of to use are these human sig significant categories that are significant to humans , like phonemes , things like that . but that 's what you want to avoid . and so that feels i how to get out of this . professor b: , here 's a , here 's a generic and possibly useless thought , which is , what do you really , in a sense the only s systems that make sense , are ones that have something from top - down in th in them . because if e even the smallest organism that 's trying to learn to do anything , if it does n't have any reward for doing or penal penalty for doing anything , then it 's just going to behave randomly . so whether you 're talking about something being learned through evolution or being learned through experience , it 's got ta have something come down to it that gives its reward or , at least some reinforcement learning , phd e: we could stop at words , but we do n't , we go all the way down to phonemes . professor b: right , but i me that maybe in some ways part of the difficulty is trying to deal with the with these phonemes . , and i it 's almost like you want categories if our , , metric of goodness , i if our correction if our metric of badness is word error rate then , maybe we should be looking at words . , for very , reasons we ' ve looked for a while at syllables , and they have a lot of good properties , but i if you go all the way to words , that 's really , d w in many applications you wanna go further . you wanna go to concepts , or have concepts , actions , this thing . phd e: , so the common right , the common wisdom is you ca n't do words because there 's too many of them , so you have to have some smaller set that you can use , and so everybody goes to phonemes . but the problem is that we build models of words in terms of phonemes and these models are really cartoon - ish , so when you look at conversational speech , you do n't see the phonemes that you have in your word models . professor b: but but we 're not trying for models of words here . see , so her here 's maybe where if the issue is that we 're trying to come up with , some intermediate categories which will then be useful for later , then maybe it does n't matter that we ca n't have enough , what you wanna do is build up these categories that are best for word recognition . and and somehow if that 's built into the loop of what the categories , we do this every day in this very gross way of running o a thousand experiments because we have fast computers and picking the thing that has the best word error rate . in some way , we derive that all the time . in some ways it 's really not a bad thing to do because it tells you how your adjustments at the very low level affect the final goal . , so maybe there 's a way to even put that in a much more automatic way , where you take , something about the error at the level of the word or some other it could be syllable but in some large unit , and , you may not have word models , you have phone models , whatever , but you do n't worry about that , and just somehow feed it back through . , so that 's , wh what i called a useless comments because i ' m not really telling you how to do it . but , it 's a it 's , it phd e: no , but the important part in there is that , if you want to be discriminative , you have to have , categories . and this the important categories are the words , and not the phones . maybe . and so if you can put the words in to the loop somehow for determining goodness of your sets of clusters professor b: now , that being said , that if you have something that is , i once you start dealing with spontaneous speech , all the things you 're saying are really true . if you have read speech that 's been manually annotated , like timit , then , i you the phones are gon na be right , actually , for the most part . so so , it does n't really hurt them to do that , to put in discrimination at that level . , if you go to spontaneous speech then it 's trickier and , the phones are , it 's gon na be based on bad pronunciation models that you have of and , and it wo n't allow for the overlapping phenomenon phd e: so it 's almost like there 's this mechanism that we have that , when we 're hearing read speech and all the phonemes are there , we deal with that , but when we go to conversational , and then all of a sudden not all the phonemes are there , it does n't really matter that much to us as humans because we have some mechanism that allows for these word models , whatever those models are , to be munged , and it does n't really hurt , and i ' m not how to build that in . professor b: , i the other thing i is to think of a little bit , we when y when you start looking at these results it usually is pretty intuitive , but start looking at , what are the kinds of confusions that you do make , , between words if you want or , even phones in read speech , say , when there is noise . , so is it more across place or more across manner ? or is it cor , is it ? , i know one thing that happens is that you , you lose the , , low energy phones . , if there 's added noise then low energy phones sometimes do n't get heard . and if that is if it , if that turns it into another word or different , or another pair of words , then it 's more likely to happen . but , i w i would that you 'd w i . anyway , that 's phd e: part of the difficulty is that a l a lot of the robustness that we have is probably coming from a much higher level . , we understand the context of the situation when we 're having a conversation . and so if there 's noise in there , our brain fills in and imagines what should be there . professor b: , that 's really big . , but , even if you do , diagnostic rhyme test things , where there really is n't an any information like that , people are still better in noise than they are in , than the machines are . so , that 's i right . we ca n't we ca n't get it without any language models . language models are there and important if we 're not working on that then we should work on something else and improve it , but especially if it looks like the potential is there . should we do some digits ? since we 're here ?
###summary: icsi's meeting recorder group at berkeley met mainly to discuss work on their main project , the aurora task , but also talked about the work of one of their student members. some members of the group met recently with research partners to settle on the current state of their software , and decide on the future work they would investigate , and these decisions were relayed to the rest of the group. of the three areas for the future , they touched mostly upon the use of a second , parallel , data stream. the group also discussed a new part to the evaluation , the use of a chunk of the wall street journal. speaker me006 is working on data clustering , and discussion of related issues led to more general acoustic matters. after the meeting , mn052 volunteered to get the second data stream up and running in the current software. since he has been asked to assist in a particular aspect of the work , me018 has to contact the relevant persons in order to be added to the according mailing list. there is a new system coming from ogi for dealing with the wall street journal data , and me013 wanted everyone to pay attention to areas where the groups might get hurt because of their features. the group's code has been updated , and in it's frozen state , runs at the second best level in the project. the cvs allowing access to the code is up and running , but now one at ogi is actually working on it yet. speaker me006 has been reading literature on data clustering , with particular attention to a previous work , and is contemplating how to generalise it.