Can you  , say your name and talk into your mike one at a time ? phd c: This is Eric on channel three , I believe . postdoc e: Oh , darn , what am I doing wrong ? undergrad d: Can you see me on channel four ? Really ? grad a: Yeah , I s undergrad d: My lucky day . postdoc e: screen no , it is , oh , maybe it just warmed up ? grad a: No . postdoc e: Oh , darn , can you can't see channel five yet ? grad a: well , the mike isn't close enough to your mouth , so . postdoc e: Oh , this would be k OK , is that better ? grad a: S  , try speaking loudly , undergrad d: I like the high quality labelling . grad a: so , postdoc e: Hello , grad a: OK , good . undergrad d: David , can we borrow your labelling machine to improve the quality of the labelling a little bit here ? postdoc e: hello . phd b: One t undergrad d: How how many are there , one to five ? phd b: One five , yeah . postdoc e: Would you like to join the meeting ? grad a: Well , we don't wanna renumber them , postdoc e: I bet grad a: cuz we 've already have like , forms filled out with the numbers on them . grad a: OK , Dan , are you on ? phd b: I 'm on I 'm on two and I should be on . undergrad d: Want to join the meeting , Dave ? Do we do do we have a spare ,  grad a: And I 'm getting lots of responses on different ones , so I assume the various and assorted P Z Ms are on . undergrad d: We ' r we 're we ' r This is this this is a meeting meeting . postdoc e: This is abou we 're we 're mainly being taped but we 're gonna talk about , transcription for the m future meeting meetings . grad a: Right , Dan ? undergrad d: So , I don't understand if it 's neck mounted you don't get very good performance . grad a: What are you doing ? undergrad d: Cuz when you do this , you can Rouww - Rouww . postdoc e: Why didn't I you were saying that but I could hear you really well on the on the transcription on the , tape . grad a: Well , I m I would prefer that people wore it on their head phd b: I I don't know . grad a: It 's very badly designed so it 's phd b: It 's very badly designed ? undergrad d: What do you mean it doesn't go over the ears ? phd b: Why ? It 's not s It 's not supposed to cover up your ears . grad a: Yeah but , there 's nowhere to put the pad so it 's comfortable . phd b: it 's only badly postdoc e: So that 's what you 're d He 's got it on his temples so it cuts off his circulation . grad a: Well , you know , I 'm just that sort of digit - y g sorta guy . phd b: We didn't postdoc e: Now , just to be sure , the numbers on the back , this is the channel ? phd b: That 's the microphone number . phd b: Transcript number phd c: OK , this is Eric on microphone number three , undergrad d: This is Beck on mike four . Should I turn off the VU meter Dan ? Do you think that makes any difference ? phd b: Oh , God . grad a: Why ? Are you gonna do something other than hit " quit " ? phd b: No , but I 'm gonna look at the  , logs as well . postdoc e: you said turn off the what ? grad a: The VU meter which tells you what the levels on the various mikes are and there was one hypothesis that perhaps that the act of recording the VU meter was one of the things that contributed to the errors . undergrad d: Yeah , but Eric , you didn't think that was a reasonable hypothesis , right ? postdoc e: I See . grad a: That was me , undergrad d: Oh , I 'm sorry y grad a: I thought that was undergrad d: That was malarkey . undergrad d: Are - are yo are you recording where the table mikes are by the way ? phd b: No . undergrad d: Why not ? phd b: I think , I s I spoke to somebody , Morgan , about that . undergrad d: Why don't you just do this ? grad a: that 's what we 've done before . phd b: I know what they they 're they 're four , three , two , one . grad a: Which means we need to move this thing , and sorta decide how we 're actually going to do things . phd b: It 'll be a lot easier if we have a if we have them permanently in place or something like that . undergrad d: Is this b is this being recorded ? grad a: That 's right . undergrad d: I think Lila actually is almost getting r pretty close to even getting ready to put out the purchase order . grad a: OK , so , topic of this meeting is I wanna talk a little bit about transcription .  , I 've looked a little bit into commercial transcription services and Jane has been working on doing transcription .  , and so we wan wanna decide what we 're gonna do with that and then get an update on the electronics , and then , maybe also talk a little bit about some infrastructure and tools , and so on .  , you know , eventually we 're probably gonna wanna distribute this thing and we should decide how we 're gonna how we 're gonna handle some of these factors . phd b: Distribute what ? grad a:  ? phd b: The data ? grad a: Right .  , so we 're we 're collecting a corpus and I think it 's gonna be generally useful . grad a: and so we will undergrad d: u Using , like , audio D V Ds or something like that ? grad a: Excuse me ? phd b: Yes . Yeah , audio D V C Ds , undergrad d: Or t grad a: you know . grad a: And and so how we do we distribute the transcripts , how do we distribute the audio files , how do we how do we just do all that infrastructure ? phd c: Well , I think  , for that particular issue ther there are known sources where people go to to find these kind of things like the LDC for instance . postdoc e: Yeah , grad a: Right , but but so should we do it in the same format as LDC postdoc e: that 's right . grad a: and what does that mean to what we 've done already ? phd b: Right . The It 's not so much the actu The logistics of distribution are secondary to preparing the data in a suitable form for distribution . So , as it is , it 's sort of a ad - hoc combination of stuff Dan set and stuff I set up , which we may wanna make a little more formal . phd b: And the other thing is that , University of Washington may want to start recording meetings as well , grad a: Right . phd b: in which case w w we 'll have to decide what we 've actually got so that we can give them a copy . So I have a bunch of scripts with X Waves , and some Perl scripts , and other things that make it really easy to extract out and align where the digits are . And if U d UW 's going to do the same thing I think it 's worth while for them to do these digits tasks as well . grad a: And what I 've done is pretty ad - hoc , so we might wanna change it over to something a little more standard . undergrad d: An - and there 's interest up there ? grad a: What 's that ? undergrad d: There 's interest up there ? grad a: Well they they certainly wanna collect more data . grad a: And I would like that just to get a disjoint speaker set and a disjoint room .  , one of the things Morgan and I were talking about is we 're gonna get to know this room really well , phd c:   undergrad d: Did you notice the fan difference ? phd b: Oh , now you 've touched the fan control , now all our data 's gonna be undergrad d: Hear the difference ? grad a: Oh , it 's enormous . undergrad d: Do you wanna leave it off or not ? postdoc e: That 's better . phd b: That 's undergrad d: Yeah , the You sure ? phd b: Oh , yeah . phd b: So it could be that it 's not actually wired backwards undergrad d: That 's right . phd c: It would you know , it would real really mean that we should do short meetings when you turn off the turn off the air conditioning , grad a: Carbon monoxide poisoning ? undergrad d: Short meetings , that 's right . undergrad d: Actually , the a th air the air conditioning 's still working , that 's just an auxiliary fan . grad a: So phd c: So , in addition to this issue about the UW stuff there was announced today , via the LDC , a corpus from I believe Santa Barbara . phd c: And I don't know exactly how they recorded it but apparently there 's a lot of different styles of speech and what not . phd c: And postdoc e: They had people come in to a certain degree and they and they have DAT recorders . Right ? postdoc e: I I assume so , actually , I hadn't thought about that . Unless they added close field later on but , I 've listened to some of those data and I , I 've been I I was actually on the advisory board for when they set the project up . phd b: What 's it sound like ? postdoc e: I 'm glad to see that it got released . grad a: Yeah , I I wish postdoc e: So it it 's a very nice thing . grad a: I wish we had someone here working on adaptation phd c: S grad a: because it would nice to be able to take that stuff and adapt it to a meeting setting . You know phd c: But it may be it may be useful in postdoc e: How do you mean do you mean mechanical adaptation or grad a: No , software , to adapt the speech recognition . phd c: Well , what I was thinking is it may be useful in transcribing , if it 's far field stuff , grad a:   phd c: right ? In doing , some of our first automatic speech recognition models , it may be useful to have that kind of data postdoc e: Great idea . grad a: Well that 's not good , right ? phd c: That 's that 's not great . postdoc e: It sounds undergrad d: Tr postdoc e: well but what  is that ,  undergrad d: But far field means great distance ?  grad a: Just these . undergrad d: And so that 's why they 're getting away with just two channels or something , or are they using multiple DATs ? postdoc e: oh , good question and I can't ans answer it . grad a: So we can get that just with , media costs , undergrad d: Still a point . grad a: is that right ? phd c: in fact we get it for free grad a: Oh . phd c: So , I can I can actually arrange for it to arrive in short order if we 're postdoc e: The other thing too is from from a grad a: Well , it 's silly to do unless we 're gonna have someone to work on it , so maybe we need to think about it a little bit . postdoc e: The other thing too is that their their jus their transcription format is really nice and simple in in the discourse domain . Maybe we should maybe we should get a copy of it just to see what they did phd b: Yeah , absolutely . grad a: OK , why don't you go ahead and do that then Eric ? phd b: Absolutely . postdoc e: but , phd c: Well they 're postdoc e: this is clever . phd c: Apparently this was like phase one postdoc e: Got it through the LDC . phd c: and the there 's still more that they 're gonna do apparently or something like that unless of course they have funding issues postdoc e: Great . phd c: and then then it ma they may not do phase two but from all the web documentation it looked like , " oh , this is phase one " , whatever that means . Yeah , that  , they 're really well respected in the linguistics d side too and the discourse area , phd c: OK . phd c: But , it  it would also maybe help be helpful for Liz , if she wanted to start working on some discourse issues , you know , looking at some of this data and then , grad a: Right . grad a: Actually , that 's another thing I was thinking about is that maybe Jane should talk to Liz , to see if there are any transcription issues related to discourse that she needs to get marked . undergrad d: That would be a meeting meeting meeting ? grad a: A meeting meeting meeting . But maybe we should ,  find some day that Liz  , Liz and Andreas seem to be around more often . phd c: So maybe we should find a day when they 're gonna be here and and Morgan 's gonna be here , and we can meet , at least this subgroup . grad a: Well , I was even thinking that maybe we need to at least ping the U - dub to see phd c: We need we need to talk to them some more . grad a: you know , say " this is what we 're thinking about for our transcription " , if nothing else . So , well w shall we move on and talk a little bit about transcription then ? phd b:   What we 're using right now is a tool , from this French group , called " Transcriber " that seems to work very well .  , so it has a , nice useful Tcl - TK user interface and , undergrad d: Thi - this is the process of converting audio to text ? grad a: Right . So all all so what you have to do is just identify a segment of speech in time , and then write down what was said within it , and identify the speaker . And so the things we that we know that I know I want are the text , the start and end , and the speaker .  , and then things like repairs , and false starts , and , filled pauses , and all that other sort of stuff , we have to decide how much of that we wanna do . My my view on it was when you have a repair then ,  it seems  , we saw , there was this presentation in the one of the speech group meetings about how grad a:   postdoc e: and I think Liz has done some stuff too on that , that it ,  that you get it bracketed in terms of like well , if it 's parenthetical , which I know that Liz has worked on , then  y y you 'll have different prosodic aspects . postdoc e: And then also if it 's a r if it 's a repair where they 're like what I just did , then it 's nice to have sort of a sense of the continuity of the utterance , the start to be to the finish . And , it 's a little bit deceptive if you include the repai the pre - repair part and sometimes or of it 's in the middle . Anyway , so what I was doing was bracketing them to indicate that they were repairs which isn't  , very time - consuming . undergrad d: I is there already some sort of plan in place for how this gonna be staffed or done ? Or is it real is that what we 're talking about here ? grad a: Well , that 's part of the thing we 're talking about . So what we wanted to do was have Jane do basically one meeting 's worth , you know , forty minutes to an hour , postdoc e:   undergrad d: It this is this is like five times real time or ten times real time postdoc e: Yeah , as a pilot study . grad a: Ten times about , is and so one of the things was to get an estimate of how long it would take , and then also what tools we would use . And so the next decision which has to be made actually pretty soon is how are we gonna do it ? So . undergrad d: And so you make Jane do the first one so then she can decide , oh , we don't need all this stuff , just the words are fine . postdoc e: I wanna hear about these  , we have a g you were s continuing with the transcription conventions for s grad a: R right , so so one one option is to get linguistics grad students and undergrads to do it .  , it will require a post pass ,  people will have to look at it more than once to make sure that it 's been done correctly , but I just can't imagine that we 're gonna get anything that much better from a commercial one . undergrad d: Can't we get Joy to do it all ? grad a: Yeah right . postdoc e: No , that 's grad a: We will just get Joy and Jane to do everything . undergrad d: Is tha wasn't that what she was doing before ? Yeah , that 's right . grad a: But , you know , that 's what we 're talking about is getting some slaves who who need money undergrad d: Right . grad a: and , duh , again o postdoc e: I object to that characterization ! phd b: Oh , really . And so again , I have to say " are we recording " postdoc e: Oh , thank you . grad a: and then say , Morgan has has consistently resisted telling me how much money we have . Right ? undergrad d: Well , we already we already We already have a plan in place for the first meeting . undergrad d: Right ? That 's postdoc e: Well th there is als Yeah , really . There is also the o other possibility which is if you can provide not money but instructional experience or some other perks , you can you could get people to to  , to do it in exchange . undergrad d: Well , i b but seriously , I  , Morgan 's obviously in a bind over this and thing to do is just the field of dreams theory , which is we we go ahead as though there will be money at the time that we need the money . grad a: So at any rate , Jane was looking into the possibility of getting students , at is that right ? Talking to people about that ? postdoc e: I 'm afraid I haven't made any progress in that front yet . So ,  undergrad d: I d do So until you actually have a little experience with what this this French thing does we don't even have postdoc e: And I do have grad a: She 's already done quite a bit . So that 's where you came up with the f the ten X number ? postdoc e: yeah . undergrad d: Or is that really just a guess ? postdoc e: Actually that 's the the one people usually use , ten X . phd c: How fast are you ? postdoc e: And I haven't really calculated How fast am I ? undergrad d: Yeah i postdoc e: I haven't done a s see , I 've been at the same time doing kind of a boot strapping in deciding on the transcription conventions that that are you know , and and stuff like , you know , how much phd b: Mmm . postdoc e: There 's some interesting human factors problems like , yeah , what span of of time is it useful to segment the thing into in order to  , transcribe it the most quickly . postdoc e: Cuz then , you know , you get like if you get a span of five words , that 's easy . And then there 's the issue of it 's easier to hear it th right the first time if you 've marked it at a boundary instead of somewhere in the middle , phd b:   postdoc e: cuz then the word 's bisected or whatever and And so  , I 've been sort of playing with , different ways of mar cuz I 'm thinking , you know , if you could get optimal instructions you could cut back on the number of hours it would take . undergrad d: D does  this tool you 're using is strictly it doesn't do any speech recognition does it ? grad a: No . undergrad d: But but is there anyway to to wire a speech recognizer up to it and actually run it through postdoc e: That 's an interesting idea . grad a: We 've we 've thought about doing that postdoc e: Hey ! grad a: but the recognition quality is gonna be horrendous . undergrad d: First of all the time marking you 'd get you could get by a tool . undergrad d: And so if the if if the issue really postdoc e: That 's interesting . undergrad d: I 'm think about the close caption that you see running by on on live news casts . And in a lot of them you see typos and things like that , grad a:   undergrad d: but it but it occurs to me that it may be a lot easier to correct things than it is to do things from scratch , no matter how wonderful the tool is . Yeah , we undergrad d: But if if there was a way to merge the two phd c: Well , but sometimes it 's easier to type out something instead of going through and figuring out which is the right grad a: we 've talked about it postdoc e: That 'd be fun . grad a: but phd c: it depends on the error rate , right ? undergrad d: Well s but but again the timing is for fr should be for free . grad a: We don't care about the timing of the words , just of the utterances . postdoc e: No ,  the the boundary phd c: We cut it s s phd b: We don't we don't know , actually . phd b: We haven't decided which which time we care about , and that 's kind of one of the things that you 're saying , is like you have the option to put in more or less timing data and , be in the absence of more specific instructions , we 're trying to figure out what the most convenient thing to do is . grad a: Yeah , so so what what she 's done so far , is sort of more or less breath g not breath groups , sort of phrases , continuous phrases . grad a: And so , that 's nice because you you separate when you do an extract , you get a little silence on either end . postdoc e: Although I was I you know , the alternative , which I was sort of experimenting with before I ran out of time , recently was ,  that , you know , ev if it were like an arbitrary segment of time i t pre - marked cuz it does take time to put those markings in . postdoc e: It 's really the i the interface is wonderful because , you know , the time it takes is you listen to it , and then you press the return key . But then , you know , it 's like , you press the tab key to stop the flow and and , the return key to p to put in a marking of the boundary . But , you know , obviously there 's a lag between when you hear it and when you can press the return key phd b: Yeah . postdoc e: so it 's slightly delayed , so then you you listen to it a second time and move it over to here . undergrad d: i a postdoc e: Now if it could all be pre - marked at some , l you know , good undergrad d: ar but grad a:  . undergrad d: Are are those d delays adjustable ? Those delays adjustable ? See a lot of people who actually build stuff with human computer interfaces understand that delay , phd b: Yeah . undergrad d: and and so when you by the time you click it it 'll be right on because it 'll go back in time to put the postdoc e: Yeah . postdoc e: It has other grad a: couldn't we Dan ? Yeah , mis Mister TCL ? phd b: Yeah . grad a: But ,  postdoc e: it 's not bad grad a: But , if we tried to do automatic speaker ID . phd b: Yeah , yeah , but grad a: But that would be phd b: But we 've got we 've got the most channel data . postdoc e: Oh , good point ! Ah ! grad a: Yeah , I guess the question is how much time will it really save us versus the time to write all the tools to do it . but the chances are if we if we 're talking about collecting ten or a hundred hours , which is going to take a hundred or a thousand hours to transcribe undergrad d: If grad a: But undergrad d: if we can go from ten X to five X we 're doing a big grad a: We 're gonna need we 're gonna need ten to a hundred hours to train the tools , and validate the tools the do the d to to do all this anyway . phd b: If we 're just doing silence detection postdoc e: But but it op grad a: I knew you were gonna do that .  , it it 's it 's maybe like a week 's work to get to do something like this . postdoc e: Could you get it so that with so it would it would detect volume on a channel and insert a marker ? And the the format 's really transparent . postdoc e: It 's just a matter of a very c clear it 's XML , isn't it ? grad a:   postdoc e: It 's very  , I looked at the the file format and it 's just it has a t a time a time indication and then something or other , and then an end time or something or other . phd c: and undergrad d: Is this already in the past or already in the future ? phd c: Already in the past . undergrad d: You 've already you 've already done some ? grad a: She 's she 's done about half a meeting . phd c: She she 's done one she 's one postdoc e: Yes I have . grad a: About half ? phd c: I 'm go postdoc e: S I 'm not sure if it 's that 's much but anyway , enough to work with . And then go through  , and go through and and try and re - transcribe it , given that we had perfect boundary detection . undergrad d: And forgetting all the words because you 've been thr postdoc e: Yeah , that 's what I was thinking . phd c: Yeah , that 's part of the problem is , is that what we really need is somebody else to come along . phd b: Well , no , you should do it you should do it Do it again from scratch and then do it again at the boundaries . undergrad d: And so you 'll get you 'll get down to one point two X by the time you get done . undergrad d: No , but the thing is the fact that she 's she 's did it before just might give a lower bound . undergrad d: It 's And if the lower bound is nine X then w it 's a waste of time . postdoc e: Well ,  but there 's an extra problem which is that I didn't really keep accurate phd b: Oh ! postdoc e: it wasn't a pure task the first time , phd b: Yeah . So I think though it 's a good proposal to be used on a new a new batch of text that I haven't yet done yet in the same meeting . phd b: The point we where do we get the the the oracle boundaries from ? phd c: Right . grad a: Yeah , one person would have to assign the boundaries and the and the other person would have to postdoc e: Well , but couldn't I do it for the next phd b: We we we could get fake grad a:  that 's easy enough . postdoc e: Well , but the oracle boundaries would come from volume on a partic specific channel wouldn't they ? grad a: No , no . You wanna know given Given a perfect human segmentation , you wanna know how well postdoc e: Yeah . phd c: the the question is , is it worth giving you the segmentation ? postdoc e: Oh , I see what you mean . grad a: I could generate the segmentation and and you could do the words , and time yourself on it . grad a: That would at least tell us whether it 's worth spending a week or two trying to get a tool , that will compute the segmentations . undergrad d: And the thing to keep in mind too about this tool , guys is that sure , you can do the computation for what we 're gonna do in the future but if if UW 's talking about doing two , or three , or five times as much stuff and they can use the same tool , then obviously there 's a real multiplier there . postdoc e: And the other thing too is with with speaker identification , if if that could handle speaker identification that 's a big deal . postdoc e: That 's a major that 's like , one of the two things that phd c: there 's gonna there 's gonna be in the meeting , like the reading group meeting that we had the other day , that 's it 's gonna be a bit of a problem phd b: OK . grad a: That undergrad d: But you didn't say anything worth while anyway , right ? grad a: That 'll s phd b: Right . phd b: So I I need to we need to look at what what the final output is but it seems like we it doesn't it seems like it 's not really not that hard to have an automatic tool to generate the phrase marks , and the speaker , and speaker identity without putting in the words . I 've already become pretty familiar with the format , postdoc e: That 'd be so great . grad a: If you 'd tell me where it is , huh ? postdoc e: We didn't finish the the part of work already completed on this , did we ?  , you you talked a little bit about the transcription conventions , grad a:   postdoc e: and , I guess you 've mentioned in your progress report , or status report , that you had written a script to convert it into So , I when I i the it 's quickest for me in terms of the transcription part to say something like , you know , if if Adam spoke to ,  to just say , " A colon " , Like who could be , you know ,  at the beginning of the line . postdoc e: and E colon instead of entering the interface for speaker identification and clicking on the thing , indicating the speaker ID . So , and then he has a script that will convert it into the the thing that , would indicate speaker ID .  , I looked at Cyber Transcriber postdoc e: Yeah , well that 's true , but grad a: which is a service that you send an audio file , they do a first - pass speech recognition . postdoc e: What i just approximately , what did you find out in terms of price or or whatever ? grad a: Well , for Cyber Transcriber they don't quote a price . postdoc e: Or of action ? grad a: For thirty dollars an hour for of their work . Oh , of their grad a: So so if it 's ten times it 's three hundred dollars an hour . postdoc e: Oh ! phd c: So that 's three that 's three hours . undergrad d: D did you talk to anybody that does closed captioning for for  , TV ? phd c: Right . undergrad d: Cuz they a usually at the end of the show they 'll tell what the name of the company is , the captioning company that 's doing it . undergrad d: Well , you know , the the thing the thing about this is thinking kind of , maybe a little more globally than I should here but that really this could be a big contribution we could make .  , we 've been through the STP thing , we know what it what it 's like to to manage the manage the process , and admittedly they might have been looking for more detail than what we 're looking for here but it was a it was a big hassle , right ? phd b: Yeah . And it 's it 's only our time , where " our " of course includes Dan , Dan and you guys . j Just seems like phd b: Yeah ,  I don't know if we 'd be able to do any thing f to help STP type problems . But certainly for this problem we can do a lot better than undergrad d: Bec Why ? Because they wanted a lot more detail ? grad a: Right . Because they had because they only had two speakers , right ?  , the the segmentation problem is grad a: Trivial . So what took them so long ? grad a: mostly because they were doing much lower level time . grad a: And so we 're w we decided early on that we were not gonna do that . But there 's still the same issue of managing the process , of of reviewing and keeping the files straight , and all this stuff , that which is clearly a hassle . And so so what I 'm saying is that if we hire an external service I think we can expect three hundred dollars an hour . phd c: And who who knows if they 're gonna be able to m manage multal multiple channel data ? phd b: Yeah , they won't . phd b: No , but  , they they they won't they won't they will refuse to transcribe this kind of material . postdoc e: And then there 's the problem also that phd b: That 's not what they 're d quoting for , right ? grad a: Yes , it is . undergrad d: Well , they might they might quote it phd b: For quoting meetings ? grad a: Sev - several of them say that they 'll do meetings , and conferences , and s and so on . undergrad d: Th - th the th there may be just multiplier for five people costs twice as much and for ten people co Something like that . So what they had is , if it 's an easy task it costs twenty - four dollars an hour and it will take maybe five or six times real time . And what they said is for the hardest tasks , bad acoustics , meeting settings , it 's thirty - two dollars an hour and it takes about ten times real time . undergrad d: A lot of companies I 've worked for y the ,  the person leading the meeting , the executive or whatever , would sort of go around the room and and mentally calculate h how many dollars per hour this meeting was costing , grad a: So . But you know , it 's a lot like , " he 's worth fifty an hour , he 's worth " And so he so here we 're thinking , " well let 's see , if the meeting goes another hour it 's going to be another thousand dollars . " You know ? It 's grad a: Yep , we have to have a short meeting . But at any rate , so we we have a ballpark on how much it would cost if we send it out . undergrad d: And we 're talking about do doing how many hours worth of meetings ? grad a: Thirty or forty . phd c: So it 's thirty dollars an hour , essentially , right ? undergrad d: Yeah . And the question is what 's the difference phd b: How how much lower are they ? phd c: or ei eight dollars . What do you know what the going rate is ? It 's it 's on the order of eight to ten . postdoc e: yeah , I was gonna say eight you 'd say ten ? phd c: Let 's say ten . undergrad d: The - these are not for engineering graduate students , right ? grad a: Right , these are linguistics grad students . phd c: Yeah , I I I don't I don't know what the I don't know what the standard undergrad d: That 's right . phd c: so that means that even if it takes them thirty times real time it 's cheaper to to do graduate students . grad a: that 's why I said originally , that I couldn't imagine sending it out 's gonna be cheaper . postdoc e: The other thing too is that , if they were linguistics they 'd be you know , in terms of like the post editing , i  tu  content wise they might be easier to handle cuz they might get it more right the first time . grad a: we can't tell them , you know , " for this meeting we really wanna mark stress postdoc e: Good point . grad a: And and they 're not gonna provide they 're not gonna provide stress , they 're not gonna re provide repairs , they 're not gonna provide they they may or may not provide speaker ID . undergrad d: Just hypoth hypothetically assuming that that we go ahead and ended up using graduate students . I who who 's the person in charge ? Who 's gonna be the Steve here ? grad a: I hope it 's Jane . undergrad d: You ? grad a: Is that alright ? postdoc e: Oh , interesting .  , now would this involve some manner of  , monetary compensation or would I be the voluntary , coordinator of multiple transcribers for checking ? grad a: I would imagine there would be some monetary involved but we 'd have to talk to Morgan about it . See ? postdoc e: Oh , undergrad d: That 's why Dave should have been here . grad a: Well , I would like you to do it because you have a lot more experience than I do , postdoc e: Oh , cool . grad a: but if if that 's not feasible , I will do it with you as an advisor . undergrad d: W we 'd like you to do it and we 'd like to pay you . phd c: That 's a undergrad d: And and then postdoc e: OK . undergrad d: an an an and be and be sure and say , would you like fries with that when you 're thinking about your pay scale . Yeah , no , that I I would be interested in that in becoming involved in the project in some aspect like that grad a: OK . grad a: any more on transcript we wanna talk about ? phd b: What s so what are you so you 've done some portion of the first meeting . phd b: To carry on doing it ? postdoc e: What Well , you know what I thought was right now we have p So I gave him the proposal for the transcription conventions . And so my next plan would be phd b: What what do they what do they cover ? postdoc e: They 're very minimal . postdoc e: and this is a way which meshes well with with , making it so that , you know , on the At the phd b: Yeah . When you when you get the ,  you you get the speech signal you also get down beneath it , an indication of , if you have two speakers overlapping in a s in a single segment , you see them one displayed one above each other . And then at the same time the top s part of the screen is the actual verbatim thing . You can clip click on individual utterances and it 'll take you immediately to that part of the speech signal , and play it for you . undergrad d: Is there a limit to the number of speakers ? grad a: the user interface only allows two . And so , using this the convention that Jane and I have discussed , you can have as many overlapping speakers as you want . undergrad d: Th - this is the French software , right ? grad a: Yeah , French . phd b: Oh , really ? undergrad d: did you ask them to change the interface for more speakers ? postdoc e: Oh . grad a: Yes , and they said that 's on in in the works for the next version . grad a: Multichannels was also Well , they said they wanted to do it but that the code is really very organized around single channels . undergrad d: Do - do you know what they 're using it for ? Why 'd they develop it ? grad a: For this exact task ? phd c: For transcription . undergrad d: Are they linguists ? phd c: It 's undergrad d: But  , are they are they linguists or are they speech recognition people ? grad a: I think they 're linguists . phd c: They 're they have some connection to the LDC cuz the LDC has been advising them on this process , the Linguistic Data Consortium . grad a: And they have they 've actually asked if we are willing to do any development and I said , well , maybe . grad a: So if we want if we did if we did something like programmed in a delay , which actually I think is a great idea , I 'm sure they would want that incorporated back in . You know , they they do have So you have when you when you play it back , it 's it is useful to have , a a break mark to se segment it . But it wouldn't be strictly necessary cuz you can use the  , the tabbed key to toggle the sound on and off . And then also once you 've put a break in then you have the option of cycling through the unit . undergrad d: Loop it ? Yo - you n you know , there 's al also the the user interface that 's missing . postdoc e: Or or or undergrad d: It 's missing from all of our offices , and that is some sort of analog input for something like this . It 's something that wh when you move your hand further , the sound goes faster past it , like fast forward . You know , like a joy stick or a  , you could wire a mouse or trackball to do something like that . postdoc e: Why , that 's That 's not something I wanted to have happen . undergrad d: No , but I 'm saying if this is what professionals who actually do this kind of thing for for for m for video or for audio where you you need to do this , postdoc e: I see . undergrad d: and so you get very good at sort of jostling back and forth , rather than hitting tab , and backspace , and carriage return , and enter , and things like that . grad a: Yeah , we talked about things like foot pedals and other analog phd c: Yeah . grad a: So  , tho those are things we could do but I I just don't know how much it 's worth doing . postdoc e: They thought about and also it 'll go around the c the , I wanna say cursor but I 'm not sure if that 's the right thing . postdoc e: Anyway , you can so they thought about different ways of having windows that you c  work within , phd b:   postdoc e: and But so in terms of the con the conventions , then , basically , it 's strictly orthographic which means with some w provisions for , w  , colloquial forms . So if a person said , " cuz " instead of " because " then I put a an apostrophe at the beginning of the word and then in in double ang angle brackets what the full lexical item would be . postdoc e: And this could be something that was handled by a table or something but I think to have a convention marking it as a non - standard or wha I don't mean standard but a a a non  , ortho orthographic , whatever . And and there would be limits to how much refinement you want in indicating something as non - standard pres pronunciation . phd c: How are you handling backchannels ? postdoc e: Backchannels ? grad a: Comments . postdoc e: you know oh , yes , there was some in my view , when i when you 've got it densely overlapping , I didn't worry about I didn't worry about s specific start times . phd c: What do you mean by du postdoc e: I sort of thought that this is not gonna be easily processed anyway and maybe I shouldn't spend too much time getting exactly when the person said " no " , or , you know , i " immediate " . postdoc e: And instead just sort of rendered " within this time slot , there were two people speaking during part of it phd b: Yeah . postdoc e: and if you want more detail , figure it out for yourself " , phd b:   grad a: Well , I think what w what Eric was talking about was channels other than the direct speech , phd c: I see . postdoc e: was sort of the way I felt @ @ grad a: right ? phd c: Well , yeah , what  is wh  , when somebody says "  - huh " in the middle of , a @ @ grad a: Yep . phd c: Oh , cuz I was I was listening to Dan was agreeing a lot to things that you were saying as you were talking . Well , if it if there was a word like " right " , you know , then I wou I would indicate that it happened within the same tem time frame grad a: Yeah , there 's an overlapping mark . grad a: when when no one i when we 're not actually in the meeting , and we 're all sort of separated , and and doing things . But even during the meeting there 's a lot of overlap but it it 's marked pretty clearly .  , some of the backchannel stuff Jane had some comments and but I think a lot of them were because you were at the meeting . grad a: Yeah , but someone who , was just the transcriber wouldn't have known that . grad a: Or when Dan said , " I wa I wasn't talking to you " . undergrad d: So you take a bathroom break in the middle and and keep your head mount grad a: You have to turn off your mike . postdoc e: Well he was so so he was checking the meter levels and and we were handling things while he was labeling the the whatever it was , the PDA ? grad a:   postdoc e:  And and so he was in sort of you were sort of talking you know , so I was saying , like " and I could label this one left . grad a: But postdoc e: But really , no ,  w you know in the context if you know he can't hear what he 's saying grad a: but when you w when you listen to it undergrad d: he he It was a lot funnier if you were there though . postdoc e: yeah , grad a: Well what what it what happens is if you 're a transcriber listening to it it sounds like Dan is just being a total totally impolite . grad a:  But but if you knew that that I wasn't actually in the room , and that Dan wasn't talking to me , it it became OK . undergrad d: So th postdoc e: And that 's w that 's where I added comments . postdoc e: The rest of the time I didn't bother with who was talking to who but but this was unusual circum circumstance . undergrad d: So this is this is gonna go on the meeting meeting transcriber bloopers tape , right ? grad a: Yes . postdoc e: Well and part of it was funny ,  reason was because it was a mixed signal so you couldn't get any clues from volume that , you know , he was really far away from this conversation . I should rewrite the mix tool to put half the people in one channel and half in the other . I have a auto - gain - mixer tool that mixes all the head mounted microphones into one signal postdoc e: That 's a good idea . postdoc e: But I thought it would be you know , I I didn't wanna add more contextual comments than were needed but that , it seemed to me , clarified that the con what was going on . And ,  OK , phd c: So , s postdoc e: so normalization phd c: I was just gonna ask , so I just wanted to c sort of finish off the question I had about backchannels , phd b: Mmm . phd c: which which was , so say somebody 's talking for a while postdoc e: Yeah . phd c: and somebody goes "  -  " in the middle of it , and and and what not , does the conversation come out from the or the person who 's speaking for the long time as one segment and then there 's this little tiny segment of this other speaker or does it does the fact that there 's a backchannel split the the the it in two . postdoc e: OK , my my focus was to try and maintain conten con content continuity and , to keep it within what he was saying . Like I wouldn't say breath groups but prosodic or intonational groups as much as possible . So if someone said "  -  " in the middle of a of someone 's , intonational contour , I I indicated it as , like what you just did . postdoc e: then I indicated it as a segment which contained @ @ this utterance plus an overlap . phd b: But that 's but there 's only one there 's only one time boundary for both speakers , phd c: OK . And you know , it could be made more precise than that phd c: I see , postdoc e: but I just thought phd c: I see , OK . undergrad d: I think whenever we use these speech words we should always do the thing like you 're talking about , accent , postdoc e: Oh , I see what you mean . OK , and so then , in terms of like words like "  " and "  " I just wrote them because I figured there 's a limited number , and I keep them to a  , limited set because it didn't matter if it was " mmm " or "  " , you know , versus "  " . It 'd be good to have that in the in the conventions , what 's to be used . grad a: I I did notice that there were some segments that had pauses on the beginning and end . undergrad d: Well , what 's that mean ? postdoc e: Yeah , OK . phd b: We have to mark those ? postdoc e: So I had phd b: Don't they d can't we just leave them unmarked ? postdoc e: I d Well , you see , that 's possible too . grad a: Well , I wanna leave the marked I don't want them to be part of another utterance . Now that 's refinement that , maybe it could be handled by part of the part of the script or something more phd b: yeah , it seems like it seems like the , tran the transcription problem would be very different if we had these automatic speaker detection turn placing things . Because suddenly  , I don't know , actually it sounds like there might be a problem putting it into the software if the software only handles two parallel channels . Well you were saying , I think it can read grad a: It can read and write as many as you want , it 's just that it postdoc e:  - huh . phd b: But what if you wanna edit it ? Right ?  , the point is we 're gonna generate this transcript with five five tracks in it , but with no words .  , and if there are five five people speaking at once , grad a: Right , i it 's I didn't explain it well . If we use the the little the conventions that Jane has established , I have a script that will convert from that convention to their saved convention . postdoc e: And it can be m edited after the fact , grad a: Yes . postdoc e: can't it also ? But their but their format , if you wanted to in indicate the speakers right there instead of doing it through this indirect route , then i they a c window comes up and it only allows you to enter two speakers . undergrad d: But you 're saying that by the time you call it back in to from their saved format it opens up a window with window with five speakers ? postdoc e: So . undergrad d: Oh ! That is sort of f grad a: It 's just user interface . undergrad d: They didn't quite go the whole grad a: So i it 's undergrad d: Yeah , they didn't go the whole route , grad a: the the the whole saved form the saved format and the internal format , all that stuff , handles multiple speakers . undergrad d: did they ? They just grad a: It 's just there 's no user interface for specifying multiple any more than two . So your your script solves Doesn't it solve all our problems , postdoc e: And that grad a: Yep . undergrad d: cuz we 're always gonna wanna go through this preprocessing grad a: Yep . Oh , yes , I I wanted to have So sometimes a pers I  in terms of like the continuity of thought for transcriptions , it 's i it isn't just words coming out , it 's like there 's some purpose for an utterance . And sometimes someone will do a backchannel in the middle of it but you wanna show that it 's continued at a later point . So I have I have a convention of putting like a dash arrow just to indicate that this person 's utterance continues . And then when it  , catches back up again then there 's an arrow dash , and then you have the opposite direction to indicate continuation of ones own utterance versus , sometimes we had the situation which is you know , which you which you get in conversations , of someone continuing someone else 's utterance , phd b: Mmm . postdoc e: and in that case I did a tilde arrow versus a arrow tilde , to indicate that it was continuation but it wasn't Oh , I guess I did equal arrow for the for the own for yourself things phd b:   postdoc e: And then you could track whether it was the same speaker or not by knowing you know , at the end of this unit you 'd know what happened later . grad a: So phd b: But the only time that becomes ambiguous is if you have two speakers . Like , if you If you only have one person , if you only have one thought that 's continuing across a particular time boundary , you just need one arrow at each end , and if it 's picked up by a different speaker , it 's picked up by a different speaker . I guess if you have more than one thread going , then you then you need to know whether they were swapped or not . You know , undergrad d: It l ou grad a: if you were trying to do a remembrance agent . grad a: But , a lot of these issues , I think that for  , from my point of view , where I just wanna do speech recognition and information retrieval , it doesn't really matter . I you know I did this I did this transcription and I marked that , I marked it with ellipsis because it seemed like there was a difference . It 's something you wanted to indicate that it that I this was the end of the phrase , this was the end of that particular transcript , but it was continued later . Well that 's you know ,  I that 's why I didn't I didn't do it n  , that 's why I thought about it , and and re - ev phd b: Yeah , yeah . postdoc e: and it didn't do I didn't do it in ten times the the time . grad a: Well , so anyway , are we interested then in writing tools to try to generate any of this stuff automatically ? postdoc e: Yeah . postdoc e: I also wanted to ask you if you have a time estimate on the part that you transcribed . Do you have a sense of how long phd b: Yeah , it took me half an hour to transcribe a minute , but I didn't have any I didn't even have a postdoc e: OK . So I was doing it by typ typing into a text file and trying to fit It was horrible . postdoc e: Well , that 's that 's because you didn't have the segmentation help and all the other grad a: But I think for a first try that 's about right . phd b: Is it phd c: So so if we hired a who if we hired a whole bunch of Dan 's undergrad d: That 's right . phd b: It was actually it was quite it was a t undergrad d: a grad a: If we hire an infinite number of Dan 's phd b: it w undergrad d: It 'd b a a postdoc e: And there 's always a warm up thing of grad a: Are we gonna run out of disk space by the way ? phd b: Yeah . undergrad d: d Doesn't it beep in the other room when you 're out of disk space ? phd c: So Is there grad a: No . phd c: Maybe we should s consider also , starting to build up a web site around all of these things . phd b: We could have like business - to - business E - commerce as well ! phd c: That 's right . No , but I 'm it would be interesting it would be interesting to see grad a: Can we sell banner ads ? undergrad d: Get get paid for click - throughs ? grad a: What a good idea , phd b: Yeah . phd c: I want to introduce I I want to introduce the word " snot - head " into the conversation at this point . phd b: We can have undergrad d: You wanna word that won't be recognized ? phd c: You see , cuz  , cuz Exactly . Alright , see here 's here 's here 's my thought behind it which is that , the the stuff that you 've been describing , Jane , I gu one has to , of course indicate , i is very interesting , postdoc e: Alright . phd c: and I I 'd like to be able to to pore through , you know , the the types of tr conventions that you 've come up with and stuff like that . postdoc e: OK , now , w the alternative to a web site would be to put it in Doctor speech . postdoc e: Cuz cuz what I have is a soft link to my transcription that I have on my account phd c: Either 's fine . undergrad d: Web site 's what ? phd b: We could actually maybe we could use the TCL plug - in . See he said the word TCL and and that 's undergrad d: But he does such a good job of it . But I should be allowed to but undergrad d: If you just did a crappy job , no nobody would want you to do it . phd b: I sh I shouldn't be allowed to by m by my own by my according to my own priorities . grad a: And we have we have quite a disparate number of web and other sorts of documents on this project sort of spread around . grad a: and phd c: Right , so we can add in links and stuff like that to other things . phd c: The phd b: Well so then th grad a: Try try to s consolidate . phd c: Well , we could put we could put sort of a disorganized sort of group gestalt undergrad d: Why ? What what 's what 's the issue ? phd b: No one owns the project . undergrad d: W well Do But grad a: It 's mine ! All mine ! phd b: Well then you have to do the web site . undergrad d: But grad a: " Wah - hah - hah - hah - hah - hah . undergrad d: b but but but what are you what are you talking about for web site hacking ? phd b: No undergrad d: You 're talking about writing HTML , right ? grad a: Yeah , I I 'm talking about putting together all the data in a form that that is legible , and pleasant to read , and up to date , and et cetera , et cetera , et cetera . undergrad d: But , is it against the law to actually use a tool to help your job go easier ? grad a: Absolutely . undergrad d: You y grad a: It 's just as easy to use to edit the raw HTML as anything else . undergrad d: No kidding ? phd b: That 's obviously not true , grad a: It 's obviously not true . phd b: but you have undergrad d: No , it it it 's obviously true that he hasn't found any he likes . postdoc e: Which one do you use Jim ? undergrad d: I use something called Trellix . grad a: Now , I guess if I were if I were doing more powerful excuse me more complex web sites I might want to . postdoc e: Well , would this be to document it also for outside people or mainly for in house use ? grad a: But . phd b: I I I I grad a: OK , well , send me links and I wi send me pointers , rather , and I 'll put it together . I don't think we should say , " oh , it 's internal therefore we don't have to make it very good " . phd b: therefore we can put data in it that we don't we don't have to worry about releasing " . postdoc e: Yeah , it is true , that is it benefits to undergrad d: Cuz you 're gonna have to wor do the work sooner or later . undergrad d: d we we out of tape out of disk ? phd b: No , we 're doing we 're doing great . undergrad d: I I was looking for the actual box I plan to use , but I c all I could I couldn't find it at the local store . And it 's two by two , by one , and it would fit right under the right under th the the the the lip , grad a: Yeah , does everyone know about the lip on the table ? It 's great . undergrad d: And , it oc I p especially brought the bottom along to try and generate some frequencies that you may not already have recorded . grad a: Clink ! Clink ! undergrad d: Let 's see see what it does to the But this was the  just just to review , and I also brought this along rather than the projector so we can put these on the table , and sort of w push them around . grad a: And and crinkle them and phd b: What ? postdoc e: And th " that " being a diagram . phd b: What ? undergrad d: That that 's the six tables that we 're looking at . phd b:  th undergrad d: Well why not ?  , cuz this is what 's gonna happen . And and and you So this is the box 's grad a: Get your paper off my PDA ! phd b: Yeah . And so the the two head set jacks would be in the front and then the little LED to indicate that that box is live . The the important issue about the LED is the fact that we 're talking about eight of these total , which would be sixteen channels . And , even though we have sixteen channels back at the capture , they 're not all gonna be used for this . undergrad d: So there 'd be a subset of them used for obviously j just use the ones at this end for for this many . you 'd like a a way to tell whether your box is live , so the LED wouldn't be on . undergrad d: So if you 're plugged in it doesn't work and the LED is off that 's that 's a tip off . And then the ,  would wire the all of the cables in a in a bundle come through here and o obviously collect these cables at the same time . undergrad d: so this this notion of putting down the P Z Ms and taking them away would somehow have to be turned into leaving them on the table grad a: Right . undergrad d: and even though there 's eight cables they 're not really very big around so my model is to get a a a p piece of grad a: Sleeve . undergrad d: yeah , that that stuff that people put with the little you slip the wires into that 's sort of shaped like that cross section . I 'm I 'm r a I 'm going up and then I 'm going down . grad a: Whoo ! phd b: Speed bump ! postdoc e: Sleeping pol phd b: Speed bump . grad a: Speed A " sleeping policeman " ! undergrad d: Yeah , it 's like a speed bum An postdoc e: Speed bump . undergrad d: And they 're ac they 're actually ext extruded from plastic . phd c: What does that mean ? phd b: That 's the s that 's British for speed bump , phd c: Is it a speed bump ? undergrad d: So that the wires go through here . postdoc e: Ah ! grad a: OK , so that undergrad d: s So it would c basically go on the diagonal here . grad a: So why do we have sixteen channels instead of like some fewer number ? phd b: Yeah . undergrad d: because the phd b: How else are you gonna distribute them around the tables ? undergrad d: Because they 're there . Why two each ? phd b: Oh , because then you don't have to just have one each . So that if t if you have two people sitting next to each other they can actually go into the same box . undergrad d: And to See , thi this is really the way people sit on this table . phd b: Only if you had Well it depends on this box , right ? undergrad d: Oh , true enough . grad a: Oh , I didn't understand undergrad d: This this box Thi - thi thi this box is a one off deal . And , it 's function is to s to , essentially a wire converter to go from these little blue wires to these black wires , plus supply power to the microphones cuz the the he the , cheap head mounteds all require low voltage . grad a: So so you 'd imagine some sort of in some sort of patch panel on top to figure out what the mapping was between each d of these two and each of those one or what ? phd b:  ! undergrad d: Well I w I I the simplest thing I could imagine , i which is really , really simple is to quite literally that these things plug in . And there 's a there 's a plug on the end of each of these these , ei eight cables . undergrad d: An - and there 's only there 's only four slots that are you know , in in the first version or the version we 're planning to to build . undergrad d: So that that was the whole issue with the LED , that you plug it in , the LED comes on , and and and you 're live . undergrad d: Now the the the subtle issue here is that tha I I haven't really figured out a solution for this . What happens if somebody unplugs this because they plug in more of something else ? grad a:   undergrad d: Well the there 's no clever way to let the up stream guys know that you 're really not being powered . So th there will be a certain amount of looking at cables has to be done if people , rewire things . phd b: But  there are actually that you know , there 's an extra there 's a mix out on the radio receiver ? undergrad d:   phd b: So there are actually six XLR outs on the back of the radio receiver and only five cables going in , I had the wrong five , so I ended up not recording one of the channels and recording the mix . undergrad d: Wonder whether it works any phd b: But I subtracted the four that I did have from the mix and got a pretty good approximation of the @ @ . undergrad d: Is it is phd b: It 's not bad , grad a: Wow . grad a: So what 's the schedule on these things ? undergrad d: So phd b: But , you always postdoc e: Wow . undergrad d: well I was wrestling with th with literally the w number of connectors in the cable and the the , powering system . And I I was gonna do this very clever phantom power and I decided a couple days ago not to do it . So I think the other thing I 'd like to do is , do something about the set up phd b: See grad a: so that it 's a little more presentable and organized . Dan - Dan and I have sort of gone around on this , and we were thinking about something that opened up in the top to allow access to the mixer for example . undergrad d: But there 's these things sticking out of the mixer which are kind of a pain , so you end up with this thing that if if you stuck the mixer up here and the top opened , it 'd be it 'd be fine . You wouldn't necessarily Well , you s understand what I 'm grad a: Yeah , I understand . undergrad d: the the you can you can start start s sketching it out , grad a: So . undergrad d: and I can certainly build it out of oak no problem , would it you know , arb you know , arbitrarily amount of grad a: I need a desk at home too , alright ? Is that gonna be a better solution than just going out and buy one ? undergrad d: Well , the as we found out with the the thing that , Jeff bought a long time ago to hold our stereo system the stuff you buy is total crap . Works fine for holding a Kleenex , grad a: Right , Kleenex and telephones . grad a: so yeah , I g I guess it 's just a question , is that something you wanna spend your time on ? undergrad d: Oh , I I 'm paid for . phd b: Oh , yeah ! grad a: I 'm telling you , I 'm just gonna cart one of them away if they stay there much longer . undergrad d: let the record show at  at f four thirty - five Adam Janin says postdoc e: Wow . grad a: So , j  , then the other question is do we wanna try to do a user interface that 's available out here ? phd b: Sorry ? undergrad d: Slipped almost slipped it by Dan . grad a: And how do we want to do that ? postdoc e: You mean like see see meter readings , from while sitting here . undergrad d: What 's wrong with yours ? If we bought you a a phd b: Oh , a Applecard . phd c: Well undergrad d: What ? grad a: I have an IRAM machine I 've borrowed and we can use it . phd b: I or the undergrad d: N no , I 'm I 'm I 'm serious . Does does the wireless thing work on your grad a: Wait , isn't that an ethernet connection or is that a phone ? phd b: that 's an ethernet connection . grad a: We jus undergrad d: I 'm serious , that that it it phd b: Yeah . T to have it  , just undergrad d: It 's very convenient especially if Dan happens to be sitting at that end of the table to not have to run down here and and look in the thing every so often , phd b: Yeah . And given given that we 've got a wireless that we 've got a we got the field . undergrad d: Y  , we need obviously need to clear this with ACIRI but , how tough can that be ? There it you 'd all you need 's web access , isn't it ? phd b: W we don't need X access undergrad d: In in theory . That 's that 's what it does , undergrad d: OK , phd b: yeah . grad a: phd b: So grad a: right , so it 's just a question of getting a laptop and a wireless modem . phd b: With a with a with a w undergrad d: No , and he he had , reque @ @ my my proposal is you have a laptop . undergrad d: If if we bought you the thing would you mind using it with i the the phd b: No , I would love to but I 'm not sure if my laptop is compatible with the wave LAN thing they 're using . phd b: Well Apple has their own thing , right ? phd c: He 's undergrad d: Your new one ? grad a: Airport . phd b: Yeah , I think what I think you I think it just plug plugs in a PC card , so you could probably make it run with that , but . grad a: The question is , is there an Apple driver ? undergrad d: I e phd b: Yeah , I 'm sure . But  anyway there are there are abs there are a bunch of machines at ICSI that have those cards undergrad d: But the two t phd b: and so I think if w if it doesn't we should be able to find a machine that does that . I  I know that doesn't don't don't the important people have those little blue VAIOs that undergrad d: Well , b that to me that 's a whole nother . undergrad d: The the idea of con convincing them that we should use their network i is fairly straight forward . undergrad d: The idea of being able to walk into their office and say , " oh , can I borrow your machine for a while " , is is is a non - starter . So , either either we figure out how to use a machine somebody already in the group already owns , a a and the idea is that if it 's it perk , you know , it 's an advantage not not a disadvan or else we we literally buy a machine e exactly for that purpose . undergrad d: Certainly it solves a lot of the problems with leaving a monitor out here all the time . undergrad d: I I I I 'm I 'm not a big fan of doing things to the room that make the room less attractive for other people , grad a: Right . undergrad d: right ? Which is part of the reason for getting all this stuff out of the way grad a: Yeah . undergrad d: and and , so a monitor sitting here all the time you know people are gonna walk up to it and go , " how come I can't get , you know , Pong on this " or , whatev grad a:   I 've I 've borrowed the IRAM VAIO Sony thingy , phd c: Well grad a: and I don't think they 're ever gonna want it back . undergrad d: Well , the certainly , u you should give it a shot first See whether you you can get compatible stuff . Who knows , they might have an extra hardware s phd b: I 'd trade them a flat panel display for it . phd c: What is the , projector supposed to be hooked up to ? undergrad d: Good . There 's gonna be actually a a plug at the front that 'll connect to people 's laptops so you can walk in and plug it in . undergrad d: So people sitting here are going " testing , one , two , three " ! phd c: But  , that 's another that 's another possibility that , you know , solves undergrad d: It a phd b: Yeah . phd b: That 's an end undergrad d: But but but I think the idea of having a control panel it 's that 's there in front of you is really cool . phd b: I think and  , having having it on wireless is is the neatest way neatest way to do it . undergrad d: R grad a: I had undergrad d: As long as you d as l as long as you 're not tempted to sit there and f keep fiddling with the volume controls going , " can you talk a bit louder ? " phd b: Yeah . grad a: I had actually earlier asked if I could borrow one of the cards to do wireless stuff phd b: Yeah . undergrad d: And and it 's a a PCMCIA card , right ? grad a: Yep . undergrad d: right ? In your new machine ? phd c: It 's it really come down to the driver . phd c:  grad a: Right , and if and if his doesn't work , as I said , we can use the PC . undergrad d: Right , i it 'll it 'll work It 'll work the first time . phd b: grad a: So phd b: well , that sounds like a d good solution one way or the other . grad a: So Jim is gonna be doing wiring and you 're gonna give some thought to cabinets ? undergrad d: y yeah .  phd b: We 'd I think undergrad d: Hey , what are those green lights doing ? grad a: They 're flashing ! phd b:  - oh !  - oh ! Does that it means it means it 's gonna explode . undergrad d: Cut the red wire , the red wire ! phd b:  grad a: When people talk , it they go on and off . phd b: using using these undergrad d: And it and the f the five thousand for the wires , so if I 'm gonna do No . phd b: that 's true undergrad d: I have to do phd b: but we haven't spent that , right ? But once we once we 've done the intellectual part of these , we can just knock them out , right ? grad a: Cheap . phd b: And then we could Washington could have a system that didn't have any wireless but would had what 's based on these undergrad d:   phd b: And that 's about cuz you wouldn't even need the mixer if you didn't have the Oh th the P Z undergrad d: Right . But anyway you 'd save , on the seven seven or eight thousand for the for the wireless system . undergrad d: What ? postdoc e: It 's nice it 's nice to be thinking toward that . grad a: There 's a speech compression program that works great on things like this , cuz if the dynamic range is low it encodes it with fewer bits . phd c: Now , shorter words wouldn't would induce more dynamics , right ? You want to have phd b: Yeah , but if the words are more predictable . postdoc e: That 's a long word ! grad a: How do you spell that ? postdoc e: I don't know . grad a: OK , can you do one more round of digits ? Are we done talking ? undergrad d: Well it 's a choice if we get a choice , let 's keep talking . grad a: Are you done ? postdoc e: I 'm done , grad a: I 'm done . Like , you know , candy bars or something , grad a: I 'll I 'll remember to bring M and M 's next time . undergrad d: or or or or or a little ,  you know , toothbrushes like they give you at the d dentist . postdoc e: We don't know that for sure , do we ? grad a: I should have mentioned that s  , to pause between lines but undergrad d: No , I know . grad a: It 's it 's only a hard time for the transcriber not for the speech recognizer . phd b: Tha - tha postdoc e: But I also think you said channel four grad a: Me . phd b: But they 're in order ! undergrad d: But real phd b: They start , six , seven , eight , nine , zero , one , two , three , four , five , six , eight , nine . undergrad d: Where 'd they come from ? phd b: And they 're in order because they 're sorted lexically by the file names , which are have the numbers in digits . phd b: And so they 're actually this is like all the all utterances that were generated by speaker MPJ or something . phd b: It doesn't matter ! It 's like Cuz you said " six , seven , eight " . If I if not I undergrad d: But the real question I have is that , why bother with these ? postdoc e: Oh , interesting . undergrad d: Why don't you just ask people to repeat numbers they already know ? Like phone numbers , you know , social security numbers . phd b: Cuz we have these writt written down , right ? grad a: Because Right . phd b: That 's why grad a: If we have it ,  undergrad d: I know . undergrad d: Credit card numbers , grad a: We don't have to tran undergrad d: yeah . undergrad d: Yeah , so you just say say your credit card numbers , say your phone numbers , say your mother 's maiden name . undergrad d: This grad a: Actually , this I got this directly from another training set , from Aurora . postdoc e: I was I the reason I made my mistake was grad a: What ? postdoc e: Wa - was this ? phd b: There were no there were no direct driver errors , by the look of it , which is good <doc-sep>grad b: Could I hit hit F - seven to do that ? on the Robert ? grad a: I 'm grad b: Oh , the remote will do it OK . grad b: Cuz I 'm already up there ? grad a: in control here . grad b: So , we were Ah ! grad c: Johno , where are you ? grad b: OK . grad c: Should you go back to the first one ? grad b: Do I wanna go back to the first one ? grad c: Well grad b: OK . grad d: I 'm sorry I grad c: Well , just to grad b: OK . It 's basically  talks about  It just refers to the fact that  one of main things we had to do was to decide what the intermediate sort of nodes were , grad d: I can read ! I 'm kidding . grad a: But if you really want to find out what it 's about you have to click on the little light bulb . grad b: Although I 've I 've never I don't know what the light bulb is for . grad a: Do you wanna try ? grad d: Ach u grad b: I 'd prefer not to . Is that the idea ? grad a: Why are you doing this in this mode and not in the presentation mode ? grad d: OK . grad b: Because I 'm gonna switch to the JavaBayes program grad a: Oh ! OK . grad b: You want me to Wait , what do you want me to do ? grad c: Can you maximize the window so all that stuff on the side isn't doesn't appear ? grad a: No , It 's OK . grad b: Well I can do that , but then I have to end the presentation in the middle so I can go back to open up grad c: OK , fine . grad b: So then the features we decided or we decided we were talked about , right ?  the the prosody , the discourse , verb choice . Whether the and this i we actually have a separate feature but I decided to put it on the same line for space . " Nice walls " which we can look up because  if you 're gonna get real close to a building in the Tango mode , right , there 's gotta be a reason for it . And it 's either because you 're in route to something else or you wanna look at the walls . The context , which in this case we 've limited to " business person " , " tourist " , or " unknown " , the time of day , and " open to suggestions " , isn't actually a feature . can I just ask the nice walls part of it is that  , in this particular domain you said be i it could be on two different lines but are you saying that in this particular domain it happens the that landmark - iness cor is correlated with grad b: Oh grad c: No . grad b: I either could put " nice walls " on its own line or " open to suggestions " off the slide . grad c: Like you could have a p grad d: And and By " nice " you mean grad c: You Like you could have a post office with  you know , nice murals or something . grad b: Or one time I was at this grad d: So " nice walls " is a stand in for like architecturally it ,  significant grad b: But see the thing is , if it 's grad c: Architecturally appealing from the outside . grad b: Yeah but if it 's architecturally significant you might be able to see it from Like you m might be able to " Vista " it , grad a:   grad b: Yeah , versus , like , I was at this place in Europe where they had little carvings of , like , dead people on the walls or something . grad b: But if you looked at it real close , you could see the the in intricacy of the of the walls . grad a: The grad d: Something you want to inspect at close range because it 's interesting . grad b: Robert ? grad a: Well there there is a term that 's often used . And I was just wondering whether that 's the same as what you describe as " landmark - iness " . There 's landmark for  , touristic reasons and landmark for I don't know navigational reasons or something . Tourist - y landmarks also happen to be Wouldn't couldn't they also be They 're not exclusive groups , are they ? Like non - tourist - y landmarks and grad a: Or it can be als grad b: direct navigational grad d: They 're not mutually exclusive ? grad b: Yeah . grad b: OK , So our initial idea was not very satisfying , because  our initial idea was basically all the features pointing to the output node . grad b: And  , so we Reasons being , you know , it 'd be a pain to set up all the probabilities for that . If we moved onto the next step and did learning of some sort ,  according Bhaskara we 'd be handicapped . grad c: Well usually , you know , N If you have N features , then it 's two to the N or exponential in N . So then our next idea was to add a middle layer , right ? So the thinking behind that was we have the features that we 've drawn from the communication of some Like , the someone s The person at the screen is trying to communicate some abstract idea , like " I 'm " the the abstract idea being " I am a tourist I want to go to this place . " Right ? So we 're gonna set up features along the lines of where they want to go and what they 've said previously and whatnot . Right ? but the middle thing , we were thinking along the lines of maybe trying to figure out , like , the concept of whether they 're a tourist or whether they 're running an errand or something like that along those lines . So then the hidden variables hair variables we came up with were whether someone was on a tour , running an errand , or whether they were in a hurry , because we were thinking  , if they were in a hurry there 'd be less likely to like or th grad c: Want to do Vista , grad b: Right . grad c: right ? Because if you want to view things you wouldn't be in a hurry . grad b: Or they might be more likely to be using the place that they want to go to as a like a navigational point to go to another place .  right now it 's still kind of in a toy version of it , because we didn't know the probabilities of or Well I 'll talk about it when I get the picture up . " Verb used " is actually personally amusing mainly because it 's it 's just whether the verb is a Tango verb , an Enter verb , or a Vista verb . grad c: Yeah , that one needs a lot of grad d: And are those mutually exclusive sets ? grad b: No . grad c: But  that would 've made the probably significantly be more complicated to enter , grad d: Got it . grad c: so we decided that for the purposes of this it 'd be simpler to just have three verbs .  Why don't you mention things about this , Bhaskara , that I am not that are not coming to my mind right now . grad c: OK , so Yeah , so note the four nodes down there , the sort of , the things that are not directly extracted . The " closed " is also not directly extracted I guess , from the  grad b: Well i it 's grad c:  . grad d: From the utterance ? grad b: it 's so it sort of is grad c: Actually , no , wait . grad b: because it 's because have the the time of day grad c: It is . grad c: Right , so f Right , but the other ones , the final destination , the whether they 're doing business , whether they 're in a hurry , and whether they 're tourists , that kind of thing is all  sort of you know probabilistically depends on the other things . So we haven't  , managed Like we don't have nodes for " discourse " and " parse " , although like in some sense they are parts of this belief - net . grad c: But  The idea is that we just extract those features from them , so we don't actually have a node for the entire parse , grad d:   grad d: So some of the the top row of things What 's what 's " Disc admission fee " ? grad c: whether they discuss the admission fees . So we looked at the data and in a lot of data people were saying things like " Can I get to this place ? " grad d: Oh . So that 's like a huge  clue that they 're trying to Enter the place rather than  to Tango or Vista , grad d:  - huh . grad b: There were there 'd be other things besides just the admission fee , but you know , we didn't have grad d:   So there are certain cues that are very strong either lexical or topic - based  , concept cues grad b: From the discourse that Yeah . And then in that second row or whatever that row of Time of Day through that So all of those Some of them come from the utterance and some of them are sort of either world knowledge or situational things . grad d: " Unmark @ @ Time of Day " grad c: Yeah , I m  grad a: Yeah . I would actually suggest we go through this one more time so we we all  , agree on what what the meaning of these things is at the moment and maybe what changes we grad b: Yeah , th OK . so one thing I I 'm you know unsure about , is how we have the discus  the " admission fee " thing set up . So one thing that we were thinking was by doing the layers like this ,  we kept  things from directly affecting the mode beyond the concept , but you could see perhaps discus the " admission fee " going directly to the mode pointing at " Enter " , grad a:   grad b: right ? Versus pointing to just at " tourist " , grad d:   grad b: But we just decided to keep all the things we extracted to point at the middle and then down . That 's because we 're talking about landmarks as touristic landmarks not as possible  grad b: Right . grad c: Disc - " admission fee " is a binary thing , " time of day " is like morning , afternoon , night . grad b: That 's how we have it currently set up , grad a: Yep . grad b: but it could be , you know , based upon hour grad c: Yeah . Normally context will include a huge amount of information , but  , we are just using the particular part of the context which consists of the switch that they flick to indicate whether they 're a tourist or not , I guess . grad c: So Right , grad d: Right ? grad c: so it 's not really all of context . Similarly prosody is not all of prosody but simply for our purposes whether or not they appear tense or relaxed . grad a: The the So the context is a switch between tourist or non - tourist ? grad c: and grad a: Or also unknown ? grad b: Or un unknown , grad a: OK . Unknown , right ? grad d: So final dest So it seems like that would really help you for doing business versus tourist , grad c: Which is th Which one ? grad d: but OK . so the the context being  , e I don't know if that question 's sort of in general , " are you "  the ar ar are do they allow business people to be doing non - business things at the moment ? grad c: Yeah , it does . So then you just have some probabilities over grad c: Everything is probablistic , and There 's always grad d: OK . " Verb used " is like , right now we only have three values , but in general they would be a probability distribution over all verbs . grad c:  " nice walls " is binary , " closed " is binary " final destination " , again Yeah , all those are binary I guess . grad c: Yeah , anything with a question mark after it in that picture is a binary node . Right ? grad c: Which things ? grad a: Nice walls ? grad b: Wi grad d:   grad b: It is binary but it doesn't have question mark because it 's extracted . grad a: So we can either be in a hurry or not , but we cannot be in a medium hurry at the moment ? grad c: Well , we To do that we would add another  value for that . grad c: And that would require s updating the probability distribution for " mode " as well . grad d: So  , of course this will happen when we think more about the kinds of verbs that are used in each cases grad a: Yeah , yeah . grad d: but you can imagine that it 's verb plus various other things that are also not in the bottom layer that would that would help you Like it 's a conjunction of , I don't know , you know , the verb used and some other stuff that that would determine grad c: Right . grad a: well the the sort of the landmark is is sort of the object right ? the argument in a sense ? grad d: Usually . I I don't know if that 's always the case I I guess haven't looked at the data as much as you guys have . grad a: that 's always warping on something some entity , grad d:   grad a: and   maybe at this stage we will we do want to  sort of get  modifiers in there grad b:  . grad a: because they may also tell us whether the person is in a hurry or not grad b: I want to get to the church quickly , grad c: Yeah . Do we have anything else to say about this ? grad c: We can do a little demo . grad a: No , then it wouldn't be a demo I was just gonna s grad c:  We can do a demo in the sense that we can  , just ob observe the fact that this will , in fact do inference . grad c: So we can , you know , set some of the  nodes and then try to find the probability of other nodes . grad c: just  I don't know , say they discussed the admission fee grad b: OK . grad c: and  the place has nice walls grad b: I love nice walls , OK ? I 'm a big fan . grad d: it 's starting to grow on me grad b: And the time of day is night ? grad c: Yeah , no wait . grad b: One thing that bugs me about JavaBayes is you have to click that and do this . grad c: So that is the probability that they 're Entering , Vista - ing or Tango - ing . grad c: And  grad d: So slightly biased toward " Tango " ing grad c: Yeah . grad b: If it 's night time , they have not discussed admission fee , and the n walls are nice . The reason I say the demo doesn't work very well is yesterday we  observed everything in favor of taking a tour , and it came up as " Tango " , right ? Over and over again . grad c: Like , we totally hand - tuned the probabilities , grad d: Yeah . We were like "  , well if the person does this and this and this , let 's say forty percent for this , grad d: OK . grad a: Yeah but it it grad d: Maybe the bias toward " Tango " ing was yours , then ? grad b: Yeah , grad c: Yeah . grad b: that 's that 's at grad c: It 's So we have to like fit the probabilities . grad d: So , the real case ? grad a: However you know , it The purpose was not really , at this stage , to come up with meaningful probabilities but to get thinking about that hidden middle layer . grad a: And grad b: We would actually I guess once we look at the data more we 'll get more hidden nodes , grad a:   grad b: No , I think we should have  exponentially more middle nodes than features we 've extracted . Whether you 're It 's whether It 's not grad d: And are th grad c: I think it 's more like " Are you are tourist ? are you in Ham - like Heidelberg for a " grad d: Oh , so , I thought that was directly given by the context switch . What if the context , which is not set , but still they say things like , " I want to go  , see the  the the castle and  , et cetera . " grad a: Is it grad b: Well the I kind of thought of " doing business " as more of running an errand type thing . grad a: So if you run out of cash as a tourist , and and and you need to go to the AT grad b: So i wi th grad d: OK . grad a: " How do I get to the bank ? " grad d: I see . grad c: And that 'll affect whether you want to enter or you if you kinda thing . grad c: Yeah , I think this context node is a bit of a I don't know , like in d  Do we wanna have Like it 's grad d: Are you assuming that or not ? Like is that to be  if that 's accurate then that would determine tourist node . grad c: If the context were to set one way or another , that like strongly   , says something about whether whether or not they 're tourists . grad c: So what 's interesting is when it 's not when it 's set to " unknown " . grad a: We - what set the they set the context to " unknown " ? grad d: OK . grad c: Right now we haven't observed it , so I guess it 's sort of averaging over all those three possibilities . grad a: And if we now do leave everything else as is the results should be the same , grad b: Oops . grad c: Well no , because we Th - the way we set the probabilities might not have Yeah , it 's it 's an it 's an issue , right ? Like grad a: Pretty much the same ? grad c: Yeah , it is . So the issue is that  in belief - nets , it 's not common to do what we did of like having , you know , a d bunch of values and then " unknown " as an actual value . What 's common is you just like don't observe the variable , grad d: Yeah . grad c: But  We didn't do this because we felt that there 'd I guess we were thinking in terms of a switch that actually grad b: We were thi Yeah , grad a:   grad b: We were th grad c: But  I don't know y what the right thing is to do for that . grad a: Why don't we Can we ,  How long would it take to to add another node on the observatory and , play around with it ? grad c: Another node on what ? grad b: well it depends on how many things it 's linked to . If we create something that for example would be  So th some things can be landmarks in your sense but they can never be entered ? So for example s a statue . grad a: So maybe we wanna have " landmark " meaning now " enterable landmark " versus ,  something that 's simply just a vista point , for example . grad a: Yeah ?  , a statue or  grad c: So basically it 's addressing a variable that 's " enterable or not " . grad b: Also you know , didn't we have a size as one ? The size of the landmark . grad c: What ? grad b: Cuz if it 's grad c:  . grad b: For some reason I had that OK , that was a thought that I had at one point but then went away . grad c: So you want to have a a node for like whether or not it can be entered ? grad a: Well , for example , if we include that , yeah ? grad c: Yeah . grad a: accessibility or something , yeah ? " Is it Can it be entered ? " grad c:  . In the sense that , you know , if it 's Tom the house of Tom Cruise , you know , it 's enterable but you may not enter it . grad a: Yeah ? and  And these are very observable sort of from the from the ontology sort of things . grad b: Way Does it actually help to distinguish between those two cases though ? Whether it 's practically speaking enterable , or actually physically enterable or not ? grad a: y y If If you 're running an errand you maybe more likely to be able to enter places that are usually not al w you 're not usually not allowed to  m grad d: It seems like it would for  ,  determining whether they wanna go into it or not . grad b: Well I can see why grad d: Cuz they grad a: Let 's get this  b clearer . grad b: Whether it 's a Whether it 's a public building , and whether it 's actually has a door . grad a: This is sort of  grad b: So Tom Cruise 's house is not a public building grad d:   grad b: OK , sh explain to me why it 's necessary to distinguish between whether something has a door and is not public . Or , if something It seems like it 's equivalent to say that it doesn't have a door a and it grad a:   grad b: Or " not public " and " not a door " are equivalent things , grad a: Yeah . So we would have What does it mean , then , that we have to we have an object type statue . grad a: And then we have , for example , an object type , that 's a hotel . It 's the hotel Zum Ritter , which is the only Renaissance building in Heidelberg that was left after the big destruction and for the Thirty Years War , blah - blah - blah .  - And lots of detail , c and carvings , engravings and so forth , grad b: Excellent . So I guess your question is so far I have no really arg no real argument why to differentiate between statues as statues and houses of celebrities , from that point of view . Let Let 's do a Can we add , just so I can see how it 's done , a " has door " property or ? grad b: OK . grad c: What would it , connect to ? Like , what would , it affect ? grad a: I think , it might affect Oh actually it 's it it wouldn't affect any of our nodes , right ? grad c: What I was thinking was if you had a like grad a: Oh it 's it affects th The " doing business " is certainly not . grad b: You could affect Theoretically you could affect " doing business " with " has door " . grad a: right ? grad c: Yeah , I don't know if JavaBayes is nice about that . It might be that if you add a new thing pointing to a variable , you just like it just overwrites everything . Whew ! grad c: Well that 's fine , but we have to see the function now . grad b: This grad c: What would be nice if it is if it just like kept the old function for either value but . grad b: Oh wait , it might be Did we w Yes , that 's not good . grad a: Maybe you can read in ? grad c: Ha - So have you used JavaBayes a lot ? grad d: Yes . Really I ha I 've I haven't used it a lot and I haven't used it in the last you know many months so grad c: OK . grad c: Like , we looked at sort of  a page that had like a bunch of grad d: Yeah . grad c:  in a way this is a lot of good features in Java it 's cra has a GUI and it 's  grad d:   grad c: What ? grad b: Maybe it did a little bit of learning , grad c: OK . grad a: What is the c code ? Can w can we see that ? How do you write the code grad b: The c grad a: or do you actually never have to write any code there ? grad c: Yeah . grad b: Oh man , grad c: Like , there 's the grad b: I didn't n Is there an ampersand in DOS ? grad c: Nope . grad c: It 'll ask you what you what it wants what you want to open it with and see what BAT , I guess . grad c: That 's Oh ! grad b: Maybe it was just grad a: Oh . grad b: I like I like Word Pad because it has the  the returns , grad a: Wordpad ? I grad b: the carriage returns on some of them . grad b: You know how they get " auto - fills " I guess , grad a: Mmm  grad b: It just basically looks like it just specifies a bunch of grad a:   grad b: It just that it 's grad c: But they 're not very friendly . grad b: Yeah the ordering isn't very clear on grad c: So you 'd have to like figure out Like you have to go and grad d: Right . grad c: it 's not grad b: We were doing it grad c: Yeah we can maybe write an interface th for  entering probability distributions easily , something like like a little script . I actually seem to recall Srini complaining about something to do with Entering probability so this is probably grad c: The other thing is it is in Java grad d: Yeah , it 's Yeah . grad b: Or grad a: Do you have the true source files or just the class ? grad b: I don't know if he actually grad c: Yeah . we do grad b: Does he grad c: I I saw directory called " source " , grad b: Oh . grad c: I think it might it might be simpler to just have a script that , you know It 's , like , friendly , grad d: The d the data tables . grad a: But if th if there is an XML file that or format that it can also read  it just reads this , right ? When it starts . grad b: Yeah I know there is an I was looking on the we web page and he 's updated it for an XML version of I guess Bayes - nets . grad c: The JavaBayes guy ? So but , e he doesn't use it . So in what sense has he updated it ? grad b: Well th you can either you ca or you can read both . grad b: Because  Well at least the  I could have misread the web page , I have a habit of doing that , but . grad a: So you got more slides ? grad b: Do I have more slides ?  yes , one more . E That 's maybe , I don't know If grad b:  that 's future future work . grad b: And of course if you have a presentation that doesn't have something that doesn't work at all , then you have " What I learned " , as a slide . grad b: I know what I like about these meetings is one person will nod , and then the next person will nod , and then it just goes all the way around the room . So this means  grad b: Should I pull up the net again ? grad d: Yeah . grad d: So a more general thing than " discussed admission fee "  , could be I I 'm just wondering whether the context , the background context of the discourse might be I don't know , if there 's a way to define it or maybe you know generalize it some way  , there might be other cues that , say , in the last few utterances there has been something that has strongly associated with say one of the particular modes  , I don't know if that might be grad a:   I think we grad d: and and into that node would be various various things that that could have specifically come up . grad a: I think a a sort of general strategy here You know , this is this is excellent because  it gets you thinking along these terms is that maybe we ob we could observe a couple of  discourse phenomena such as the admission fee , and something else and something else , that happened in the discourse before . And maybe there are two  So maybe this could be sort of a separate region of the net , which has two has it 's own middle layer . Maybe this , you know , has some kind of  , funky thing that di if this and this may influence these hidden nodes of the discourse which is maybe something that is  , a more general version of the actual phenomenon that you can observe . So things that point towards grad b: So instead of single node , for like , if they said the word " admission fee " grad d: Exactly . grad b: " admission fee " , or maybe , you know , " how much to enter " grad d: Yeah . Yeah ? And then maybe there are some discourse acts if they happened before ,  it 's more for  a cue that the person actually wants to get somewhere else and that you are in a in a in a route  , sort of proceeding past these things , so this would be just something that where you want to pass it .  ? Is that it ? However these are of course then the the nodes , the observed nodes , for your middle layer . So this again points to " final destination " , " doing business " , " tourist hurry " and so forth . we have a whole region " in a e grad d: That 's a whole set of discourse related cues to your middle layer . grad d: Right ? grad a: So e because at the end the more we  add , you know , the more spider - web - ish it 's going to become in the middle and the more of hand editing . They ra may have there own hidden layer that points to some of the the real hidden layer ,  or the general hidden layer . grad a: And the same we will be able to do for syntactic information , the verbs used , the object types used , modifiers . grad b: One thing that 's kind of been bugging me when I more I look at this is that the I guess , the fact that the there 's a complete separation between the observed features and in the output . grad b: For instance if the discourse does grad d: What do you mean by that ? grad b: well for instance , the " discourse admission fee " node seems like it should point directly to the grad d:  - huh . grad b: or increase the probability of " enter directly " versus " going there via tourist " . Like we could add a node like do they want to enter it , which is affected by admission fee and by whether it 's closed and by whether it has a door . And if it if you do it If you could connect it too hard you may get such phenomenon that like " So how much has it cost to enter ? " and the answer is two hundred fifty dollars , and then the persons says  " Yeah I want to see it . " Yeah ? meaning " It 's way out of my budget "  grad b: There are places in Germany where it costs two hundred fifty dollars to enter ? grad a: nothing comes to mind . But i you know , i we can Something Somebody can have discussed the admission fee and u the answer is s if we  , you know ,  still , based on that result is never going to enter that building . So the discourse refers to " admission fee " but it just turns out that they change their mind in the middle of the discourse . you have to have some notion of not just  there 's a there 's change across several turns of discourse grad b: Right . grad d: so I don't know how if any of this was discussed but how i if it all this is going to interact with whatever general  , other other discourse processing that might be happen . grad b: What sort of discourse processing is  are the How much is built into SmartKom and grad a: It works like this . The first thing we get is that already the intention is sort of t They tried to figure out the intention , right ? simply by parsing it . And this  m won't differentiate between all modes , yeah ? but at least it 'll tell us " OK here we have something that somebody that wants to go someplace , now it 's up for us to figure out what kind of going there is is is happening , and  , if the discourse takes a couple of turns before everything all the information is needed , what happens is you know the parser parses it and then it 's handed on to the discourse history which is ,  o one of the most elaborate elaborate modules . It 's it 's actually the the whole memory of the entire system , that knows what wh who said what , which was what was presented . It helps an an anaphora resolution and it and it fills in all the structures that are omitted , so , because you say " OK , how can I get to the castle ? " Oh , how how much is it ? " and  " yeah I would like   to g let 's do it " and so forth . So even without an a ana anaphora somebody has to make sure that information we had earlier on is still here . so whenever the  ,  person is not actually rejecting what happened before , so as in " No I really don't want to see that movie . I 'd rather stay home and watch TV "  What movie was selected in what cinema in what town is is going to be sort of added into the disc into the representations every di at each dialogue step , by the discourse model discourse model , Yeah , that 's what it 's called . and , it does some help in the anaphora resolution and it also helps in coordinating the gesture screen issues . So a person pointing to something on the screen , you know , the discourse model actually stores what was presented at what location on the s on the screen grad b:  . grad a: so it 's a it 's a rather huge huge thing but   we can sort of It has a very clear interface . We can query it whether admission fees were discussed in the last turn and and the turn before that or you know how deep we want to search grad b: OK . How deep do we want to sear , you know ?  but we should try to keep in mind that , you know , we 're doing this sort of for research , so we we should find a limit that 's reasonable and not go , you know , all the way back to Adam and Eve . You know , did that person ever discuss admissions fee fees in his entire life ? And the dialogues are pretty pretty you know concise and Anyway . grad d: So one thing that might be helpful which is implicit in the use of " admission fee discussion " as a cue for entry , is thinking about the plans that various people might have . This person is  , finding out information about this thing in order to go in as a tourist or finding out how to get to this place in order to do business .  , because then anything that 's a cue for one of the steps would be slight evidence for that overall plan . They 're in in non in sort of more traditional AI kinds of plan recognition things you sort of have you know , some idea at each turn of agent doing something , " OK , wha what plans is this a consistent with ? " and then get s some more information and then you see " here 's a sequence that this sort of roughly fits into " . grad d: I I don't know how you know you 'd have to figure out what knowl what knowledge representation would work for that . grad a: You know ? and it it 's fifty steps , grad d:   grad a: huh ? just for buying a ticket at a ticket counter , you know , and and maybe that 's helpful to look at it to look at those . W when we talked  we had the example , you know , of you being  a s a person on a ticket counter working at railway station and somebody r runs up to you with a suitcase in his hands , says New York and you say Track seven , huh ? And it 's because you know that that person actually is following , you know You execute a whole plan of going through a hundred and fifty steps , you know , without any information other than " New York " , huh ? inferring everything from the context .  , even though there is probably no train from here to New York , right ? grad d: Mmm . Right ? Is that t San Francisco , Chicago ? grad b: I think grad a: Is that possible ? grad b: One time I saw a report on trains , and I think there is a l I don't know if I thought there was a line that went from somewhere , maybe it was Sacramento to Chicago , grad a:   grad d: The Transcontinental Railroad , doesn't that ring a bell ? grad b: Yeah but I don't know if it 's still grad d: I think it has to exist somewhere . grad a: Well it never went all the way , right ?  you always had to change trains at Omaha , grad d: Well most of the way . grad a: right ? One track ended there and the other one started at five meters away from that grad d:  . grad a: yeah ? Has anybody ever been on an Amtrak ? grad d: I have . grad c: What ? Why ? grad b: I just They seem to have a lot of accidents on the Amtrak . grad a: But you know , I don't know whether it 's which ones are safer , you know , statistically . Yeah , they 're Yeah , it 's way better grad a: yeah I used  Amtrak quite a bit on the east coast and I was surprised . grad a:  ? grad c: I I want to see what it does with  " landmark - iness " . grad d: So by the way tha that structure that Robert drew on the board was like more  , cue - type - based , right , here 's like we 're gonna segment off a bit of stuff that comes from discourse and then some of the things we 're talking about here are more you know , we mentioned maybe if they talk about  , I don't know , entering or som you know like they might be more task - based . grad d: So I I don't know if there There 's obviously some m more than one way of organizing the variables into something grad a: I think that  What you guys did is really nicely sketching out different tasks , and maybe some of their conditions . grad a: One task is more likely you 're in a hurry when you do that kind of s doing business , grad d:   grad a: and and less in a hurry when  you 're a tourist  tourists may have never have final destinations , you know because they are eternally traveling around so maybe what what what happened what might happen is that we do get this sort of task - based middle layer , grad d:   grad a: and then we 'll get these sub - middle layers , that are more cue - based . So ,  I suggest w to for to proceed with this in in the sense that maybe throughout this week the three of us will will talk some more about maybe segmenting off different regions , and we make up some some toy a observable " nodes " is that what th grad b: Refined y re just refine the grad a: What 's the technical term ? grad c: OK . For which ? grad a: For the  nodes that are observable ? The " outer layer " ? grad c: Just observable nodes , grad b: The features , grad c: evidence nodes ? grad b: I don't know , whatever you grad a: Feature ma make up some features for those Identify four regions , grad c: Yeah . grad a: maybe make up some features for each region and  and  ,  and  middle layer for those . And then these should then connect somehow to the more plan - based deep space grad c: Yeah . grad c: Yeah , this is totally like The probabilities and all are completely ad - hoc .  but , they 're even like  like , close to the end we were like , you know we were like  really ad - hoc . grad c: Right ? Cuz if it 's like ,  If it 's four things coming in , right ? And , say , some of them have like three possibilities and all that . So you 're thinking like like a hundred and forty four or something possible things numbers to enter , grad d: And That 's terrible . grad b: Some of them are completely absurd too , like they want to enter , but it 's closed , grad d: That 's  Well grad b: it 's night time , you know there are tourists and all this weird stuff happens at the line up and you 're like grad c: Yeah , the only like possible interpretation is that they are like come here just to rob the museum or something to that effect . grad d: In which case you 're supposed to alert the authorities , and see appropriate action . Yeah , another thing to do , is also to ,  I guess to ask around people about other Bayes - net packages . grad d: Sorry , Wednesday , grad b: Who 's talking on Wednesday ? grad c: Maybe we can ask him about it . grad b: I haven't J Jerry never sent out a sent out an email , did he , ever ? grad c: No . grad d: Ben ? grad a: Ben , then , grad d: I think it 's Ben actually , grad a: Ben . I actually  , have  , also we can  , start looking at the SmartKom tables and I will grad b: Right . grad b: Do you want to trade ? grad a: no I I actually made a mistake because it it fell asleep and when Linux falls asleep on my machine it 's it doesn't wake up ever , so I had to reboot grad d: Oh , no . grad a: And if I reboot without a network , I will not be able to start SmartKom , because I need to have a network . grad b:  grad a: So we 'll do that t maybe  grad c: But . But once you start sart start SmartKom you can be on You don't have to be on a network anymore . grad b: Why does SmartKom need a network ? grad a:  it looks up some stuff that , you know , is is that is in the written by the operating system only if it if you get a DHCP request , so it you know , my computer does not know its IP address , you know ? grad b: Ah . grad a: And I don't have an IP address , they can't look up they don't know who localhost is , and so forth and so forth . grad a: She 's willing to do it , meaning be the wizard for the data collection , also maybe transcribe a little bit , if she has to , but also recruiting subjects , organizing them , and so forth . Jerry however suggested that we should  have a trial run with her , see whether she can actually do all the  spontaneous , eloquent and creativeness that we  expect of the wizard . And I talked to Liz about this and it looks as if Friday afternoon will be the time when we have a first trial run for the data . grad c: So who would be the subject of this trial run ? grad a: Pardon me ? grad c: Who Will there be a Is one Is you one of you gonna be the subject ? Like are you grad a:  Liz also volunteered to be the first subject , which I think might be even better than us guys . grad a: If we do need her for the technical stuff , then of course one of you has to sort of  jump in . grad c: Well I just figured it has to be someone who 's , familiar enough with the data to cause problems for the wizard , so we can , see if they 're you know good .  that 's what we wanna check , right ? grad a: grad d: Well , in this case it 's a p it 's a sort of testing of the wizard rather than of the subject . grad c: Isn't that what it is ? grad d: It 's  grad a: yes w we we would like to test the wizard , but you know , if we take a subject that is completely unfamiliar with the task , or any of the set up , we get a more realistic grad c: I guess that would be reasonable . grad d: I 'm sure if we  , You think there 's a chance we might need Liz for , whatever , the technical side of things ? I 'm sure we can get other people around who don't know anything  , if we want another subject . So , is it a experimental setup for the  , data collection totally ready determined ? grad b: I like that . grad a:  I think it 's it 's it 's  experimental setup u on the technical issue yes , except we st I think we still need  a recording device for the wizard , just a tape recorder that 's running in a room . grad a: But  in terms of specifying the scenario ,    we 've gotten a little further grad d:   grad a: but  we wanted to wait until we know who is the wizard , and have the wizard partake in the ultimate sort of definition probe . So so if if on Friday it turns out that she really likes it and and we really like her , then nothing should stop us from sitting down next week and getting all the details completely figured out . So the ideal task  , will have whatever I don't know how much the structure of the evolving Bayes - net will af affect Like we wanna we wanna be able to collect as much of the variables that are needed for that , grad a: Mmm - yea - some . grad d: right ? in the course of the task ? Well not all of them but you know . grad a: Bu - e e e I 'm even This this Tango , Enter , Vista is sort of , itself , an ad - hoc scenario . So we wanted just to collect data , to get that that that elicits more , that elicits richer language . grad a: And we actually did not want to constrain it too much , grad d:   And then maybe we 'll discover the phenomenon the phenomena that we want to solve , you know , with whatever engine we we come up with . So this this this is a parallel track , you know , there they hopefully meet , grad d: OK . grad a: but since grad d: It could it could be used for not just this task . grad a: It should tell us , you know , what kind of phenomenon could occur , it should tell us also maybe something about the difference between people who think they speak to a computer versus people who think they speak to a human being grad d:   So it may get us some more information on the human - machine pragmatics , that no one knows anything about , as of yesterday . And secondly , now that of course we have sort of started to lick blood with this , and especially since  Johno can't stop Tango - ing , we may actually include , you know , those those intentions . So now I think we should maybe have at least one navigational task with with sort of explicit  grad d:   grad a: not ex it 's implicit that the person wants to enter , grad d:   grad a: and maybe some task where it 's more or less explicit that the person wants to take a picture , grad d:   grad a: Whereas , you know , if we 'd just get data we 'd never know what they actually wanted , we 'd get no cues . grad b: So is this the official end of the meeting now ? grad c: Yep . grad c: So what 's " Economics , the fallacy " ? grad a: Ma grad b: I just randomly label things . grad c: Oh , really ? grad a: Maybe we ought to switch off these things before we continue <doc-sep>OK , so , you 've got some , Xerox things to pass out ? phd a: Yeah , professor d: That are phd a:  .  , lowering the word hour rate is fine , but having big font ! phd a: Next time we will put colors or something . phd a: OK , s so there is kind of summary of what has been done professor d:  Go ahead . So since last week we 've started to fill the column with   features w with nets trained on PLP with on - line normalization but with delta also , because the column was not completely professor d:    phd a: well , it 's still not completely filled , professor d:  phd a: but we have more results to compare with network using without PLP and finally , hhh ,  ehhh PL -  delta seems very important . If you take  , let 's say , anyway Aurora - two - B , so , the next t the second , part of the table , professor d:   phd a:  when we use the large training set using French , Spanish , and English , you have one hundred and six without delta and eighty - nine with the delta . professor d: a And again all of these numbers are with a hundred percent being , the baseline performance , phd a: Yeah , on the baseline , yeah . So professor d: but with a mel cepstra system going straight into the HTK ? phd a: Yeah . So now we see that the gap between the different training set is much   much smaller professor d: Yes . phd a: But , actually , for English training on TIMIT is still better than the other languages . If you take the second set of experiment for Italian , so , the mismatched condition , professor d:   phd a:  when we use the training on TIMIT so , it 's multi - English , we have a ninety - one number , professor d:   phd a: And , yeah , and here the gap is still more important between using delta and not using delta . If y if I take the training s the large training set , it 's we have one hundred and seventy - two , professor d: Yes . Yeah , so the second point is that we have no single cross - language experiments , that we did not have last week .  , so this is training the net on French only , or on English only , and testing on Italian . phd a: And training the net on French only and Spanish only and testing on ,  TI - digits . What we see is that these nets are not as good , except for the multi - English , which is always one of the best . Yeah , then we started to work on a large dat database containing , sentences from the French , from the Spanish , from the TIMIT , from SPINE ,  from  English digits , and from Italian digits . phd a: and  , actually we did this before knowing the result of all the data , so we have to to redo the  the experiment training the net with ,  PLP , but with delta . Well , it 's it 's better than the net using French , Spanish , and English only . The first one , yeah , is combining , two feature streams ,  using and each feature stream has its own MPL . And the third one is to u use a single KLT trans transform features as well as MLP outputs . You know you can you can comment these results , phd b: Yes , I can s I would like to say that , for example , mmm , if we doesn't use the delta - delta ,  we have an improve when we use s some combination . But when phd a: Yeah , we ju just to be clear , the numbers here are  recognition accuracy . phd b: w Yeah , this Yeah , this number recognition acc phd a: So it 's not the Again we switch to another phd b: Yes , and the baseline the baseline have i is eighty - two . phd b: Yeah phd a: So it 's experiment only on the Italian mismatched for the moment for this . phd b: And first in the experiment - one I I do I I use different MLP , professor d:   And I try to combine different type of feature , but the result is that the MSG - three feature doesn't work for the Italian database because never help to increase the accuracy . phd a: Yeah , eh , actually , if w we look at the table , the huge table , we see that for TI - digits MSG perform as well as the PLP , professor d:   phd a: but this is not the case for Italian what where the error rate is c is almost  twice the error rate of PLP . phd a: So , well , I don't think this is a bug but this this is something in probably in the MSG  process that  I don't know what exactly . Perhaps the fact that the the there 's no low - pass filter , well , or no pre - emp pre - emphasis filter and that there is some DC offset in the Italian , or , well , something simple like that . But that we need to sort out if want to  get improvement by combining PLP and MSG professor d:   phd a: And as Carmen said , if we combine the two , we have the result , basically , of PLP . professor d: I  , the  , baseline system when you said the baseline system was  ,  eighty - two percent , that was trained on what and tested on what ? That was ,  Italian mismatched d  , digits , is the testing , phd b: Yeah . professor d: So the " mismatch " just refers to the noise and and ,  microphone and so forth , phd a: Yeah . professor d: right ? So ,  did we have So would that then correspond to the first line here of where the training is is the  Italian digits ? phd b: The train the training of the HTK ? professor d: The phd b: Yes . So ,  So what that says is that in a matched condition , we end up with a fair amount worse putting in the  PLP . Now w would do we have a number , I suppose for the matched I I don't mean matched , but  use of Italian training in Italian digits for PLP only ? phd b:  yes ? phd a:  yeah , so this is basically this is in the table . phd a: Fift - So No , it 's it 's the phd b: No . professor d: No , fifty - two percent of eighty - two ? phd a: Of of of  eighteen phd b: Eighty . So  professor d: Oh this is accuracy !  phd a: so we have nine nine let 's say ninety percent . professor d: OK , so even just PLP , it is not , in the matched condition  I wonder if it 's a difference between PLP and mel cepstra , or whether it 's that the net half , for some reason , is not helping . It 's not Do you have this result with PLP alone , j fee feeding HTK ? professor d: So , s phd a: That That 's what you mean ? phd b: Yeah , phd a: Just PLP at the input of HTK . professor d: so adding MSG phd a:  professor d:  Well , but that 's yeah , that 's without the neural net , phd a: Yeah , that 's without the neural net professor d: right ? phd a: and that 's the result basically that OGI has also with the MFCC with on - line normalization . phd b:  phd a: Eighty - two is the it 's the Aurora baseline , so MFCC . Then we can use well , OGI , they use MFCC th the baseline MFCC plus on - line normalization professor d: Oh , I 'm sorry , I k I keep getting confused because this is accuracy . phd a: So what happ what happens is that when we apply on - line normalization we jump to almost ninety percent . If we use n neural network , even if the features are not correctly normalized , we jump to ninety percent . So professor d: So we go from eighty - si eighty - eight point six to to ninety , or something . phd a: Well , ninety No , I  ninety It 's around eighty - nine , ninety , eighty - eight . So ,  So actually , the answer for experiments with one is that adding MSG , if you  does not help in that case . professor d: The other ones , we 'd have to look at it , but And the multi - English , does  So if we think of this in error rates , we start off with ,  eighteen percent error rate , roughly . professor d:  and we  almost ,  cut that in half by  putting in the on - line normalization and the neural net . About ,  sixteen percent or something of the error , if we use multi - English instead of the matching condition . OK ? So then you 're assuming multi - English is closer to the kind of thing that you could use since you 're not gonna have matching , data for the  for the new for the other languages and so forth .  , one qu thing is that ,  I think I asked you this before , but I wanna double check . When you say " ME " in these other tests , that 's the multi - English , phd a: That 's it 's a part it 's professor d: but it is not all of the multi - English , right ? It is some piece of part of it . professor d: And the multi - English is how much ? phd b: You have here the information . professor d: Oh , so you used almost all You used two thirds of it , phd a: Yeah . So , it it 's still it hurts you seems to hurt you a fair amount to add in this French and Spanish . grad c: Well Stephane was saying that they weren't hand - labeled , phd a: Yeah , it 's phd b: Yeah . Mmm , with the experiment type - two , I first I tried to to combine , nnn , some feature from the MLP and other feature another feature . phd b: And we s we can first the feature are without delta and delta - delta , and we can see that in the situation , the MSG - three , the same help nothing . phd b: And then I do the same but with the delta and delta - delta PLP delta and delta - delta . And they all p but they all put off the MLP is it without delta and delta - delta . And we have a l little bit less result than the the the baseline PLP with delta and delta - delta . phd b: Maybe if when we have the new the new neural network trained with PLP delta and delta - delta , maybe the final result must be better . phd a: Actually , just to be some more phd b:  phd a: Do This number , this eighty - seven point one number , has to be compared with the professor d: Yes , yeah ,  it can't be compared with the other phd a: Which number ? professor d: cuz this is ,  with multi - English , training . professor d: So you have to compare it with the one over that you 've got in a box , which is that ,  the eighty - four point six . professor d: So phd a: Yeah , but  in this case for the eighty - seven point one we used MLP outputs for the PLP net professor d: Yeah . phd b: No , but they they feature @ @ without phd a: So we use feature out  , net outputs together with features . So yeah , this is not perhaps not clear here but in this table , the first column is for MLP and the second for the features . So you 're saying w so asking the question , " What what has adding the MLP done to improve over the , phd a: So , just Yeah so , actually it it it decreased the the accuracy . phd a: And even the MLP alone What gives the MLP alone ? Multi - English PLP . So we have our eighty - three point six and now eighty - eighty point six , phd b: But phd a: that gives eighty - seven point one . Eighty - s I thought it was eighty Oh , OK , eighty - three point six and eighty eighty - eight point six . But I don't know but maybe if we have the neural network trained with the PLP delta and delta - delta , maybe tha this can help . professor d: Well , that 's that 's one thing , but see the other thing is that ,  it 's good to take the difficult case , but let 's let 's consider what that means . What what we 're saying is that one o one of the things that  my interpretation of your your s original suggestion is something like this , as motivation . When we train on data that is in one sense or another , similar to the testing data , then we get a win by having discriminant training . professor d: When we train on something that 's quite different , we have a potential to have some problems . professor d: And , if we get something that helps us when it 's somewhat similar , and doesn't hurt us too much when it when it 's quite different , that 's maybe not so bad . professor d: So the question is , if you took the same combination , and you tried it out on ,  on say digits , phd a: On TI - digits ? OK .  , then does that , eh you know maybe with similar noise conditions and so forth , does it does it then look much better ? phd a:   professor d: And so what is the range over these different kinds of  of tests ? So , an anyway . phd b: And , with this type of configuration which I do on experiment using the new neural net with name broad klatt s twenty - seven , d I have found more or less the same result . phd a: So , it 's slightly better , phd b: Little bit better ? phd a: yeah . professor d: And and you know again maybe if you use the , delta there , you would bring it up to where it was ,  you know at least about the same for a difficult case . phd a: It 's either less information from the neural network if we use only the silence output . And then w with the first configuration , I f I am found that work , doesn't work professor d: Yeah . Because I for the del Engli - PLP delta and delta - delta , here I have eighty - five point three accuracy , and with the second configuration I have eighty - seven point one . professor d: by the way , there is a another , suggestion that would apply , to the second configuration , which , was made , by , Hari . And that was that , if you have  feed two streams into HTK , and you , change the ,  variances if you scale the variances associated with ,  these streams  , you can effectively scale the streams . Right ? So , you know , without changing the scripts for HTK , which is the rule here , you can still change the variances phd a:   professor d: And , so , if you do that , for instance it may be the case that , the MLP should not be considered as strongly , for instance . professor d: And , so this is just setting them to be , excuse me , of equal equal weight . professor d: Right ? You know , I I 'm sorry to say that gives more experiments if we wanted to look at that , but but , you know on the other hand it 's just experiments at the level of the HTK recognition . Yeah , you have to change the No , you can just do it in as once you 've done the training grad c: And then you can vary it . professor d: Yeah , the training is just coming up with the variances so I guess you could you could just scale them all . But Is it i th  the HTK models are diagonal covariances , so I d Is it professor d: That 's  , exactly the point , I think , that if you change  , change what they are phd a:  . professor d: It 's diagonal covariance matrices , but you say what those variances are . professor d: So , that you know , it 's diagonal , but the diagonal means th that then you 're gonna it 's gonna it 's gonna internally multiply it and and  , i it im  implicitly exponentiated to get probabilities , and so it 's it 's gonna it 's it 's going to affect the range of things if you change the change the variances of some of the features . phd b: do ? professor d: So , i it 's precisely given that model you can very simply affect , the s the strength that you apply the features . So it could just be that h treating them equally , tea treating two streams equally is just just not the right thing to do . Of course it 's potentially opening a can of worms because , you know , maybe it should be a different number for for each kind of test set , or something , phd a:   professor d: So I guess the other thing is to take you know if one were to take , you know , a couple of the most successful of these , phd a: Yeah , and test across everything . phd a: So , the next point , yeah , we 've had some discussion with Steve and Shawn , about their  , articulatory stuff ,  . phd a: discussion with Hynek , Sunil and Pratibha for trying to plug in their our our networks with their within their block diagram , where to plug in the the network , after the the feature , before as  a as a plugin or as a anoth another path , discussion about multi - band and TRAPS , actually Hynek would like to see , perhaps if you remember the block diagram there is , temporal LDA followed b by a spectral LDA for each  critical band . And he would like to replace these by a network which would , make the system look like a TRAP . Basically , this is a TRAP system kind of TRAP system , but where the neural network are replaced by LDA .  , yeah , and about multi - band , I started multi - band MLP trainings ,  mmh Actually , I w I w hhh prefer to do exactly what I did when I was in Belgium . So I take exactly the same configurations , seven bands with nine frames of context , and we just train on TIMIT , and on the large database , so , with SPINE and everything . So , this would would be something between TRAPS and multi - band because we still have quite large bands , and but with a lot of context also . So  Yeah , we still have to work on Finnish , basically , to make a decision on which MLP can be the best across the different languages .  , well , the next part of the document is , well , basically , a kind of summary of what everything that has been done . We have seventy - nine M L Ps trained on one , two , three , four , three , four , five , six , seven ten on ten different databases . phd a: the number of frames is bad also , so we have one million and a half for some , three million for other , and six million for the last one .  , yeah ! As we mentioned , TIMIT is the only that 's hand - labeled , and perhaps this is what makes the difference . First ,  with respect to the on - line normalization , there are that use bad on - line normalization , and other good on - line normalization . With respect to the features , with respect to the use of delta or no ,  with respect to the hidden layer size and to the targets . s What 's this ? We only have two hundred eighty six different tests And no not two thousand . phd b: I say this morning that @ @ thought it was the professor d: Alright , now I 'm just slightly impressed , OK . but when the M - MLP is trained on the  is not trained on the target task , it increased the error rate compared to using straight features . Except if the features are bad  , actually except if the features are not correctly on - line normalized . In this case the tandem is still better even if it 's trained on not on the target digits . phd a: so the fourth point is , yeah , the TIMIT plus noise seems to be the training set that gives better the best network . professor d: So , on the MSG  problem  , I think that in in the  , in the short time solution  , that is , trying to figure out what we can proceed forward with to make the greatest progress , phd a:   professor d: much as I said with JRASTA , even though I really like JRASTA and I really like MSG , phd a:   professor d: I think it 's kind of in category that it 's , it it may be complicated . professor d: And  it might be if someone 's interested in it , certainly encourage anybody to look into it in the longer term , once we get out of this particular rush  for results . professor d: But in the short term , unless you have some some s strong idea of what 's wrong , phd a: I don't know at all but I 've perhaps I have the feeling that it 's something that 's quite quite simple or just like nnn , no high - pass filter professor d: Yeah , probably . professor d: There 's supposed to well MSG is supposed to have a an on - line normalization though , right ? phd a: It 's There is , yeah , an AGC - kind of AGC . professor d: Yeah , but also there 's an on - line norm besides the AGC , there 's an on - line normalization that 's supposed to be  , yeah , phd a: Mmm . professor d: In fac in fact the on - line normalization that we 're using came from the MSG design , phd a:  . Are your results are still with the bad the bad phd b: Maybe , may No ? With the better phd a: With the O - OLN - two ? phd b: No ? phd a: Ah yeah , you have you have OLN - two , phd b: Oh ! Yeah , yeah , yeah ! With " two " , with " on - line - two " . professor d: " Two " is good ? phd a: And professor d: No , " two " is bad . It 's probably something simple  , i if if  someone , you know , wants to play with it for a little bit . professor d: but but my my guess would be that it 's something that is a simple thing that could take a while to find . That 's that what we were concerned about is that if it 's not on the target task If it 's on the target task then it it it helps to have the MLP transforming it . professor d: If it  if it 's not on the target task , then , depending on how different it is ,  you can get  , a reduction in performance . professor d: And the question is now how to how to get one and not the other ? Or how to how to ameliorate the the problems . professor d: because it it certainly does is nice to have in there , when it when there is something like the training data . So , the the reason Yeah , the reason is that the perhaps the target the the task dependency the language dependency , and the noise dependency professor d: So that 's what you say th there . phd a: Well , the e e But this is still not clear because , I I I don't think we have enough result to talk about the the language dependency . Well , the TIMIT network is still the best but there is also an the other difference , the fact that it 's it 's hand - labeled .  , I d I don't think we want to mess with the microphones but it 's  Just  , have a seat . s Summary of the first  ,  forty - five minutes is that some stuff work and works , and some stuff doesn't OK , phd a: We still have  this One of these perhaps ? phd b: Yeah .  professor d: Yeah , I guess we can do a little better than that but I think if you if you start off with the other one , actually , that sort of has it in words and then th that has it the associated results . So you 're saying that  , although from what we see , yes there 's what you would expect in terms of a language dependency and a noise dependency . That is , when the neural net is trained on one of those and tested on something different , we don't do as well as in the target thing . But you 're saying that  , it is Although that general thing is observable so far , there 's something you 're not completely convinced about . What what do you mean ? phd a: mmm , that the the fact that s Well , for for TI - digits the TIMIT net is the best ,  which is the English net . But you have two two effects , the effect of changing language and the effect of training on something that 's Viterbi - aligned instead of hand hand - labeled . professor d: Do you think the alignments are bad ?  , have you looked at the alignments at all ? What the Viterbi alignment 's doing ? phd a: Mmm . Because , that is just looking but  ,  It 's not clear to me you necessarily would do so badly from a Viterbi alignment . But , perhaps it 's not really the the alignment that 's bad but the just the ph phoneme string that 's used for the alignment professor d: Aha ! phd a: Mmm .  professor d: The pronunciation models and so forth phd a:  for We It 's single pronunciation ,  professor d: Aha . phd a: so we asked people to listen to the  the sentence and we gave the phoneme string and they kind of correct them .  , the third The third  issue is the noise dependency perhaps but , well , this is not clear yet because all our nets are trained on the same noises and professor d: I thought some of the nets were trained with SPINE and so forth . professor d: OK , yeah , just don't just need more more results there with that @ @ .  , with respect to the network size , there 's one experiment that 's still running and we should have the result today , comparing network with five hundred and one thousand units . We can , we can tell which training set gives the best result , but we don't know exactly why . professor d: " Multi - multi - English " just means " TIMIT " , phd a: Yeah . professor d: It 's sort of , yes it 's mul it 's multi -  - purpose . So Yeah , the training targets actually , the two of the main issues perhaps are still the language dependency and the noise dependency . And perhaps to try to reduce the language dependency , we should focus on finding some other kind of training targets . For moment you use we use phonetic targets but we could also use articulatory targets , soft targets , and perhaps even ,  use networks that doesn't do classification but just regression so  , train to have neural networks that  , professor d:   phd a: does a regression and well , basically com com compute features and noit not , nnn , features without noise . phd a:  professor d: Yeah , that seems like a good thing to do , probably ,  not  again a short - term sort of thing . professor d:  one of the things about that is that  it 's e u the ri I guess the major risk you have there of being is being dependent on very dependent on the kind of noise and and so forth . phd a: So , this is w w i wa wa this is one thing , this this could be could help could help perhaps to reduce language dependency and for the noise part  we could combine this with other approaches , like , well , the Kleinschmidt approach . I think Kleinschmidt was using more than fifty different noises to train his network , phd b: Yeah . phd a: and So this is one approach and the other is multi - band  , that I think is more robust to the noisy changes . phd a: So perhaps , I think something like multi - band trained on a lot of noises with  , features - based targets could could could help . professor d: Yeah , if you i i It 's interesting thought maybe if you just trained up  w yeah , one one fantasy would be you have something like articulatory targets and you have  some reasonable database ,  but then which is  copied over many times with a range of different noises , phd a:   professor d: And  If Cuz what you 're trying to do is come up with a a core , reasonable feature set which is then gonna be used  , by the the   system . The future work is , well , try to connect to the to make to plug in the system to the OGI system . professor d: And I guess , you know , the the the real open question , e u there 's lots of open questions , but one of the core quote " open questions " for that is  , if we take the  you know , the best ones here , maybe not just the best one , but the best few or something You want the most promising group from these other experiments .  , how well do they do over a range of these different tests , not just the Italian ? phd a: Mmm , professor d:  . professor d: y Right ? And then  then see , again , how We know that there 's a mis there 's a  a a loss in performance when the neural net is trained on conditions that are different than than ,  we 're gonna test on , but well , if you look over a range of these different tests  , how well do these different ways of combining the straight features with the MLP features ,  stand up over that range ? phd b:   And  , take let 's say , just take  multi - English cause that works pretty well for the training . How does that How does that compare between the phd a: So all the all the test sets you mean , yeah . professor d: All the different test sets , phd a: And professor d: and for and for the couple different ways that you have of of of combining them . And perhaps doing this for cha changing the variance of the streams and so on getting different scaling phd b:   Yeah , so thi this sh would be more working on the MLP as an additional path instead of an insert to the to their diagram . Perhaps the insert idea is kind of strange because nnn , they they make LDA and then we will again add a network does discriminate anal nnn , that discriminates , professor d: Yeah . It 's a little strange phd a: or ? Mmm ? professor d: but on the other hand they did it before . And because also perhaps we know that the when we have very good features the MLP doesn't help .  , we we wanna get their path running here , right ? If so , we can add this other stuff . professor d: as an additional path right ? phd a: Yeah , the the way we want to do professor d: Cuz they 're doing LDA RASTA . phd a: The d What ? professor d: They 're doing LDA RASTA , phd a: Yeah , the way we want to do it perhaps is to just to get the VAD labels and the final features . professor d: yeah ? phd a: So they will send us the Well , provide us with the feature files , professor d: I see . phd a: and with VAD  , binary labels so that we can  , get our MLP features and filter them with the VAD and then combine them with their f feature stream . First thing of course we 'd wanna do there is to make sure that when we get those labels of final features is that we get the same results as them . You mean Oh , yeah ! Just re re retraining r retraining the HTK ? professor d: Yeah just th w i i Just to make sure that we have we understand properly what things are , our very first thing to do is to is to double check that we get the exact same results as them on HTK . But ,  just for the testing , jus just make sure that we get the same results so we can duplicate it before we add in another phd a: Mmm . Yeah , so fff , LogRASTA , I don't know if we want to We can try networks with LogRASTA filtered features . But professor d: Oh ! You know , the other thing is when you say comb I 'm I 'm sorry , I 'm interrupting . that u  , when you 're talking about combining multiple features ,  Suppose we said , " OK , we 've got these different features and so forth , but PLP seems pretty good . We have different languages , we have different different noises ,  If we have some drastically different conditions and we just train up different M L Ps with them . What what What Mike found , for the reverberation case at least , who knows if it 'll work for these other ones . That is , that yes , if you knew what the reverberation condition was gonna be and you trained for that , then you got the best results . But if you had , say , a heavily - reverberation ca heavy - reverberation case and a no - reverberation case , and then you fed the thing ,  something that was a modest amount of reverberation then you 'd get some result in between the two . So you you think it 's perhaps better to have several M L Yeah but professor d: It works better if what ? phd a: Yea professor d: I see . Well , see , i oc You were doing some something that was So maybe the analogy isn't quite right . Here the problem seems to be is that we don't have a hug a really huge net with a really huge amount of training data . But we have s f for this kind of task , I would think , sort of a modest amount . We have a modest amount of of  training data from a couple different conditions , and then  in yeah , that and the real situation is that there 's enormous variability that we anticipate in the test set in terms of language , and noise type  , and  , channel characteristic , sort of all over the map . And so , I 'm just concerned that we don't really have  , the data to train up  one of the things that we were seeing is that when we added in we still don't have a good explanation for this , but we are seeing that we 're adding in  , a fe few different databases and  the performance is getting worse and  , when we just take one of those databases that 's a pretty good one , it actually is is is is is better . And  that says to me , yes , that , you know , there might be some problems with the pronunciation models that some of the databases we 're adding in or something like that . But one way or another we don't have  , seemingly , the ability to represent , in the neural net of the size that we have , all of the variability that we 're gonna be covering . So that I 'm I 'm I 'm hoping that  , this is another take on the efficiency argument you 're making , which is I 'm hoping that with moderate size neural nets , that  if we if they look at more constrained conditions they they 'll have enough parameters to really represent them .  i i e The  I think it 's true that the OGI folk found that using LDA RASTA , which is a kind of LogRASTA , it 's just that they have the  it 's done in the log domain , as I recall , and it 's it  it 's just that they d it 's trained up , right ? That that  benefitted from on - line normalization . So will it be in our case , where we 're using the neural net ?  they they were not not using the neural net . We 've been sort of ignoring that , haven't we ? phd a: Yeah , so I don't know . professor d: But phd a: But we have to address the problem of CPU and memory we professor d: Yeah , but I li Well , I think My impression You you folks have been looking at this more than me . But my impression was that  , there was a a a a strict constraint on the delay , phd b: Yeah . professor d: but beyond that it was kind of that  using less memory was better , and using less CPU was better . We have to get some reference point to where we Well , what 's a reasonable number ? Perhaps be because if it 's if it 's too large or large or @ @ professor d: well I don't think we 're  completely off the wall .  I think that if we if we have  ,  the ultimate fall back that we could do If we find   we may find that we we 're not really gonna worry about the M L You know , if the MLP ultimately , after all is said and done , doesn't really help then we won't have it in . professor d: If the MLP does , we find , help us enough in some conditions , we might even have more than one MLP . professor d: And it 's  We do the other manipulations that we 're doing before that .  , what what are they What are they gonna be working Do we know what they 're gonna be working on while we take their features , phd a: They 're They 're starting to wor work on some kind of multi - band . Sunil , what was he doing , do you remember ? phd b: Sunil ? phd a: Yeah . He was doing something new or ? phd b: I I don't re I didn't remember . phd a: I think they were also mainly , well , working a little bit of new things , like networks and multi - band , but mainly trying to tune their their system as it is now to just trying to get the best from this this architecture . So I guess the way it would work is that you 'd get There 'd be some point where you say , " OK , this is their version - one " or whatever , and we get these VAD labels and features and so forth for all these test sets from them , phd a:   We have a certain level we try to improve it with this other path and then  , when it gets to be  , January some point  , we say , " OK we we have shown that we can improve this , in this way . So now   what 's your newest version ? " And then maybe they 'll have something that 's better and then we we 'd combine it .  I I I used to work with  folks who were trying to improve a good  ,  system with  with a neural net system and  , it was a common problem that you 'd Oh , and this Actually , this is true not just for neural nets but just for in general if people were working with  , rescoring  , N - best lists or lattices that come came from  , a mainstream recognizer .  , You get something from the the other site at one point and you work really hard on making it better with rescoring . So by the time you have  , improved their score , they have also improved their score phd a: Mmm . professor d: So , I guess at some point we 'll have to phd a: So it 's professor d: I I don't know . I think we 're we 're integrated a little more tightly than happens in a lot of those cases . I think at the moment they they say that they have a better thing we can we e e phd a: Mmm . professor d: What takes all the time here is that th we 're trying so many things , presumably  , in a in a day we could turn around  , taking a new set of things from them and and rescoring it , phd a: Mmm . I think that the most wide open thing is the issues about the  , you know , different trainings . So we we can for we c we can forget combining multiple features and MLG perhaps , professor d: That 's sort of wide open . phd a: or focus more on the targets and on the training data and ? professor d: Yeah , I think for right now  , I th I I really liked MSG . And I think that , you know , one of the things I liked about it is has such different temporal properties . And  , I think that there is ultimately a really good  , potential for , you know , bringing in things with different temporal properties .  , but  , we only have limited time and there 's a lot of other things we have to look at . professor d: And it seems like much more core questions are issues about the training set and the training targets , and fitting in  what we 're doing with what they 're doing , and , you know , with limited time . And then , you know , once we  , having gone through this process and trying many different things , I would imagine that certain things  , come up that you are curious about  , that you 'd not getting to and so when the dust settles from the evaluation  , I think that would time to go back and take whatever intrigued you most , you know , got you most interested  and  and and work with it , you know , for the next round .  , as you can tell from these numbers  , nothing that any of us is gonna do is actually gonna completely solve the problem . professor d: Well I figured that , but That what what what were you involved in in this primarily ? grad c: helping out  , preparing Well , they 've been kind of running all the experiments and stuff and I 've been  ,  w doing some work on the on the preparing all all the data for them to to  , train and to test on . Right now , I 'm I 'm focusing mainly on this final project I 'm working on in Jordan 's class . What 's what 's that ? grad c: I 'm trying to  So there was a paper in ICSLP about  this this multi - band  , belief - net structure . grad c:  basically it was two H M Ms with with a with a dependency arrow between the two H M professor d:  - huh . grad c: And so I wanna try try coupling them instead of t having an arrow that that flows from one sub - band to another sub - band . And  , I 'm just gonna see if if that that better models  ,  asynchrony in any way or  Yeah . OK , so speaking of which , if we don't have anything else that we need You happy with where we are ? phd a: Mmm . professor d: Know know wher know where we 're going ?  phd a: I think so , yeah . grad e: Al - actually I should mention So if  , about the Linux machine " Swede . grad e: And  Dan Ellis I believe knows something about using that machine so phd a: Mmm . grad e: If people are interested in in getting jobs running on that maybe I could help with that . phd a: Yeah , but I don't know if we really need now a lot of machines . professor d: Right ?  there 's there 's some different things that we 're trying to get at now . Yeah , as far as you can tell , you 're actually OK on C - on CPU  , for training and so on ? Yeah . Well , more is always better , but mmm , I don't think we have to train a lot of networks , now that we know We just select what works fine professor d: OK . to work professor d: And we 're OK on And we 're OK on disk ? phd a: and It 's OK , yeah . Alright , so  , since  , we didn't ha get a channel on for you , you don't have to read any digits but the rest of us will . We didn't  I think I won't touch anything cuz I 'm afraid of making the driver crash which it seems to do , pretty easily . OK , so we 'll  I 'll start off the   connect the phd a: My battery is low <doc-sep>You know how to toggle the display width function grad a: Well maybe she hadn't just started transcribing me yet .  , so we haven't had a meeting for a while , and and probably won't have one next week , I think a number of people are gone .  , so Robert , why don't you bring us up to date on where we are with EDU ? grad b:  in a in a smaller group we had  , talked and decided about continuation of the data collection . So Fey 's time with us is almost officially over , and she brought us some thirty subjects and , t collected the data , and ten dialogues have been transcribed and can be looked at . She 's actually suggested to look  , at the psychology department students , because they have to partake in two experiments in order to fulfill some requirements . And  , we want to design it so that they really have to think about having some time , two days , for example , to plan certain things and figure out which can be done at what time , and , sort of package the whole thing in a in a re in a few more complicated  , structure . As for SmartKom , I 'm the last SmartKom meeting I mentioned that we have some problems with the synthesis , which as of this morning should be resolved . grad b: " should be " means they aren't yet , but but I think I have the info now that I need . Plus , Johno and I are meeting tomorrow , so maybe   , when tomorrow is over , we 're done . And ha n hav we 'll never have to look at it again Maybe it 'll take some more time , to be realistic , but at least we 're we 're seeing the end of the tunnel there .  , the  ,  I don't think we need to discuss the formalism that 'll be done officially s once we 're done .  , something happened , in on Eva 's side with the PRM that we 're gonna look at today , and  , we have a visitor from Bruchsal from the International University . grad a: So when you said " Andreas " I thought you were talking about Stolcke . grad b: And , grad a: Now I know that we aren't , OK . grad b: Eh grad c: Cuz there is another Andreas around , grad a:  . grad b: So my scientific director of the EML is also the dean of the International University , one of his many occupations that just contributes to the fact that he is very occupied . And , the  , he @ @ might tell us a little bit about what he 's actually doing , and why it is s somewhat related , and by  using maybe some of the same technologies that we are using .  , so , I 've be just been looking at , Ack ! What are you doing ? Yeah .  , so , this is , sort of like the latest thing I have on it , and I sorta constructed a couple of classes . Like , a user class , a site class , and and you know , a time , a route , and then and a query class . And I tried to simplify it down a little bit , so that I can actually  , look at it more .  , so basically I took out a lot of stuff , a lot of the decision nodes , and then tried to The red lines on the , graph are the  , relations between the different  , classes . Like , a user has like , a query , and then , also has , you know  , reference slots to its preferences , the special needs and , you know , money , and the user interest . And so this is more or less similar to the flat Bayes - net that I have , you know , with the input nodes and all that . And So I tried to construct the dependency models , and a lot of these stuff I got from the flat Bayes - net , and what they depend on , and it turns out , you know , the CPT 's are really big , if I do that , so I tried to see how I can do ,  put in the computational nodes in between . And so I ended up making several classes Actually , you know , a class of with different attributes that are the intermediate nodes , and one of them is like , time affordability money affordability , site availability , and the travel compatibility . And so some of these classes are s some of these attributes only depend on stuff from , say , the user , or s f just from , I don't know , like the site . S like , these here , it 's only like , user , but , if you look at travel compatibility for each of these factors , you need to look at a pair of , you know , what the  , preference of the user is versus , you know , what type of an event it is , or you know , which form of transportation the user has and whether , you know , the onsite parking matters to the user , in that case . And that makes the scenario a little different in a PRM , because , then you have one - user objects and potentially you can have many different sites in in mind . And , they all depend on the same users , but different sites , and that makes a I 'm tr I w I wa have been trying to see whether the PRM would make it more efficient if we do inferencing like that . And so , I guess you end up having fewer number of nodes than in a flat Bayes - net , cuz otherwise you would c well , it 's probably the same . But  , No , you would definitely have be able to re - use , like , all the user stuff , and not not having to recompute a lot of the stuff , because it 's all from the user side . But , you know , in the case where , it depends on both the user and the site , then I 'm still having a hard time trying to see how  , using the PRM will help .  , so anyhow , using those intermediate nodes then , this this would be the class that represent the intermediate nodes . And that would basically it 's just another class in the model , with , you know , references to the user and the site and the time . And then , after you group them together this no the dependencies would of the queries would be reduced to this .  , so I think that 's about as far as I 've gone on the PRM stuff . And one is basically how desirable a site is meaning , how good it matches the needs of a user .  , so , instead of  , doing a lot of , you know , computation about , you know , which one site it wants of the user wants to visit , I 'll come well , try to come up with like , sort of a list of sites . And for each site , you know , where h how how well it fits , and basically a rating of how well it fits and what to do with it . She 's ac   Eva 's got a little write - up on it that  , probably gives the the details to anybody who needs them .  , so the You you didn't look at all yet to see if there 's anybody has a implementation . So one so one of the questions , you know , about these P R Ms is grad d:   professor f: we aren't gonna build our own interpreter , so if if we can't find one , then we  , go off and do something else and wait until s one appears .  , so one of the things that Eva 's gonna do over the next few weeks is see if we can track that down . If there is an interpreter , it looks like you know , what Eva 's got should run and we should be able to actually  , try to solve , you know , the problems , to actually take the data , and do it .  , I actually think it is cleaner , and the ability to instantiate , you know , instance of people and sites and stuff , will help in the expression . I think there are things that you can express this way which you can't express in a normal belief - net , without going to some incredible hacking of sort of rebuilding it on the fly .  , the notion of instantiating your el elements from the ontology and stuff fits this very nicely and doesn't fit very well into the extended belief - net . So , people who have thought about the problem , like Robert i it looked to me like if Eva were able to come up with a you know , value for each of a number of  , sites plus its EVA thing , that a travel planner should be able to take it from there . And you know , with some other information about how much time the person has and whatever , and then plan a route . grad b:  -  , well , first of all  , great looks , mu much cleaner , nnn , nnn , Certain certain beauty in it , so , if beauty is truth , then ,  we 're in good shape . So if you have a write - up then  , I 'd love to read it grad d:   grad b: and  because , i Can you go all the way back to the the very top ? grad d: Yeah . grad b:  these @ @ these w w when these are instantiated they take on the same values ? that we had before ? grad d: I can't really see the whole thing . grad b: or are they have they changed , in a sense ? grad d: Well I think I basically leave them to similar things . grad d: Some of the things might that might be different , maybe like are that the hours for the site . grad d: And , eventually I meant that to mean whether they 're open at this hour or not . grad d: And status would be , you know , more or less like , whether they 're under construction , and and or stuff like that . grad b: And the , other question I would have is that presumably , from the way the Stanford people talk about it , you can put the probabilities also on the relations . I don't remember whether they carried that over to this or not , grad a: Mmm . grad b: It 's sort of in the definition or in the in Daphne 's definition of a PRM is that classes and relations , professor f: OK . grad b: and you 're gonna have CPT 's over the classes and their relations . grad d: I remember them learning when , you know , you don't know the structure for sure , professor f: Yeah . grad d: but I don't remember reading how you specify grad b: Yeah , that would be exactly my question . So , the the plan is is when Daphne gets back , we 'll get in touch and supposedly , we 'll actually get s deep seriously connected to to their work and grad b: Yep . professor f: somebody 'll  , you know If it 's a group meeting once a week probably someone 'll go down and , whatever . We get rocking and rolling on this again , once we get a package , if , when , and how , then this becomes foregrounded grad d:   grad b: And  , until then we 'll come up with a something that 's @ @ that 's way more complicated for you . grad b: Because this was laughingly easy , right ? grad d: Actually I had to take out a lot of the complicated stuff , cuz I I made it really complicated in the beginning , and Jerry was like , " this is just too much " . So , you could , from this , go on and say suppose there 's a group of people traveling together and you wanted to plan something that somehow , with some Pareto optimal  , thing for grad a: That 's good . professor f: or grad a: Except for humans can't really solve it either , so . Well that 's the that would that would be a  , you could sell it , as a grad a: Yeah . professor f: OK , eh you don't have to fight about this , just give your preferences to the grad a: And then you can blame the computer . But what does it  Would a pote potential result be to to split up and never talk to each other again ? You know . So there i there are some some u  , you know , elaborations of this that you could try to put in to this structure , but I don't think it 's worth it now . But  , it 's good , yeah and and there were a couple other ideas of of  , things for Eva to look at in in the interim . So , I 'm not really involved in any project , that 's  that 's relevant to you  , a at the moment , the the reason is really for me  , to have an opportunity to talk to some other researchers in the field . And and so I 'll just n sort of give you a real quick introduction to what I 'm working on , and  , I just hope that you have some comments or , maybe you 're interested in it to find out more , and and so I 'll be  , happy to talk to you and and  , I 'd also like to find out some more and and maybe I 'll just walk around the office and and then and ask some some questions , in a couple days . OK , so , what I started looking at , to begin with is just  , content management systems  , i i in general . So  ,  what 's  Sort of the state of the art there is to   you have a bunch of of  documents or learning units or learning objects , and you store meta - data  , associate to them . So there 's some international standards like the I - triple - E ,  There 's an I - triple - E , LON standard , and  , these fields are pretty straightforward , you have  author information , you have  , size information , format information and so on . One is  you store keywords associated with the  with the document , and one is  , you have sort of a , well , what is the document about ? So it 's some sort of taxonomic  , ordering of of the of the units . Now , if you sort of put on your semantic glasses ,  you say , well that 's not all that easy , because there 's an implicit  , assumption behind that is that  , all the users of this system share the same interpretation of the keyword and the same interpretation of  , whichever taxonomy is used , and  , I think that 's a that 's a very that 's a key point of these systems and they sort of always brush over this real quickly without really elaborating much of that and  As a matter of fact , the only thing that m apparently really works out so far are library ordering codes , which are very , very coarse grain , so you have some like , science , biology , and then But that 's really all that we have at the moment . Now , what this  a standard like this would give us is we could  , sort of  with a search engine just query  , different repositories all over the world . But we can't really  , so what I 'm what I try to do is  , to have  ,  So . So the scenario is the following , you you 're working on some sort of project and you encounter a certain problem . Now , what what we have at our university quite a bit is that  , students  , try to u program a certain assignment , for example , they always run into the same problems , and they always come running to us , and they 'll say why 's it not it 's not working , and we always give out the same answer , so we thought , well , it 'd be nice to have a system that could sort of take care of this , and so , what I want to build is basically a a smart F A Q system . Now , what you  need to do here is you need to provide some context information which is more elaborate than " I 'm looking for this and this and this keyword . I 'm I 'm sure you have the same when when somebody utters a sentence in a certain , context it , and and the same sentence in another context makes a huge difference . So , I want to be able to model information like , so in the in the context of in the context of developing distributed systems , of a at a computer science school , what kind of software is the person using , which homework assignment is he or she working on at the moment , maybe what 's the background of that student 's  , which  , which error message was encountered . Now , basically giving this   so we somehow need to have a formalized  , way of writing this down basically , and that 's where the shared interpretation of of certain terms and keywords comes in again . And , using this and some some  , knowledge about the domain I think you can do some some simple inferences . Like you know that when somebody 's working about  , working on on servlets for example , he 's using Java , cuz servlets are used are written in Java . So some some inferences like that , now , u using this you can infer more information , and you could then match this to the meta - data of  off the documents you 're you 're searching against . So ,  what I wanna do is basically have some sort of  given these inputs , and then I can compute how many documents match , and use this as a metric in the search . Now , what I plan to do is I want to  sort of do a   try to improve the quality of the search results , and I want to do this by having a depth  ,  steepest descent approach . So if I knew which operating system the person was working on , would this improve my search result ? And and having  ,  a symbolic formalized model of this I could simply compute that , and find out which  which questions are worth  , asking . And that 's what I then propagate back to the user , and and sort of try to optimize the search in this way . Now , the big problem that I 'm facing right now is  , it 's fairly easy to hack up a system  quickly , that that works in the small domain , but the problem is obviously the scalability . And   , so Robert was mentioning  , earlier today is that  , Microsoft for example with their printer set up program has a Bayesian network , which does exactly this , but there you face a problem that these are very hard to extend . And so ,  what I 'm What I try to do is basically try to model this  , in a way that you could really combine  , knowledge from very different sources , and and  , sort of looking into some of the ideas that the semantic web community  , came up with . Trying to to have  , an approach how to integrate s  certain  representation of certain concepts and also some computational rules , what you can do with those . What I 'm also looking into is a probabilistic approach into this because document retrievals is a very fuzzy procedure , so it 's probably not that easy to simply have a symbolic  , computational model . And then  ,  sort of as an add - on to this whole idea ,  that would be now , depending on what the search engine or the content repository depending on which  , which  , rules and which ontologies it it uses , or basically its view of the world ,  you can get very different results . And there you could have an idea where you actually have sort of a a peer to peer approach , where we 're all sort of carrying around our individual bookshelves , and  , if you have a question about a homework , it 's probably makes sense to ask somebody who 's in your class with you , sort of the guru in the certain area , rather than going to some Yahoo - like  , search engine . And I think a lot of the even though it 's a it 's a very different domain , but I think a lot of the , issues are are fairly similar . grad a: And so some of the I don't know how much you know about the larger Heidelberg project , I Are you grad c:  I know , yeah I know abou about it . It 's like , you know , the c context - based factors that influence how you interpret , grad c:   We - we 've kind of talked about that , but we haven't worried too much about that end of the discourse . grad b: Well , in a in one t one s mmm , small difference in a in a way , is that he doesn't have to come up with an answer , but he wants to point to the places w w grad a: Documents that have the answers . So I 'm I 'm not I 'm not building an expert grad a: you have to s still m understand what the content says about itself , and then match it to what you think the informational needs grad a:   You 're just taking the keywords as a topic text , as grad c: I I assume that that the there will be learning systems that that tag their their content . grad c: And  , m @ @ and basically what I what I envision is that you rather than just supplying a bunch of keywords you could basically for for an FAQ for example you could state sort of like a logic condition , when this document applies . So " this document explains how to set up your  , mail account on Linux " or something like this . So something something very specific that you can then But the I think that the key point with these  , learning systems is that  , a learning system is only as good as  the amount of content it it carries . grad c: You can have the best learning system with the best search interface , if there 's no content inside of it , it 's not very useful . So I think ultimately because  ,  developing these these rules and these inference  inferences I think is very costly , so  ,  I think you must be able to reuse some some existing  , domain domain information , or or or ontologies that that  other people wrote and then try to integrate them , and then also search the entire web basically , rather than just the small  , content management system . grad c: So I think that 's that 's crucial for for the success of or @ @ grad a: So , you 're not I guess I 'm trying to figure out how how it maps to the kinds of things that we 've talked about in this group , and , actually associated groups , grad c:   grad a: cuz some of us do pretty detailed linguistic analyses , and I 'm guessing that you you won't be doing that ? OK . grad a: So , you take the query , and and professor f: On the other hand , FrameNet could well be useful . professor f: Because , you know , the standard story is that keyworks keywords evoke frames , and the frames may well give you additional keywords or  , if you know that that that a a bunch of keywords  , indicate a frame , then you can find documents that actually have the whole frame , rather th than just  , individual grad c: Mmm . There 's linguists and stuff and there 's a lot of it and they 're they 're busily working away . And this looks t it seems to be that this is a place where you might be able to do that . I 'm sure I could learn a lot about  , yeah , just how to how to come up with these structures , grad a: Mmm . grad c: cuz it 's it 's very easy to whip up something quickly , but it maybe then makes sense to to me , but not to anybody else , and and if we want to share and integrate things , they must well , they must be well designed really . grad b: And Andreas and I tried to come up wi or we had come up actually with a eh with him working on an interface for FrameNet , as it was back then , that would p do some of the work for this machine , professor f: Right . grad b: which  , never got done because Prashant found a happy occupation professor f: W yeah , I know ,  it it he w he did w what what he did was much more s sensible for him . professor f: I think  , grad b: But so I 'm just saying , the  , we had that idea professor f: you know Yeah . professor f: Yeah , actually you guys never grad b: And Srini 's doing information extraction also , right ? professor f: Right . grad c: professor f: this was supposedly an exchange program , and I we you know , it 's fine . We don't care , but it just I 'm a little surprised that  , Andreas didn't come up with anyone else he wanted to send .  I had forgotten a I To be honest with you , I 'd totally forgotten we had a program . grad b:  it 's in the program ? grad c:  I I think it 's it 's really the lack of students  , at IU at the moment . I think it 's ju it 's more the lack of of students , really , and w we have all these sponsors that are always sort of eager to get some teams . grad c: Well  if if I were a student , I 'd love to come here , rather than work for some German company , or professor f: Yeah . professor f: Oh , right ! grad c: Well , I didn't say anybody to anything to offend well , except for the sponsors maybe , but professor f: Right . So I thi tha that 's that 's one of the things that might be worth looking into while you 're here . professor f: unfortunately , Srini , who is heavily involved in DAML and all this sort of stuff is himself out of town . professor f: I don't know why he @ @ oh , I , who knows ? grad a: The  professor f: Anyway , s yeah , you 'll see you 'll certainly see a lot of the people there . grad a: The other person I thought of is Dan Gildea ? because he did some work on topic spotting professor f: Yeah . I don't Depending on how well you wanna integrate with that end , grad c:   grad a: you know , like , taking the data and fig you said the learning systems that figure out We There 's someone in ICSI who actually has been working on has worked on that kinda stuff , and he 's worked with frame net , so you could talk to him about , you know , both of those things at once . grad c: So , who is that again ? grad a: And , he 's in one of the rooms on the fifth floor and stuff , grad b: Who ? I can take you to his office . professor f: Alright , was there anything else for this ? One of these times soon we 're gonna hear about construal . professor f: Good thinking ! grad b:  well , maybe I can I can bribe my way out of this . professor f: Well , no , but he 's he 's he 's he 's as you said , he 's , like the state legislature , he 's trying to offer us bribes . grad b: This  Oh , they refused the budget again ? Is it so about CITRIS ? Yeah , still nothing . professor f: this this this t the s we 're , involved in a literally three hundred million dollar  , program . And , the State of California is now a month and a half behind its legis its legally required date to approve a budget . professor f: and , yesterdayday there was this   I thought that the other branch would just approve it , but now there 's actually a little back sliding to people who who approved it got flak from there , eh anyway . So ,  Oh ! I have to tell you a wonderful story about this , OK ? And then we 'll go . In fact he 's hired to run a lot of CITRIS , even though we don't have the money they So they 've been hiring people right and left , so , they think the money 's coming . And , is now doing all the politics for CITRIS , but also , has a  , a lot of interest in  , actually doing things for society , so digital divide and stuff like that . But the really interesting thing was , he st he s he s said something about , you know I 'm interested in things that have high social multiplier , something that is of great social value . He said , " for example " , this was his only example , " if you had a adult literacy program that was as good as an individual tutor , and as compelling as a video game , then that would have a huge social impact " . So it was nice that  , he 's got this view , of A , that 's what you should try to do , and B , language would be a good way to do it . I didn't push him on the ch on the child thing , grad a:  - huh . professor f: but , you know , a again , if if you if you grad a: Oh . So this is reading , rather than teaching Another project we started on , and and didn't get funded for was , to try to build an automatic tutoring program , for kids whose first language wasn't English . So , enormous problem in California , and the idea was if we 're so smart about language understanding and speech understanding , couldn't we build  , programs that would be tutors for the kids . professor f: and  , I know none of us have the spare time to look at it right now , but it i it 's it 's interesting and I may  , talk to him some more about is em somebody already doing this , and stuff like that . So I I did manage to get pull my head out of the sling by sidetracking into CITRIS , professor f: No , no . grad b: but , I I 'll volunteer to put it right back in by stating that I am n  among some other things in the process of writing up stuff that we have been discussing at our daily meetings , professor f: Yeah . grad b: and also revising , thanks for all the comments , the c the original construal proposal . And , if I put one and one together , I may end up with a number that 's greater than one and that I I can potentially present once you get back . Yeah , so OK , so that 'd be great , but I 'd I think it 's it 's time again , right ? grad b: Absolutely . grad b: But  , and hopefully all sidetracking  , other things will have disappeared , soon 
The team discussed the best approach for incorporating the recording infrastructure in the room, considering options such as placing tapes in the lip of the table and using LEDs to indicate when the channel is live. Power issues were also a topic of discussion. In terms of representing the Bayes-net, an alternative approach was proposed that depicts context features as classes and their dependencies as relations. This model allows for instantiations of classes, which aligns better with the research objectives compared to the extended belief-net. Recent experiments revealed that adding more languages had a detrimental effect on model performance. However, it was noted that any form of normalization had a positive impact, while MSG had no effect. A demonstration was given on the structure and function of a toy version of the belief-net for the intentionality task. The feature nodes included prosody, discourse, verb choice, the "landmark-iness" of a building, time of day, and whether the admission fee was discussed.
Summarize the discussion about electronics, the segment, the results from the experiments, and the topic of intentionality.