grad g: So I think maybe what 's causing it to crash is I keep starting it and then stopping it to see if it 's working . postdoc b: And it looks like you 've found a way of  mapping the location to the without having people have to give their names each time ? phd a: Sounds like an initialization thing . postdoc b:  it 's like you have the So you know that grad g: No . postdoc b: are you going to write down that I sat here ? grad g: I 'm gonna collect the digit forms and write it down . u And I should say that  , you just pau you just read each line an and then pause briefly . professor e: So  , you see , Don , the unbridled excitement of the work that we have on this project . grad g: And I 'm surprised I sort of I 'm surprised I forgot that , professor e: Yeah , I I 'd I think it 's some grad g: but  I think that would be a good thing to add . professor e: Yeah , well , that 's  , so I I do have a a an agenda suggestion .  , we I think the things that we talk about in this meeting  tend to be a mixture of  procedural  mundane things and  research points and  I was thinking I think it was a meeting a couple of weeks ago that we we spent much of the time talking about the mundane stuff cuz that 's easier to get out of the way and then we sort of drifted into the research and maybe five minutes into that Andreas had to leave . So  I 'm suggesting we turn it around and and  sort of we have anybody has some mundane points that we could send an email later ,  hold them for a bit , and let 's talk about the the research - y kind of things .  , so  the one th one thing I know that we have on that is  we had talked a a couple weeks before   about the  the stuff you were doing with with    l l attempting to locate events , we had a little go around trying to figure out what you meant by " events " but I think , you know , what we had meant by " events " I guess was  points of overlap between speakers . But I th I gather from our discussion a little earlier today that you also mean  interruptions with something else phd d: Yeah . phd d: To professor e: So at any rate you were you 've you 've done some work on that phd d: right . professor e: and  then the other thing would be it might be nice to have a preliminary discussion of some of the other  research  areas that  we 're thinking about doing .  , I think especially since you you haven't been in in these meetings for a little bit , maybe you have some discussion of some of the p the plausible things to look at now that we 're starting to get data ,  and one of the things I know that also came up  is some discussions that that  that  Jane had with Lokendra  about some some some   work about I I I d I I don't want to try to say cuz I I 'll say it wrong , but anyway some some potential collaboration there about about the about the working with these data . Well , I don't know if we if this is sort of like everybody has something to contribute sort of thing , I think there 's just just a couple a couple people primarily  but   , wh why don't Actually I think that that last one I just said we could do fairly quickly so why don't you you start with that . postdoc b: so , he was interested in the question of you know , relating to his to the research he presented recently ,  of inference structures , and  , the need to build in , this this sort of  mechanism for understanding of language . And he gave the example in his talk about how  , e a I 'm remembering it just off the top of my head right now , but it 's something about how  , i " Joe slipped " you know , " John had washed the floor " or something like that . And I don't have it quite right , but that kind of thing , where you have to draw the inference that , OK , there 's this time sequence , but also the the the causal aspects of the  floor and and how it might have been the cause of the fall and that  it was the other person who fell than the one who cleaned it and it These sorts of things . So , I looked through the transcript that we have so far , and  , fou identified a couple different types of things of that type and  , one of them was something like  , during the course of the transcript , w we had gone through the part where everyone said which channel they were on and which device they were on , and  , the question was raised " Well , should we restart the recording at this point ? " And and Dan Ellis said , " Well , we 're just so far ahead of the game right now we really don't need to " . Now , how would you interpret that without a lot of inference ? So , the inferences that are involved are things like , OK , so , how do you interpret " ahead of the game " ? You know . So it 's the it 's i What you what you int what you draw you know , the conclusions that you need to draw are that space is involved in recording , grad g: metaphorically . postdoc b: that  , i that i we have enough space , and he continues , like " we 're so ahead of the game cuz now we have built - in downsampling " . So you have to sort of get the idea that  , " ahead of the game " is sp speaking with respect to space limitations , that  that in fact downsampling is gaining us enough space , and that therefore we can keep the recording we 've done so far . grad g: So , do you think his interest is in using this as a data source , or training material , or what ? professor e: Well , I I should maybe interject to say this started off with a discussion that I had with him , so  we were trying to think of ways that his interests could interact with ours grad g:   professor e: and   I thought that if we were going to project into the future when we had a lot of data ,  and  such things might be useful for that in or before we invested too much  effort into that he should  , with Jane 's help , look into some of the data that we 're already have and see , is there anything to this at all ? grad g:   professor e: Is there any point which you think that , you know , you could gain some advantage and some potential use for it . Cuz it could be that you 'd look through it and you say " well , this is just the wrong task for for him to pursue his " grad g: Wrong , yeah . professor e: And and  I got the impression from your mail that in fact there was enough things like this just in the little sample that that you looked at that that it 's plausible at least .  , he was he he you know We met and he was gonna go and  you know , y look through them more systematically professor e: Yeah . postdoc b: So it 's , you know , not a matter of a professor e: Yeah . professor e: So anyway , that 's that 's e a quite different thing from anything we 've talked about that , you know , might might might come out from some of this .  , he 's talking about just using text postdoc b: That 's his major I mentioned several that w had to do with implications drawn from intonational contours phd c: pretty much , or ? postdoc b: and that wasn't as directly relevant to what he 's doing . postdoc b: inferences that you draw i from professor e: he certainly could use text , but we were in fact looking to see if there is there is there something in common between our interest in meetings and his interest in in in this stuff . grad g: And I imagine that transcripts of speech  text that is speech probably has more of those than sort of prepared writing . postdoc b: Yeah , I don't think I would make that leap , because i in narratives , you know  , if you spell out everything in a narrative , it can be really tedious , grad g:   grad g: Yeah , I 'm just thinking , you know , when you 're when you 're face to face , you have a lot of backchannel and And postdoc b: Oh . And so I think it 's just easier to do that sort of broad inference jumping if it 's face to face .  , so , if I just read that Dan was saying " we 're ahead of the game " in that in that context , postdoc b: Well Yeah . grad g: I might not realize that he was talking about disk space as opposed to anything else . postdoc b: I you know , I I had several that had to do with backchannels and this wasn't one of them . postdoc b: This this one really does  m make you leap from So he said , you know , " we 're ahead of the game , w we have built - in downsampling " . postdoc b: And the inference , i if you had it written down , would be grad g: I guess it would be the same . But there are others that have backchannelling , it 's just he was less interested in those .  , I f f f I 've @ @ d A minute  , several minutes ago , I , like , briefly was was not listening and So who is " he " in this context ? phd c: Yeah , there 's a lot of pronoun phd f: OK . So I was just realizing we 've You guys have been talking about " he "  for at least  , I don't know , three three four minutes without ever mentioning the person 's name again . Actually to make it worse , Morgan uses " you " and " you " phd f: So this is this is this is gonna be a big , big problem if you want to later do  , you know , indexing , or speech understanding of any sort . phd f: You just wrote this ? phd c: and he never said Li - He looked grad g: Well , I think he 's doing that intentionally , phd c: Right , so it 's great . grad g: aren't you ? phd c: So this is really great phd f: Right . phd c: because the thing is , because he 's looking at the per even for addressees in the conversation , phd d: Yeah . phd c: Yeah , if you have the P Z Ms you should be able to pick up what a person is looking at from their voice . postdoc b: Oh , but you 'd have the phd c: Put Morgan always like this postdoc b: You 'd have fainter phd c: and postdoc b: Wouldn't you get fainter reception out here ? professor e: Well , these grad g: Sure , but I think if I 'm talking like this ? Right now I 'm looking at Jane and talking , now I 'm looking at Chuck and talking , I don't think the microphones would pick up that difference . grad g: So if I 'm talking at you , or I 'm talking at you . professor e: I probably been affect No , I th I think I 've been affected by too many conversations where we were talking about lawyers and talking about and concerns about " oh gee is somebody going to say something bad ? " and so on . professor e: And so I so I 'm I 'm tending to stay away from people 's names even though  postdoc b: I am too . phd c: Even though you could pick up later on , just from the acoustics who you were t who you were looking at . grad g: or or is that just too sensitive ? professor e: No no , it isn't sensitive at all . postdoc b: Well professor e: I was just I was just I was overreacting just because we 've been talking about it . professor e: It 's OK to postdoc b: I I came up with something from the Human Subjects people that I wanted to mention .  , it fits into the m area of the mundane , but they did say You know , I asked her very specifically about this clause of how , you know , it says " no individuals will be identified  , " in any publication using the data . " OK , well , individuals being identified , let 's say you have a a snippet that says , " Joe s  thinks such - and - such about about this field , but I think he 's wrongheaded . " Now  , we 're we 're gonna be careful not to have the " wrongheaded " part in there , but but you know , let 's say we say , you know , " Joe used to think so - and - so about this area , in his publication he says that but I think he 's changed his mind . Then the issue of of being able to trace Joe , because we know he 's well - known in this field , and all this and and tie it to the speaker , whose name was just mentioned a moment ago , can be sensitive . professor e: b But I postdoc b: So I think it 's really really kind of adaptive and wise to not mention names any more than we have to because if there 's a slanderous aspect to it , then how much to we wanna be able to have to remove ? professor e: Yeah , well , there 's that . But I  I think also to some extent it 's just educating the Human Subjects people , in a way , because there 's If  You know , there 's court transcripts , there 's there 's transcripts of radio shows  people say people 's names all the time . It 's just that i  you 're right that there 's more poten If we never say anybody 's name , then there 's no chance of of of slandering anybody , phd c: But , then it won't  , if we if we professor e: but grad g: It 's not a meeting . postdoc b: Well , my feeling on it was that it wasn't really important who said it , you know . phd f: Well , if you ha since you have to  go over the transcripts later anyway , you could make it one of the jobs of the people who do that to mark grad g: Well , we t we t we talked about this during the anon anonymization . grad g: If we wanna go through and extract from the audio and the written every time someone says a name . professor e: So it 's postdoc b: So , we need to talk about this later . I sorta knew I was doing it but it was phd f: Well , I still don't know who " he " is . phd c: No , you have to say , you still don't know who " he " is , with that prosody .  , I I would like to move it into into  what Jose  has been doing postdoc b: Yeah . I I remind that me my first objective eh , in the project is to to study difference parameters to to find a a good solution to detect eh , the overlapping zone in eh speech recorded . But eh , tsk , ehhh In that way I I I begin to to study and to analyze the ehn the recorded speech eh the different session to to find and to locate and to mark eh the the different overlapping zone . And eh so eh I was eh I am transcribing the the first session and I I have found eh , eh one thousand acoustic events , eh besides the overlapping zones , eh I I  the eh breaths eh aspiration eh , eh , talk eh , eh , clap , eh I don't know what is the different names eh you use to to name the the n speech phd a: Nonspeech sounds ? phd d: Yeah . grad g: Oh , I don't think we 've been doing it at that level of detail . Eh , I I I do I don't need to to to mmm to m to label the the different acoustic , but I prefer because eh I would like to to study if eh , I I will find eh , eh , a good eh parameters eh to detect overlapping I would like to to to test these parameters eh with the another eh , eh acoustic events , to nnn to eh to find what is the ehm the false eh , the false eh hypothesis eh , nnn , which eh are produced when we use the the ehm this eh parameter eh  pitch eh , eh , difference eh , feature grad g:   phd a: You know I think some of these  that are the nonspeech overlapping events may be difficult even for humans to tell that there 's two there . phd a: if it 's a tapping sound , you wouldn't necessarily or , you know , something like that , it 'd be it might be hard to know that it was two separate events . grad g: Well You weren't talking about just overlaps phd d: Ye grad g: were you ? You were just talking about acoustic events . phd d: I I I I t I t I talk eh about eh acoustic events in general , grad g: Someone starts , someone stops Yeah . professor e: How many overlaps were there  in it ? No no , how many of them were the overlaps of speech , though ? phd d: How many ? Eh almost eh three hundred eh in one session grad g: Oh , God ! phd d: in five eh in forty - five minutes . postdoc b: Does this ? So if you had an overlap involving three people , how many times was that counted ? phd d: Yeah , three people , two people . Eh ,  I would like to consider eh one people with difference noise eh in the background , be professor e: No no , but I think what she 's asking is if at some particular for some particular stretch you had three people talking , instead of two , did you call that one event ? phd d: Oh . This th I I I con I consider I consider eh an acoustic event , the overlapping zone , the period where three speaker or eh are talking together . grad g: Well So let 's postdoc b: For grad g: So let 's say me and Jane are talking at the same time , and then Liz starts talking also over all of us . phd d: Yeah ? grad g: Is there an event right here ? phd d: Eh no . For me is the overlapping zone , because because you you have s you have more one eh , more one voice eh , eh produced in a in in a moment . If professor e: So then , in the region between since there there is some continuous region , in between regions where there is only one person speaking . professor e: Is it Are you calling the beginning or the end of it the event , phd d: Yeah . professor e: or are you calling the entire length of it the event ? phd d: I consider the the , nnn the nnn , nnn eh , the entirety eh , eh , all all the time there were the voice has overlapped .  , I 'm not considering eh the the ehm eh , the fact of eh , eh , for example , what did you say ? Eh at first eh , eh two talkers are  , eh speaking , and eh , eh a third person eh join to to that . For me , it 's eh it 's eh , all overlap zone , with eh several numbers of speakers is eh , eh the same acoustic event . Wi - but  , without any mark between the zone of the overlapping zone with two speakers eh speaking together , and the zone with the three speakers . grad g: Well , but But you could imagine that three people talking has a different spectral characteristic than two . phd d: I I don't know what eh will will happen with the grad g: Yep . grad g: That 's a lot of overlap , phd d: Yeah ? professor e: So again , that 's that 's three three hundred in forty - five minutes that are that are speakers , just speakers . postdoc b: But a a a th professor e: So that 's about eight per minute . postdoc b: But a thousand events in twelve minutes , that 's phd d: Yeah , but Yeah . postdoc b: Well , but a thousand taps in eight minutes is a l in twelve minutes is a lot . phd c: Actually phd d: I I con I consider I consider acoustic events eh , the silent too . grad g: Silence starting or silence ending phd d: Yeah , silent , ground to bec to detect eh because I consider acoustic event all the things are not eh speech .  silent , I I I I don't I I haven't the eh I I would like to to do a stylistic study professor e: Yeah . phd d: and give you eh with the report eh from eh the the study from the the the session one session . When eh eh I w I I was eh look at eh nnn , the difference speech file , for example , eh if eh we use the ehm the mixed file , to to transcribe , the the events and the words , I I saw that eh the eh speech signal , collected by the eh this kind of mike eh of this kind of mike , eh are different from the eh mixed signal eh , we eh collected by headphone . The the the I I I knew that eh the signal eh , eh would be different , but eh the the problem is eh , eh we eh detected eh difference events in the speech file eh collected by by that mike  qui compared with the mixed file . And so if when you transcribe eh only eh using the nnn the mixed file , it 's possible eh if you use the transcription to evaluate a different system , it 's possible you eh in the eh i and you use the eh speech file collected by the eh fet mike , to eh to nnn to do the experiments with the the system , professor e:   phd d: its possible to evaluate eh , eh or to consider eh acoustic events that which you marked eh in the mixed file , but eh they don't appear in the eh speech signal eh collected by the by the mike . The the reason that I generated the mixed file was for IBM to do word level transcription , not speech event transcription . grad g: So I agree that if someone wants to do speech event transcription , that the mixed signals here phd d: Yeah . grad g: if I 'm tapping on the table , you it 's not gonna show up on any of the mikes , but it 's gonna show up rather loudly in the PZM . So and I I I say eh that eh , eh , or this eh only because eh I c I I in my opinion , it 's necessary to eh to eh to put the transcription on the speech file , collected by the objective signal . phd d:  the the the signal collected by the eh , the real mike in the future , in the prototype to to eh correct the initial eh segmentation eh with the eh real speech professor e:   professor e: Yeah , well , just  , just in that that one s ten second , or whatever it was , example that Adam had that that we we passed on to others a few months ago , there was that business where I g I guess it was Adam and Jane were talking at the same time and and  , in the close - talking mikes you couldn't hear the overlap , and in the distant mike you could . So yeah , it 's clear that if you wanna study if you wanna find all the places where there were overlap , it 's probably better to use a distant mike . professor e: On the other hand , there 's other phenomena that are going on at the same time for which it might be useful to look at the close - talking mikes , phd d: Yeah . phd c: But why can't you use the combination of the close - talking mikes , time aligned ? professor e: so it 's grad g: If you use the combination of the close - talking mikes , you would hear Jane interrupting me , but you wouldn't hear the paper rustling . And so if you 're interested in phd c: I  if you 're interested in speakers overlapping other speakers and not the other kinds of nonspeech , that 's not a problem , professor e: Some of it 's masking masked . grad g: Although the other issue is that the mixed close - talking mikes  , I 'm doing weird normalizations and things like that . phd d: I I I think eh I saw the nnn the eh but eh I eh I have eh any results . I I I saw the the speech file collected by eh the fet mike , and eh eh signal eh to eh to noise eh relation is eh low . phd d: And I I found that nnn that eh , ehm , pr probably , grad g: Did Did you phd d: I 'm not sure eh by the moment , but it 's it 's probably that eh a lot of eh , eh for example , in the overlapping zone , on eh in in several eh parts of the files where you you can find eh , eh eh , smooth eh eh speech eh from eh one eh eh talker in the in the meeting , professor e:   phd d: it 's probably in in that eh in in those files you you can not find you can not process because eh it 's confused with with noise . But eh my idea is to to process only nnn , this eh nnn , this kind of s of eh speech . I 'm not sure it 's a good idea , but eh professor e: No i grad g: Well , it 's more realistic but it 'll it 'll be a lot harder . professor e: Well , it 'd be hard , but on the other hand as you point out , if your if i if if your concern is to get  the overlapping people people 's speech , you will you will get that somewhat better . professor e: Are you making any use  you were you were working with th the data that had already been transcribed . professor e: Now  did you make any use of that ? See I was wondering cuz we st we have these ten hours of other stuff that is not yet transcribed . professor e: Do you phd d: The the transcription by Jane , t eh i eh , I I I want to use to to nnn , eh to put i i it 's a reference for me . But eh the transcription eh for example , I I don't I I 'm not interested in the in the in the words , transcription words , eh transcribed eh eh in eh follow in the in the in the speech file , but eh eh Jane eh for example eh put a mark eh at the beginning eh of each eh talker , in the in the meeting ,  eh she she nnn includes information about the zone where eh there are eh there is an overlapping zone . But eh there isn't any any mark , time temporal mark , to to c eh to mmm e - heh , to label the beginning and the end of the of the professor e:   Right , so she is phd d: ta I 'm I I I think eh we need this information to professor e: Right . So the twelve you you it took you twelve hours of course this included maybe some some time where you were learning about what what you wanted to do , but but  , it took you something like twelve hours to mark the forty - five minutes , your grad g: Twelve minutes . professor e: I thought you did forty - five minutes of phd d: No , forty - five minutes is the is the session , all the session . phd d: Tw - twelve hours of work to to segment eh and label eh twelve minutes from a session of part of f professor e: Oh . I I consider all the all the session because eh I I count the nnn the nnn the overlappings marked by by Jane , professor e: Oh , OK . So it 's three hundred in forty - five minutes , but you have you have time  ,  marked twelve minute the the the  overlaps in twelve minutes of it . phd f: So , can I ask can I ask whether you found  , you know , how accurate  Jane 's   labels were as far as grad g: Well , not just the overlaps , everything . phd f: you know , did she miss some overlaps ? or did she n ? phd d: But , by by the moment , I I don't compare , my my temporal mark with eh Jane , but eh I I want to do it . Because eh eh i per perhaps I have eh errors in the in the marks , I and if I I compare with eh Jane , it 's probably I I I can correct and and and to get eh eh a more accurately eh eh transcription in the file . phd c: Well , not only a word level , but actually phd d: Well phd f: I 'm expect I 'm not expecting phd d: No , it 's phd c: you didn't need to show the exact point of interruption , you just were showing at the level of the phrase or the level of the speech spurt , or grad g: Right . postdoc b: And and sometimes , you know , it was like you could have an overlap where someone said something in the middle , phd d: Yeah . postdoc b: but , yeah , w it just wasn't important for our purposes to have it that i disrupt that unit in order to have , you know , a the words in the order in which they were spoken , it would have it would have been hard with the interface that we have . postdoc b: Now , my a Adam 's working on a of course , on a revised overlapping interface , phd d:  - huh . phd d: I I I think It 's it 's a good eh work , postdoc b: but phd d: but eh I think we need eh eh more information . phd f: I expect you to find more overlaps than than Jane grad g: Always need more for postdoc b: Yeah . I I have to go to phd f: because you 're looking at it at a much more detailed level . professor e: I have grad g: But if it takes sixty to one professor e: Well , I but I have a suggestion about that .  , obviously this is very , very time - consuming , and you 're finding lots of things which I 'm sure are gonna be very interesting , but in the interests of making progress ,  might I s how how would it affect your time if you only marked speaker overlaps ? phd d: Only . professor e: but only mark speaker Do you think that would speed it up quite a bit ? phd d: OK . I I I I w I I wanted to professor e: Do y do you think that would speed it up ?  , speed up your your your marking ? phd d: nnn , I don't understand very . professor e: Now , my suggestion was for the other thirty - three phd d: On - only to mark only to mark overlapping zone , but professor e: Yeah , and my question is , if you did that , if you followed my suggestion , would it take much less time ? phd d: Oh , yeah . professor e: Then I think it 's a good idea , because it phd d: Sure , because I I need a lot of time to to put the label or to do that . professor e: There 's there 's  continual noise  from fans and so forth , and there is  more impulsive noise from  taps and so forth phd d: Yeah . We know that all that 's there and it 's a g worthwhile thing to study , but obviously it takes a lot of time to mark all of these things . professor e: Whereas th i I would think that  you we can study more or less as a distinct phenomenon the overlapping of people talking . Then you can get the Cuz you need If it 's three hundred  i i it sounds like you probably only have fifty or sixty or seventy events right now that are really phd d: Yeah . professor e: And and you need to have a lot more than that to have any kind of  even visual sense of of what 's going on , much less any kind of reasonable statistics . phd c: Now , why do you need to mark speaker overlap by hand if you can infer it from the relative energy in the grad g: Well , that 's That 's what I was gonna bring up . phd c: you shouldn't need to do this p completely by hand , professor e: OK , yeah . professor e: So the idea was that what he was going to be doing was experimenting with different measures such as the increase in energy , such as the energy in the LPC residuals , such as  there 's a bunch of things  , increased energy is - is sort of an obvious one . professor e: and  , it 's not obvious , you could you could do the dumbest thing and get get it ninety percent of the time . But when you start going past that and trying to do better , it 's not obvious what combination of features is gonna give you the you know , the right detector . And so the i the idea of the manual marking was to say " OK this , i you know , it 's it 's really here " . phd a: But I think Liz is saying why not get it out of the transcripts ? phd c: What  is get it from the close - talking mikes . phd c: A or ge get a first pass from those , professor e: We t we t w we t we talked about that . phd c: and then go through sort of It 'd be a lot faster probably to phd f: And you can grad g: Yeah , that 's his ,  professor e: We we we talked about that . s But so it 's a bootstrapping thing and the thing is , phd c: Yeah , I just professor e: the idea was , i we i i we thought it would be useful for him to look at the data anyway , and and then whatever he could mark would be helpful , phd c: Right . You know , do you bootstrap from a simple measurement which is right most of the time and then you g do better , or do you bootstrap from some human being looking at it and then then do your simple measurements ,  from the close - talking mike . phd c: Well , that 's what I wonder , because  or how bad it is , professor e: Well phd c: be  , because that would be interesting grad g: I 'm working on a program to do that , and phd c: especially because the bottleneck is the transcription . phd c: so  it seems like one kind of project that 's not perfect , but  , that you can get the training data for pretty quickly is , you know , if you infer form the close - talking mikes where the on - off points are of speech , professor e: Right , we discussed that . phd c: you know , how can we detect that from a far - field ? grad g: And postdoc b: Oh . grad g: I 've I 've written a program to do that , phd c: OK , I 'm sorry I missed the grad g: and it ,  professor e: It 's OK . And , it seems to work , I 've I 'm sort of fiddling with the parameters , to get it to actually generate something , and I haven't I don't what I 'm working on was working on was getting it to a form where we can import it into the user interface that we have , into Transcriber . I 've worked on it for about half a day , grad h: I have to go . grad g: so give me another half day and I we 'll have something we can play with . professor e: See , this is where we really need the Meeting Recorder query stuff to be working , because we 've had these meetings and we 've had this discussion about this , and I 'm sort of remembering a little bit about what we decided , phd c: Right . phd c: It professor e: So , I think it was partly that , you know , give somebody a chance to actually look at the data and see what these are like , partly that we have e some ground truth to compare against , you know , when when he he gets his thing going , grad g: But professor e: and phd c: Well , it 's definitely good to have somebody look at it . I was just thinking as a way to speed up you know , the amount of postdoc b:   postdoc b: Was that  there m there was this already a script I believe  that Dan had written , that  handle bleedthrough ,  cuz you have this this close you have contamination from other people who speak loudly . So I I haven't tried that , but that If It it might be something it might be a good way of cleaning it up a little . postdoc b: So , some thought of maybe having Yeah , having that be a preprocessor and then run it through yours . professor e: But but that 's a refinement postdoc b: That 's what we were discussing . professor e: and I think we wanna see try the simple thing first , cuz you add this complex thing up  afterwards that does something good y y yo you sort of wanna see what the simple thing does first . professor e: But  , having having somebody have some experience , again , with with  with marking it from a human standpoint , we 're  , I don't expect Jose to to do it for  f fifty hours of of speech , but  we if  if he could speed up what he was doing by just getting the speaker overlaps so that we had it , say , for forty - five minutes , then at least we 'd have three hundred examples of it . professor e: And when when  Adam was doing his automatic thing he could then compare to that and see what it was different . phd a: You know , I did I did  something almost identical to this at one of my previous jobs , and it works pretty well .  , i almost exactly what you described , an energy detector with a median filter , you look for runs . phd a: And so I think doing that to generate these possibilities and then going through and saying yes or no on them would be a quick way to to do it . grad g: Yeah , do you have a patent on it ? phd a: It was when I was working for the government . postdoc b: Well , is this something that we could just co - opt , or is it ? phd a: Nah . I think I think it 's phd a: Yeah , he 's it it doesn't take a long time . I just thought if it was tried and true , then and he 's gone through additional levels of of development . Although if you if you have some parameters like what 's a good window size for the median filter phd a: Oh ! I have to remember . phd f: They grad g: I was doing pretty short , you know , tenth of a second , sorts of numbers . professor e: I don't know , it if if we want to  So , maybe we should move on to other other things in limited time . postdoc b: Can I ask one question about his statistics ? So so in the tw twelve minutes , if we took three hundred and divided it by four , which is about the length of twelve minutes , i  , I 'd expect like there should be seventy - five overlaps . postdoc b: Did you find  more than seventy - five overlaps in that period , or ? phd d: More than ? postdoc b: More than How many overlaps in your twelve minutes ? phd d: How many ? Eh , not @ @ I Onl - only I I transcribe eh only twelve minutes from the professor e: Yeah . phd d: but eh I I don't co eh I don't count eh the the overlap . phd d: I consider I I The the nnn The the three hundred is eh considered only you your transcription . grad g: I b I bet they 're more , because the beginning of the meeting had a lot more overlaps than than sort of the middle . grad g: Because i we 're we 're dealing with the  , in the early meetings , we 're recording while we 're saying who 's talking on what microphone , and things like that , phd d: Yeah . professor e: So so I was gonna ask , I guess about any any other things that that that either of you wanted to talk about , especially since Andreas is leaving in five minutes , that that you wanna go with . phd c: Can I just ask about the data , like very straightforward question is where we are on the amount of data and the amount of transcribed data , just cuz I 'm I wanted to get a feel for that to sort of be able to know what what can be done first and like how many meetings are we recording professor e: Right so there 's this this There 's this forty - five minute piece that Jane transcribed . phd c: and professor e: That piece was then  sent to IBM so they could transcribe so we have some comparison point . Then there 's s a larger piece that 's been recorded and  put on CD - ROM and sent  to IBM . phd c: How many meetings is that ? Like how many grad g: What 's that ? professor e: That was about ten hours , and there was about phd c: t ten It 's like ten meetings or something ?  - huh . And then then we phd a: Ten meetings that have been sent to IBM ? phd c: And professor e: Yeah . grad g: Well , I haven't sent them yet because I was having this problem with the missing files . phd a: H how many total have we recorded now , altogether ? professor e: We 're saying about twelve hours . And we 're recording only this meeting , like continuously we 're only recording this one now ? or ? professor e: No . No , so the the that 's the that 's the biggest one  , chunk so far , grad g: Nope . phd c: Do they meet every week , professor e: And then there phd c: or every professor e: they do . w w And we talked to them about recording some more and we 're going to , we 've started having a morning meeting , today  i starting a w a week or two ago , on the  front - end issues , and we 're recording those ,  there 's a network services and applications group here who 's agreed to have their meetings recorded , phd c: Great . So actually , we 're gonna h start having a a pretty significant chunk and so , you know , Adam 's sort of struggling with trying to get things to be less buggy , and come up quicker when they do crash and stuff things like that , now that  the things are starting to happen . So right now , yeah , I th I 'd say the data is predominantly meeting meetings , but there are scattered other meetings in it and that that amount is gonna grow  so that the meeting meetings will probably ultimately i if we 're if we collect fifty or sixty hours , the meeting meetings it will probably be , you know , twenty or thirty percent of it , not not not eighty or ninety . phd c: So there 's probably there 's three to four a week , grad g: That 's what we 're aiming for . professor e: Yeah and th the the other thing is I 'm not pos I 'm sort of thinking as we 've been through this a few times , that I really don't know maybe you wanna do it once for the novelty , but I don't know if in general we wanna have meetings that we record from outside this group do the digits .  in fact , the morning group is really motivated cuz they 're working on connected digits , so it 's grad g: Actually that 's something I wanted to ask , is I have a bunch of scripts to help with the transcription of the digits . grad g: We don't have to hand - transcribe the digits because we 're reading them and I have those . grad g: And so I have some scripts that let you very quickly extract the sections of each utterance .  , if I did that , is someone gonna be working on it ? professor e: yeah , I I think definitely s so Absolutely . grad g: is it something of interest ? professor e: Yeah , whoever we have working on the acoustics for the Meeting Recorder are gonna start with that .  , I I 'm I 'm interested in it , I just don't have time to do it now . phd f: I was these meetings I 'm sure someone thought of this , but these this  reading of the numbers would be extremely helpful to do  adaptation . phd c: Actually I have o grad g: I I would really like someone to do adaptation . grad g: So if we got someone interested in that , I think it would be great for Meeting Recorder . professor e: Well  , one of the things I wanted to do , that I I talked to to Don about , is one of the possible things he could do or m also , we could have someone else do it , is to do block echo cancellation , grad g: Since it 's the same people over and over . professor e: to try to get rid of some of the effects of the the the far - field effects .  ,  we have the party line has been that echo cancellation is not the right way to handle the situation phd f:   professor e: because people move around , and  , if if it 's if it 's  not a simple echo , like a cross - talk kind of echo , but it 's actually room acoustics , it 's it 's it 's you can't really do inversion , phd f:   professor e: and even echo cancellation is going to  be something It may you Someone may be moving enough that you are not able to adapt quickly and so the tack that we 've taken is more " lets come up with feature approaches and multi - stream approaches and so forth , that will be robust to it for the recognizer and not try to create a clean signal " . But it occurred to me a few months ago that  party lines are always , you know , sort of dangerous . And so we haven't had anybody try to do a good serious job on echo cancellation and we should know how well that can do . So that 's something I 'd like somebody to do at some point , just take these digits , take the far - field mike signal , and the close  mike signal , and apply really good echo cancellation .  , there was a have been some nice talks recently by by Lucent on on their b phd f:  . professor e: the block echo cancellation particularly appealed to me ,  you know , trying and change it sample by sample , but you have some reasonable sized blocks . And  , you know , th phd a: W what is the  the artifact you try to you 're trying to get rid of when you do that ? phd f: Ciao . professor e:  so it 's it you have a a direct   , what 's the difference in If you were trying to construct a linear filter , that would  phd f: I 'm signing off . that would subtract off the   parts of the signal that were the aspects of the signal that were different between the close - talk and the distant . You know , so so   I guess in most echo cancellation Yeah , so you Given that  Yeah , so you 're trying to So you 'd There 's a a distance between the close and the distant mikes so there 's a time delay there , and after the time delay , there 's these various reflections . And if you figure out well what 's the there 's a a least squares algorithm that adjusts itself adjusts the weight so that you try to subtract essentially to subtract off  different  different reflections . Right ? So let 's take the simple case where you just had you had some  some delay in a satellite connection or something and then there 's a there 's an echo . phd a: So that would mean like if you were listening to the data that was recorded on one of those .  , just the raw data , you would you might hear kind of an echo ? And and then this noise cancellation would get professor e: Well , I 'm I 'm I 'm saying That 's a simplified version of what 's really happening . What 's really happening is Well , when I 'm talking to you right now , you 're getting the direct sound from my speech , but you 're also getting , the indirect sound that 's bounced around the room a number of times . OK ? So now , if you  try to r you To completely remove the effect of that is sort of impractical for a number of technical reasons , but I but not to try to completely remove it , that is , invert the the room response , but just to try to   eliminate some of the the effect of some of the echos .  , a number of people have done this so that , say , if you 're talking to a speakerphone ,  it makes it more like it would be , if you were talking right up to it . You say I I I want to use this  this item but I want to subtract off various kinds of echos . So you construct a filter , and you have this this filtered version  of the speech  gets   gets subtracted off from the original speech . phd a: Kind of a clean up thing , that professor e: It 's a clean up thing . professor e: So , echo cancelling is is , you know , commonly done in telephony , and and and it 's sort of the obvious thing to do in this situation if you if , you know , you 're gonna be talking some distance from a mike . phd a: When  , I would have meetings with the folks in Cambridge when I was at BBN over the phone , they had a  some kind of a special speaker phone and when they would first connect me , it would come on and we 'd hear all this noise . And then it was  And then it would come on and it was very clear , professor e: Yeah . So it 's taking samples , it 's doing adaptation , it 's adjusting weights , and then it 's getting the sum . So  ,  anyway that 's that 's kind of a reasonable thing that I 'd like to have somebody try somebody look And and the digits would be a reasonable thing to do that with . I think that 'd be enough data plenty of data to do that with , and i for that sort of task you wouldn't care whether it was  large vocabulary speech or anything .  postdoc b: Is Brian Kingsbury 's work related to that , or is it a different type of reverberation ? professor e: Brian 's Kingsbury 's work is an example of what we did f f from the opposite dogma . Right ? Which is what I was calling the " party line " , which is that  doing that sort of thing is not really what we want . We want something more flexible ,  i i where people might change their position , and there might be , you know There 's also  oh yeah , noise . It 's if you have a clean situation but you just have some delays , Then we 'll figure out the right the right set of weights for your taps for your filter in order to produce the effect of those those echos . But  if there 's noise , then the very signal that it 's looking at is corrupted so that it 's decision about what the right you know , right right  delays are is ,  is right delayed signal is is is  is incorrect . And so , in a noisy situation , also in a in a situation that 's very reverberant with long reverberation times and really long delays , it 's it 's sort of typically impractical . So for those kind of reasons , and also a a c a complete inversion , if you actually I mentioned that it 's kind of hard to really do the inversion of the room acoustics .  , that 's difficult because  often times the the  the system transfer function is such that when it 's inverted you get something that 's unstable , and so , if you you do your estimate of what the system is , and then you try to invert it , you get a filter that actually  , you know , rings , and and  goes to infinity . So it 's so there 's there 's there 's that sort of technical reason , and the fact that things move , and there 's air currents  there 's all sorts of all sorts of reasons why it 's not really practical . So for all those kinds of reasons ,  we we we sort of  , concluded we didn't want to in do inversion , and we 're even pretty skeptical of echo cancellation , which isn't really inversion , and  we decided to do this approach of taking  , just picking  features , which were  will give you more something that was more stable , in the presence of , or absence of , room reverberation , and that 's what Brian was trying to do . I guess you you actually already said this thing about the  about the consent forms , which was that we now don't have to So this was the human subjects folks who said this , or that that ? postdoc b: The a apparently  , we 're gonna do a revised form , of course .  but once a person has signed it once , then that 's valid for a certain number of meetings . So I think from a s practical standpoint , maybe we could have them do it once every ten meetings , or something . It won't be that many people who do it that often , but  just , you know , so long as they don't forget that they 've done it , I guess .  , back on the data thing , so there 's this sort of one hour , ten hour , a hundred hour sort of thing that that we have . We have we have an hour  that that is transcribed , we have we have twelve hours that 's recorded but not transcribed , and at the rate we 're going ,  by the end of the semester we 'll have , I don't know , forty or fifty or something , if we if this really  Well , do we have that much ? phd c: Not really . professor e: Let 's see , we have phd c: So that 's what You know , that professor e:  eight weeks ,  is phd c: So that 's not a lot of hours . professor e: Eight weeks times three hours is twenty - four , so that 's Yeah , so like thirty thirty hours ? phd a: Three Three hours .  I was starting to think of some projects where you would use well , similar to what we talked about with  energy detection on the close - talking mikes . There are a number of interesting questions that you can ask about how interactions happen in a meeting , that don't require any transcription . So what are the patterns , the energy patterns over the meeting ? And I 'm really interested in this but we don't have a whole lot of data . So I was thinking , you know , we 've got the room set up and you can always think of , also for political reasons , if ICSI collected you know , two hundred hours , that looks different than forty hours , even if we don't transcribe it ourselves , professor e: But I don't think we 're gonna stop at the end of this semester . phd c: so professor e: Right ? So , I th I think that if we are able to keep that up for a few months , we are gonna have more like a hundred hours . phd c: is there Are there any other meetings here that we can record , especially meetings that have some kind of conflict in them or some kind of deci  , that are less well I don't  , that have some more emotional aspects to them , or strong grad g: We had some good ones earlier . phd c: There 's laughter ,  I 'm talking more about strong differences of opinion meetings , maybe with manager types , or grad g: I think it 's hard to record those . phd c: To be allowed to record them ? postdoc b: It 's also likely that people will cancel out afterwards . professor e: Yeah , people will get postdoc b: But I but I wanted to raise the KPFA idea . So I I  , I I 'd mentioned to Adam , and that was another thing I was gonna talk  , mention to them before that  there 's  It it oc it occurred to me that we might be able to get some additional data by talking to  acquaintances in local broadcast media . Because , you know , we had talked before about the problem about using found data , that that  it 's just set up however they have it set up and we don't have any say about it and it 's typically one microphone , in a ,  or and and so it doesn't really give us the the the  characteristics we want . But  , it did occur to me that we could go to friends in broadcast media and say " hey you have this panel show , or this you know , this discussion show , and  can you record multi - channel ? " And  they may be willing to record it  with phd c: With lapel mikes or something ? professor e: Well , they probably already use lapel , but they might be able to have it it wouldn't be that weird for them to have another mike that was somewhat distant . professor e: It wouldn't be exactly this setup , but it would be that sort of thing , and what we were gonna get from UW , you know , assuming they they they start recording , isn't als also is not going to be this exact setup . the other thing that occurred to me after we had that discussion , in fact , is that it 's even possible , since of course , many radio shows are not live ,  that we could invite them to have like some of their record some of their shows here . postdoc b: Wow ! phd c: Well Or The thing is , they 're not as averse to wearing one of these head - mount  , they 're on the radio , grad g: Right , as we are . phd c: Th - that 's an a side of style a style that we 're not collecting here , so it 'd be great . professor e: And and the  , the other side to it was the what which is where we were coming from I 'll I 'll talk to you more about it later is that is that there 's there 's  the radio stations and television stations already have stuff worked out presumably ,  related to , you know , legal issues and and permissions and all that . So I think it 's something we should look into , you know , we 'll collect what we collect here hopefully they will collect more at UW also and  and maybe we have this other source . But yeah I think that it 's not unreasonable to aim at getting , you know , significantly in excess of a hundred hours . The thing was , I was hoping that we could @ @ in the under this controlled situation we could at least collect , you know , thirty to fifty hours . And at the rate we 're going we 'll get pretty close to that I think this semester . Yeah I was mostly trying to think , " OK , if you start a project , within say a month , you know , how much data do you have to work with . And you you wanna s you wanna sort of fr freeze your your data for awhile so  right now and we don't have the transcripts back yet from IBM right ? Do Oh , do we now ? professor e: Well , we don't even have it for this f you know , forty - five minutes , that was phd c: So  , not complaining , I was just trying to think , you know , what kinds of projects can you do now versus  six months from now professor e: Yeah . So I was thinking right now it 's sort of this exploratory stuff where you you look at the data , you use some primitive measures and get a feeling for what the scatter plots look like , grad g: Right . professor e: and and and  and meanwhile we collect , and it 's more like yeah , three months from now , or six months from now you can you can do a lot of other things . phd c: Cuz I 'm not actually sure , just logistically that I can spend you know , I don't wanna charge the time that I have on the project too early , before there 's enough data to make good use of the time . phd c:  anyway , I shouldn't say too much , but  if someone came that was great and wanted to do some real work and they have to end by the end of this school year in the spring , how much data will I have to work with , with that person . And so it 's professor e: i Yeah , so I would think , exploratory things now .  , three months from now  ,  the transcriptions I think are a bit of an unknown cuz we haven't gotten those back yet as far as the timing , but I think as far as the collection , it doesn't seem to me l like , unreasonable to say that  in January , you know , ro roughly  which is roughly three months from now , we should have at least something like , you know , twenty - five , thirty hours . phd c: And we just don't know about the transcription part of that , professor e: So that 's postdoc b: Yeah , we need to I think that there 's a possibility that the transcript will need to be adjusted afterwards , phd c: so .  , it postdoc b: and  es especially since these people won't be  used to dealing with multi - channel  transcriptions . postdoc b: So I think that we 'll need to adjust some And also if we wanna add things like  , well , more refined coding of overlaps , then definitely I think we should count on having an extra pass through . There 'd be no reason why a person couldn't get together several  , you know , friends , and come and argue about a topic if they wanted to , right ? professor e: If they really have something they wanna talk about as opposed to something @ @  , what we 're trying to stay away from was artificial constructions , but I think if it 's a real Why not ? Yeah . postdoc b: You could do this , phd c: Well yeah , postdoc b: you know . phd c: or just if you 're if you ha If there are meetings here that happen that we can record even if we don't  have them do the digits , or maybe have them do a shorter digit thing like if it was , you know , one string of digits , or something , they 'd probably be willing to do . grad g: We don't have to do the digits at all if we don't want to . phd c: Then , having the data is very valuable , cuz I think it 's  politically better for us to say we have this many hours of audio data , especially with the ITR , if we put in a proposal on it .  , whether it 's transcribed or not  , is another issue , but there 's there are research questions you can answer without the transcriptions , or at least that you can start to answer . phd a: Would it help at all  , we 're already talking about sort of two levels of detail in meetings . One is   without doing the digits Or , I guess the full - blown one is where you do the digits , and everything , and then talk about doing it without digits , what if we had another level , just to collect data , which is without the headsets and we just did the table - mounted stuff . I 'm really scared grad g: It seems like it 's a big part of this corpus is to have the close - talking mikes . phd c:  or at least , like , me personally ? I would I couldn't use that data . phd c: So it 's a great idea , professor e: Yeah , I I b By the by the way , I don't think the transcriptions are actually , in the long run , such a big bottleneck . phd c: and if it were true than I would just do that , but it 's not that bad like the room is not the bottleneck , and we have enough time in the room , it 's getting the people to come in and put on the and get the setup going . Right ? And and  d Do you have any idea when when  the you 'll be able to send  the ten hours to them ? grad g: Well , I 've been burning two C Ds a day , which is about all I can do with the time I have . So early next week we send it to them , and then then we check with them to see if they 've got it and we we start , you know asking about the timing for it . professor e: So I think once they get it sorted out about how they 're gonna do it , which I think they 're pretty well along on , cuz they were able to read the files and so on . professor e: Right ? grad g: Yeah , but professor e: Well grad g: Yeah , who knows where they are . professor e: Yeah , but You know , so they they they have you know , they 're volunteering their time and they have a lot of other things to do , phd c: What if grad g: Yeah , you we can't complain . professor e: right ? But they But at any rate , they 'll I I think once they get that sorted out , they 're they 're making cassettes there , then they 're handing it to someone who they who 's who is doing it , and  I think it 's not going to be I don't think it 's going to be that much more of a deal for them to do thirty hours then to do one hour , I think . phd c: Really ? So it 's the amount of professor e: It 's it 's just getting it going . phd c: I don't know , if there 's any way without too much more overhead , even if we don't ship it right away to IBM even if we just collect it here for awhile , to record you know , two or three more meeting a week , just to have the data , even if they 're  not doing the digits , but they do wear the headphones ? professor e: But the lunch meetings are pretty much one person getting up and phd c: No , I meant , sorry , the meetings where people eat their lunch downstairs , maybe they don't wanna be recorded , but grad g: Oh , and we 're just chatting ? phd c: Just the ch the chatting . phd c: I actually I actually think that 's useful data ,  the chatting , grad g: Yeah , the problem with that is I would I think I would feel a little constrained to You know ?  , some of the meetings phd c: but OK . phd c: Alright , so I 'll just throw it out there , if anyone knows of one more m or two more wee meetings per week that happen at ICSI ,  that we could record , I think it would be worth it . Well , we should also check with Mari again , because they because they were really intending , you know , maybe just didn't happen , but they were really intending to be duplicating this in some level . And there 's a lot of different meetings at UW   really m a lot more than we have here right cuz we 're not right on campus , grad g: Right . phd a: Is the  , notion of recording any of Chuck 's meetings dead in the water , or is that still a possibility ? professor e: they seem to have some problems with it .  , but , again , Jerry is Jerry 's open So  , we have two speech meetings , one  network meeting ,  Jerry was open to it but I I s One of the things that I think is a little a little bit of a limitation , there is a think when the people are not involved  in our work , we probably can't do it every week . You know ? I I I I think that that people are gonna feel  are gonna feel a little bit constrained . Now , it might get a little better if we don't have them do the digits all the time . And the then so then they can just really sort of try to put the mikes on and then just charge in and grad g: Yep . phd c: What if we give people you know , we cater a lunch in exchange for them having their meeting here or something ? postdoc b: Well , you know , I I do think eating while you 're doing a meeting is going to be increasing the noise . postdoc b: But I had another question , which is  , you know , in principle , w  , I know that you don't want artificial topics , phd c: Alright , alright , alright . postdoc b: but  it does seem to me that we might be able to get subjects from campus to come down and do something that wouldn't be too artificial . postdoc b: and i you know , people who are Because , you know , there 's also this constraint . We d it 's like , you know , the the  goldibears goldi goldilocks , it 's like you don't want meetings that are too large , but you don't want meetings that are too small . And  a and it just seems like maybe we could exploit the subj human subject p p pool , in the positive sense of the word . phd a: Well , even  , coming down from campus is sort of a big thing , but what about postdoc b: We could pay subjects . phd a: or what about people in the in the building ? phd c: Yeah , I was thinking , there 's all these other peo phd a: there 's the State of California downstairs , and phd c: Yeah .  grad g: I just really doubt that  any of the State of California meetings would be recordable and then releasable to the general public . grad g: So I  I talked with some people at the Haas Business School who are i who are interested in speech recognition phd c: Alright , well . grad g: and , they sort of hummed and hawed and said " well maybe we could have meetings down here " , but then I got email from them that said " no , we decided we 're not really interested and we don't wanna come down and hold meetings . phd a: What about Joachim , maybe he can professor e: But but we c But I think , you know , we get some scattered things from this and that . professor e:  i I have better contacts in radio than in television , but phd a: You could get a lot of lively discussions from those radio ones . phd c: Well , and they 're already they 're these things are already recorded , grad g: Yep . phd c: we don't have to ask them to even and I 'm not sure wh how they record it , but they must record from individual professor e: n Well No , I 'm not talking about ones that are already recorded . I 'm talking about new ones phd c: Why why not ? professor e: because because because we would be asking them to do something different . I know for instance Mark Liberman was interested  in in LDC getting data , and professor e: Right , that 's the found data idea . professor e: But what I 'm saying is  if I talk to people that I know who do these th who produce these things we could ask them if they could record an extra channel , let 's say , of a distant mike . So , since I 'm interested in the distant mike stuff , I wanna make sure that there is at least that somewhere phd c: Right .   professor e: and  But if we ask them to do that they might be intrigued enough by the idea that they  might be e e willing to the I might be able to talk them into it . We 're getting towards the end of our disk space , so we should think about trying to wrap up here . Well I don't why don't we why d u why don't we   turn them turn grad g: OK , leave leave them on for a moment until I turn this off , cuz that 's when it crashed last time <doc-sep>You know how to toggle the display width function grad a: Well maybe she hadn't just started transcribing me yet .  , so we haven't had a meeting for a while , and and probably won't have one next week , I think a number of people are gone .  , so Robert , why don't you bring us up to date on where we are with EDU ? grad b:  in a in a smaller group we had  , talked and decided about continuation of the data collection . So Fey 's time with us is almost officially over , and she brought us some thirty subjects and , t collected the data , and ten dialogues have been transcribed and can be looked at . She 's actually suggested to look  , at the psychology department students , because they have to partake in two experiments in order to fulfill some requirements . And  , we want to design it so that they really have to think about having some time , two days , for example , to plan certain things and figure out which can be done at what time , and , sort of package the whole thing in a in a re in a few more complicated  , structure . As for SmartKom , I 'm the last SmartKom meeting I mentioned that we have some problems with the synthesis , which as of this morning should be resolved . grad b: " should be " means they aren't yet , but but I think I have the info now that I need . Plus , Johno and I are meeting tomorrow , so maybe   , when tomorrow is over , we 're done . And ha n hav we 'll never have to look at it again Maybe it 'll take some more time , to be realistic , but at least we 're we 're seeing the end of the tunnel there .  , the  ,  I don't think we need to discuss the formalism that 'll be done officially s once we 're done .  , something happened , in on Eva 's side with the PRM that we 're gonna look at today , and  , we have a visitor from Bruchsal from the International University . grad a: So when you said " Andreas " I thought you were talking about Stolcke . grad b: And , grad a: Now I know that we aren't , OK . grad b: Eh grad c: Cuz there is another Andreas around , grad a:  . grad b: So my scientific director of the EML is also the dean of the International University , one of his many occupations that just contributes to the fact that he is very occupied . And , the  , he @ @ might tell us a little bit about what he 's actually doing , and why it is s somewhat related , and by  using maybe some of the same technologies that we are using .  , so , I 've be just been looking at , Ack ! What are you doing ? Yeah .  , so , this is , sort of like the latest thing I have on it , and I sorta constructed a couple of classes . Like , a user class , a site class , and and you know , a time , a route , and then and a query class . And I tried to simplify it down a little bit , so that I can actually  , look at it more .  , so basically I took out a lot of stuff , a lot of the decision nodes , and then tried to The red lines on the , graph are the  , relations between the different  , classes . Like , a user has like , a query , and then , also has , you know  , reference slots to its preferences , the special needs and , you know , money , and the user interest . And so this is more or less similar to the flat Bayes - net that I have , you know , with the input nodes and all that . And So I tried to construct the dependency models , and a lot of these stuff I got from the flat Bayes - net , and what they depend on , and it turns out , you know , the CPT 's are really big , if I do that , so I tried to see how I can do ,  put in the computational nodes in between . And so I ended up making several classes Actually , you know , a class of with different attributes that are the intermediate nodes , and one of them is like , time affordability money affordability , site availability , and the travel compatibility . And so some of these classes are s some of these attributes only depend on stuff from , say , the user , or s f just from , I don't know , like the site . S like , these here , it 's only like , user , but , if you look at travel compatibility for each of these factors , you need to look at a pair of , you know , what the  , preference of the user is versus , you know , what type of an event it is , or you know , which form of transportation the user has and whether , you know , the onsite parking matters to the user , in that case . And that makes the scenario a little different in a PRM , because , then you have one - user objects and potentially you can have many different sites in in mind . And , they all depend on the same users , but different sites , and that makes a I 'm tr I w I wa have been trying to see whether the PRM would make it more efficient if we do inferencing like that . And so , I guess you end up having fewer number of nodes than in a flat Bayes - net , cuz otherwise you would c well , it 's probably the same . But  , No , you would definitely have be able to re - use , like , all the user stuff , and not not having to recompute a lot of the stuff , because it 's all from the user side . But , you know , in the case where , it depends on both the user and the site , then I 'm still having a hard time trying to see how  , using the PRM will help .  , so anyhow , using those intermediate nodes then , this this would be the class that represent the intermediate nodes . And that would basically it 's just another class in the model , with , you know , references to the user and the site and the time . And then , after you group them together this no the dependencies would of the queries would be reduced to this .  , so I think that 's about as far as I 've gone on the PRM stuff . And one is basically how desirable a site is meaning , how good it matches the needs of a user .  , so , instead of  , doing a lot of , you know , computation about , you know , which one site it wants of the user wants to visit , I 'll come well , try to come up with like , sort of a list of sites . And for each site , you know , where h how how well it fits , and basically a rating of how well it fits and what to do with it . She 's ac   Eva 's got a little write - up on it that  , probably gives the the details to anybody who needs them .  , so the You you didn't look at all yet to see if there 's anybody has a implementation . So one so one of the questions , you know , about these P R Ms is grad d:   professor f: we aren't gonna build our own interpreter , so if if we can't find one , then we  , go off and do something else and wait until s one appears .  , so one of the things that Eva 's gonna do over the next few weeks is see if we can track that down . If there is an interpreter , it looks like you know , what Eva 's got should run and we should be able to actually  , try to solve , you know , the problems , to actually take the data , and do it .  , I actually think it is cleaner , and the ability to instantiate , you know , instance of people and sites and stuff , will help in the expression . I think there are things that you can express this way which you can't express in a normal belief - net , without going to some incredible hacking of sort of rebuilding it on the fly .  , the notion of instantiating your el elements from the ontology and stuff fits this very nicely and doesn't fit very well into the extended belief - net . So , people who have thought about the problem , like Robert i it looked to me like if Eva were able to come up with a you know , value for each of a number of  , sites plus its EVA thing , that a travel planner should be able to take it from there . And you know , with some other information about how much time the person has and whatever , and then plan a route . grad b:  -  , well , first of all  , great looks , mu much cleaner , nnn , nnn , Certain certain beauty in it , so , if beauty is truth , then ,  we 're in good shape . So if you have a write - up then  , I 'd love to read it grad d:   grad b: and  because , i Can you go all the way back to the the very top ? grad d: Yeah . grad b:  these @ @ these w w when these are instantiated they take on the same values ? that we had before ? grad d: I can't really see the whole thing . grad b: or are they have they changed , in a sense ? grad d: Well I think I basically leave them to similar things . grad d: Some of the things might that might be different , maybe like are that the hours for the site . grad d: And , eventually I meant that to mean whether they 're open at this hour or not . grad d: And status would be , you know , more or less like , whether they 're under construction , and and or stuff like that . grad b: And the , other question I would have is that presumably , from the way the Stanford people talk about it , you can put the probabilities also on the relations . I don't remember whether they carried that over to this or not , grad a: Mmm . grad b: It 's sort of in the definition or in the in Daphne 's definition of a PRM is that classes and relations , professor f: OK . grad b: and you 're gonna have CPT 's over the classes and their relations . grad d: I remember them learning when , you know , you don't know the structure for sure , professor f: Yeah . grad d: but I don't remember reading how you specify grad b: Yeah , that would be exactly my question . So , the the plan is is when Daphne gets back , we 'll get in touch and supposedly , we 'll actually get s deep seriously connected to to their work and grad b: Yep . professor f: somebody 'll  , you know If it 's a group meeting once a week probably someone 'll go down and , whatever . We get rocking and rolling on this again , once we get a package , if , when , and how , then this becomes foregrounded grad d:   grad b: And  , until then we 'll come up with a something that 's @ @ that 's way more complicated for you . grad b: Because this was laughingly easy , right ? grad d: Actually I had to take out a lot of the complicated stuff , cuz I I made it really complicated in the beginning , and Jerry was like , " this is just too much " . So , you could , from this , go on and say suppose there 's a group of people traveling together and you wanted to plan something that somehow , with some Pareto optimal  , thing for grad a: That 's good . professor f: or grad a: Except for humans can't really solve it either , so . Well that 's the that would that would be a  , you could sell it , as a grad a: Yeah . professor f: OK , eh you don't have to fight about this , just give your preferences to the grad a: And then you can blame the computer . But what does it  Would a pote potential result be to to split up and never talk to each other again ? You know . So there i there are some some u  , you know , elaborations of this that you could try to put in to this structure , but I don't think it 's worth it now . But  , it 's good , yeah and and there were a couple other ideas of of  , things for Eva to look at in in the interim . So , I 'm not really involved in any project , that 's  that 's relevant to you  , a at the moment , the the reason is really for me  , to have an opportunity to talk to some other researchers in the field . And and so I 'll just n sort of give you a real quick introduction to what I 'm working on , and  , I just hope that you have some comments or , maybe you 're interested in it to find out more , and and so I 'll be  , happy to talk to you and and  , I 'd also like to find out some more and and maybe I 'll just walk around the office and and then and ask some some questions , in a couple days . OK , so , what I started looking at , to begin with is just  , content management systems  , i i in general . So  ,  what 's  Sort of the state of the art there is to   you have a bunch of of  documents or learning units or learning objects , and you store meta - data  , associate to them . So there 's some international standards like the I - triple - E ,  There 's an I - triple - E , LON standard , and  , these fields are pretty straightforward , you have  author information , you have  , size information , format information and so on . One is  you store keywords associated with the  with the document , and one is  , you have sort of a , well , what is the document about ? So it 's some sort of taxonomic  , ordering of of the of the units . Now , if you sort of put on your semantic glasses ,  you say , well that 's not all that easy , because there 's an implicit  , assumption behind that is that  , all the users of this system share the same interpretation of the keyword and the same interpretation of  , whichever taxonomy is used , and  , I think that 's a that 's a very that 's a key point of these systems and they sort of always brush over this real quickly without really elaborating much of that and  As a matter of fact , the only thing that m apparently really works out so far are library ordering codes , which are very , very coarse grain , so you have some like , science , biology , and then But that 's really all that we have at the moment . Now , what this  a standard like this would give us is we could  , sort of  with a search engine just query  , different repositories all over the world . But we can't really  , so what I 'm what I try to do is  , to have  ,  So . So the scenario is the following , you you 're working on some sort of project and you encounter a certain problem . Now , what what we have at our university quite a bit is that  , students  , try to u program a certain assignment , for example , they always run into the same problems , and they always come running to us , and they 'll say why 's it not it 's not working , and we always give out the same answer , so we thought , well , it 'd be nice to have a system that could sort of take care of this , and so , what I want to build is basically a a smart F A Q system . Now , what you  need to do here is you need to provide some context information which is more elaborate than " I 'm looking for this and this and this keyword . I 'm I 'm sure you have the same when when somebody utters a sentence in a certain , context it , and and the same sentence in another context makes a huge difference . So , I want to be able to model information like , so in the in the context of in the context of developing distributed systems , of a at a computer science school , what kind of software is the person using , which homework assignment is he or she working on at the moment , maybe what 's the background of that student 's  , which  , which error message was encountered . Now , basically giving this   so we somehow need to have a formalized  , way of writing this down basically , and that 's where the shared interpretation of of certain terms and keywords comes in again . And , using this and some some  , knowledge about the domain I think you can do some some simple inferences . Like you know that when somebody 's working about  , working on on servlets for example , he 's using Java , cuz servlets are used are written in Java . So some some inferences like that , now , u using this you can infer more information , and you could then match this to the meta - data of  off the documents you 're you 're searching against . So ,  what I wanna do is basically have some sort of  given these inputs , and then I can compute how many documents match , and use this as a metric in the search . Now , what I plan to do is I want to  sort of do a   try to improve the quality of the search results , and I want to do this by having a depth  ,  steepest descent approach . So if I knew which operating system the person was working on , would this improve my search result ? And and having  ,  a symbolic formalized model of this I could simply compute that , and find out which  which questions are worth  , asking . And that 's what I then propagate back to the user , and and sort of try to optimize the search in this way . Now , the big problem that I 'm facing right now is  , it 's fairly easy to hack up a system  quickly , that that works in the small domain , but the problem is obviously the scalability . And   , so Robert was mentioning  , earlier today is that  , Microsoft for example with their printer set up program has a Bayesian network , which does exactly this , but there you face a problem that these are very hard to extend . And so ,  what I 'm What I try to do is basically try to model this  , in a way that you could really combine  , knowledge from very different sources , and and  , sort of looking into some of the ideas that the semantic web community  , came up with . Trying to to have  , an approach how to integrate s  certain  representation of certain concepts and also some computational rules , what you can do with those . What I 'm also looking into is a probabilistic approach into this because document retrievals is a very fuzzy procedure , so it 's probably not that easy to simply have a symbolic  , computational model . And then  ,  sort of as an add - on to this whole idea ,  that would be now , depending on what the search engine or the content repository depending on which  , which  , rules and which ontologies it it uses , or basically its view of the world ,  you can get very different results . And there you could have an idea where you actually have sort of a a peer to peer approach , where we 're all sort of carrying around our individual bookshelves , and  , if you have a question about a homework , it 's probably makes sense to ask somebody who 's in your class with you , sort of the guru in the certain area , rather than going to some Yahoo - like  , search engine . And I think a lot of the even though it 's a it 's a very different domain , but I think a lot of the , issues are are fairly similar . grad a: And so some of the I don't know how much you know about the larger Heidelberg project , I Are you grad c:  I know , yeah I know abou about it . It 's like , you know , the c context - based factors that influence how you interpret , grad c:   We - we 've kind of talked about that , but we haven't worried too much about that end of the discourse . grad b: Well , in a in one t one s mmm , small difference in a in a way , is that he doesn't have to come up with an answer , but he wants to point to the places w w grad a: Documents that have the answers . So I 'm I 'm not I 'm not building an expert grad a: you have to s still m understand what the content says about itself , and then match it to what you think the informational needs grad a:   You 're just taking the keywords as a topic text , as grad c: I I assume that that the there will be learning systems that that tag their their content . grad c: And  , m @ @ and basically what I what I envision is that you rather than just supplying a bunch of keywords you could basically for for an FAQ for example you could state sort of like a logic condition , when this document applies . So " this document explains how to set up your  , mail account on Linux " or something like this . So something something very specific that you can then But the I think that the key point with these  , learning systems is that  , a learning system is only as good as  the amount of content it it carries . grad c: You can have the best learning system with the best search interface , if there 's no content inside of it , it 's not very useful . So I think ultimately because  ,  developing these these rules and these inference  inferences I think is very costly , so  ,  I think you must be able to reuse some some existing  , domain domain information , or or or ontologies that that  other people wrote and then try to integrate them , and then also search the entire web basically , rather than just the small  , content management system . grad c: So I think that 's that 's crucial for for the success of or @ @ grad a: So , you 're not I guess I 'm trying to figure out how how it maps to the kinds of things that we 've talked about in this group , and , actually associated groups , grad c:   grad a: cuz some of us do pretty detailed linguistic analyses , and I 'm guessing that you you won't be doing that ? OK . grad a: So , you take the query , and and professor f: On the other hand , FrameNet could well be useful . professor f: Because , you know , the standard story is that keyworks keywords evoke frames , and the frames may well give you additional keywords or  , if you know that that that a a bunch of keywords  , indicate a frame , then you can find documents that actually have the whole frame , rather th than just  , individual grad c: Mmm . There 's linguists and stuff and there 's a lot of it and they 're they 're busily working away . And this looks t it seems to be that this is a place where you might be able to do that . I 'm sure I could learn a lot about  , yeah , just how to how to come up with these structures , grad a: Mmm . grad c: cuz it 's it 's very easy to whip up something quickly , but it maybe then makes sense to to me , but not to anybody else , and and if we want to share and integrate things , they must well , they must be well designed really . grad b: And Andreas and I tried to come up wi or we had come up actually with a eh with him working on an interface for FrameNet , as it was back then , that would p do some of the work for this machine , professor f: Right . grad b: which  , never got done because Prashant found a happy occupation professor f: W yeah , I know ,  it it he w he did w what what he did was much more s sensible for him . professor f: I think  , grad b: But so I 'm just saying , the  , we had that idea professor f: you know Yeah . professor f: Yeah , actually you guys never grad b: And Srini 's doing information extraction also , right ? professor f: Right . grad c: professor f: this was supposedly an exchange program , and I we you know , it 's fine . We don't care , but it just I 'm a little surprised that  , Andreas didn't come up with anyone else he wanted to send .  I had forgotten a I To be honest with you , I 'd totally forgotten we had a program . grad b:  it 's in the program ? grad c:  I I think it 's it 's really the lack of students  , at IU at the moment . I think it 's ju it 's more the lack of of students , really , and w we have all these sponsors that are always sort of eager to get some teams . grad c: Well  if if I were a student , I 'd love to come here , rather than work for some German company , or professor f: Yeah . professor f: Oh , right ! grad c: Well , I didn't say anybody to anything to offend well , except for the sponsors maybe , but professor f: Right . So I thi tha that 's that 's one of the things that might be worth looking into while you 're here . professor f: unfortunately , Srini , who is heavily involved in DAML and all this sort of stuff is himself out of town . professor f: I don't know why he @ @ oh , I , who knows ? grad a: The  professor f: Anyway , s yeah , you 'll see you 'll certainly see a lot of the people there . grad a: The other person I thought of is Dan Gildea ? because he did some work on topic spotting professor f: Yeah . I don't Depending on how well you wanna integrate with that end , grad c:   grad a: you know , like , taking the data and fig you said the learning systems that figure out We There 's someone in ICSI who actually has been working on has worked on that kinda stuff , and he 's worked with frame net , so you could talk to him about , you know , both of those things at once . grad c: So , who is that again ? grad a: And , he 's in one of the rooms on the fifth floor and stuff , grad b: Who ? I can take you to his office . professor f: Alright , was there anything else for this ? One of these times soon we 're gonna hear about construal . professor f: Good thinking ! grad b:  well , maybe I can I can bribe my way out of this . professor f: Well , no , but he 's he 's he 's he 's as you said , he 's , like the state legislature , he 's trying to offer us bribes . grad b: This  Oh , they refused the budget again ? Is it so about CITRIS ? Yeah , still nothing . professor f: this this this t the s we 're , involved in a literally three hundred million dollar  , program . And , the State of California is now a month and a half behind its legis its legally required date to approve a budget . professor f: and , yesterdayday there was this   I thought that the other branch would just approve it , but now there 's actually a little back sliding to people who who approved it got flak from there , eh anyway . So ,  Oh ! I have to tell you a wonderful story about this , OK ? And then we 'll go . In fact he 's hired to run a lot of CITRIS , even though we don't have the money they So they 've been hiring people right and left , so , they think the money 's coming . And , is now doing all the politics for CITRIS , but also , has a  , a lot of interest in  , actually doing things for society , so digital divide and stuff like that . But the really interesting thing was , he st he s he s said something about , you know I 'm interested in things that have high social multiplier , something that is of great social value . He said , " for example " , this was his only example , " if you had a adult literacy program that was as good as an individual tutor , and as compelling as a video game , then that would have a huge social impact " . So it was nice that  , he 's got this view , of A , that 's what you should try to do , and B , language would be a good way to do it . I didn't push him on the ch on the child thing , grad a:  - huh . professor f: but , you know , a again , if if you if you grad a: Oh . So this is reading , rather than teaching Another project we started on , and and didn't get funded for was , to try to build an automatic tutoring program , for kids whose first language wasn't English . So , enormous problem in California , and the idea was if we 're so smart about language understanding and speech understanding , couldn't we build  , programs that would be tutors for the kids . professor f: and  , I know none of us have the spare time to look at it right now , but it i it 's it 's interesting and I may  , talk to him some more about is em somebody already doing this , and stuff like that . So I I did manage to get pull my head out of the sling by sidetracking into CITRIS , professor f: No , no . grad b: but , I I 'll volunteer to put it right back in by stating that I am n  among some other things in the process of writing up stuff that we have been discussing at our daily meetings , professor f: Yeah . grad b: and also revising , thanks for all the comments , the c the original construal proposal . And , if I put one and one together , I may end up with a number that 's greater than one and that I I can potentially present once you get back . Yeah , so OK , so that 'd be great , but I 'd I think it 's it 's time again , right ? grad b: Absolutely . grad b: But  , and hopefully all sidetracking  , other things will have disappeared , soon 
In the second phase of data collection, the group plans to use more complex scenarios to generate intricate dialogues. To ensure an adequate number of participants, subjects will be recruited from within the Psychology department, as their participation in experiments is mandatory. The group's goal is to collect over 100 hours of Meeting Recorder data, and they are currently revising the speaker consent forms. It has been proposed that subjects should sign a new consent form after every 10 recording sessions.
What are the future prospects for data collection?