But  we 're gonna try to make this an abbreviated meeting cuz the the next next occupants were pushing for it , so . Agenda is according to this , is transcription status , DARPA demos XML tools , disks , backups , et cetera and grad h: Does anyone have anything to add to the agenda ? professor b: OK . grad h:  phd d: I g grad h: as a somewhat segue into the next topic ,  could I get a hold of  the data even if it 's not really corrected yet just so I can get the data formats and make sure the information retrieval stuff is working ? postdoc a: Certainly . So I 'll I 'll probably just make some copies of those rather than use the ones that are there . grad h:  and then just we 'll have to remember to delete them once the corrections are made . professor b: OK , wh phd d: I also got anot a short remark to the transcription . I 've  just processed the first five EDU meetings and they are chunked up so they would they probably can be sent to IBM whenever they want them . phd d: but the other ones phd f: That 's the one that we 're waiting to hear from them on . postdoc a: These are separate from the ones that phd f: As soon as postdoc a: these are phd f: They 're the IBM set . And so as soon as we hear from Brian that this one is OK grad h: Is my mike on ? Yeah . phd f: and we get the transcript back and we find out that hopefully there are no problems matching up the transcript with what we gave them , then  we 'll be ready to go and we 'll just send them the next four as a big batch , postdoc a: Excellent . grad h: And so we 're doing those as disjoint from the ones we 're transcribing here ? phd f: Yes , exactly . phd f: We 're sort of doing things in parallel , that way we can get as much done a at once . grad h: Yeah , I think that 's the right way to do it , phd f: Yeah . grad h: Right , which  So I 've been working on using the THISL tools to do information retrieval on meeting data and the THISL tools are there 're two sets , there 's a back - end and a front - end , so the front - end is the user interface and the back - end is the indexing tool and the querying tool . So at least on the one meeting that I had the transcript for  conveniently you can now do information retrieval on it , do type in a a string and get back a list of start - end times for the meeting , phd f: What what kind of  what does that look like ? The string that you type in . phd f: What are you are you are they keywords , or are they ? grad h: Keywords . grad h: Right ? And so and then it munges it to pass it to the THISL IR which uses an SGML - like format for everything . professor b: And then does it play something back or that 's something you 're having to program ? grad h: right now , I have a tool that will do that on a command line using our standard tools , professor b: Yeah . grad h: but my intention is to do a prettier user interface based either So so that 's the other thing I wanted to discuss , is well what should we do for the user interface ? We have two tools that have already been written . grad h:  the other option is Dan did the Tcl - TK THISL GUI front - end for Broadcast News professor b: Yeah . postdoc a: I Can I ask a question ? So  as it stands within the the Channeltrans interface , it 's possible to do a find and a play . So e Are you So you 're adding like  , I don't know ,  are they fuzzy matches or are they  ? grad h: It 's a sort of standard , text - retrieval - based So it 's  term frequency , inverse document frequency scoring . grad h:  and then there are all sorts of metrics for spacing how far apart they have to be and things like that . So it it 's postdoc a: It 's a lot more sophisticated than the  the basically Windows - based grad h: i it 's like doing a Google query or anyth anything else like that . grad h: So i it uses So it pr produces an index ahead of time so you don't you 're not doing a linear search through all the documents . Cuz you can imagine if with if we have the sixty hours ' worth you do wouldn't wanna do a search . grad h:  you have to do preindexing and so that these tools do all that . And so the work to get the front - end to work would be porting it well  to get it to work on the UNIX systems , our side is just rewriting them and modifying them to work for meetings . grad h: So that it understands that they 're different speakers and that it 's one big audio file instead of a bunch of little ones and just sorta things like that . phd f: So what does the user see as the result of the query ? grad h: On which tool ? phd f: THISL . grad h: The THISL GUI tool which is the one that Dan wrote , Tcl - TK phd f: Yeah . grad h:  you type in a query and then you get back a list of hits and you can type on them and listen to them . professor b: Mmm phd f: So if you typed in " small heads " or something you could grad h: Right , you 'd get phd f: get back a   something that would let you click and listen to some audio where that phrase had occurred grad h: something You you 'd get to listen to " beep " . It 's too bad that that couldn't come into the grad h: You couldn't get a video . phd g: Guess who I practice on ? postdoc a: At some point we 're gonna have to say what that private joke is , that keeps coming up . Yeah , it loo it my my recollection of it is it 's it 's a pretty reasonable  demo sort of format . grad h: And so I think there 'd be minimal effort to get it to work , minimally phd f: That sounds really neat . grad h: and then we 'd wanna add things like query by speaker and by meeting and all that sort of stuff .  Dave Gelbart expressed some interest in working on that so I 'll work with him on it . And it it 's looking pretty good , you know , the fact that I got the query system working . grad h: If we wanna get it to Windows it 's gonna be a little more work because the THISL IR , the information retrieval tool 's  , I had difficulty just compiling them on Solaris . phd f: But you were saying that that the  that there 's that set of tools , Cygnus tools , that grad h: So . grad h: But what those they what those do is provide sort of a BSD compatibility layer , professor b:   phd f: And you have to have all the o grad h: But the problem is that that the THISL tools didn't use anything like Autoconf and so you have the normal porting problems of different header files and th some things are defined and some things aren't and  different compiler work - arounds and so on . So the fact that  it took me a day to get it c to compile under Solaris means it 's probably gonna take me s significantly more than that to get it to compile under Windows . professor b: How about having it run under free BSD ? phd e: Well what you need grad h: Free BSD would probably be easier . phd e: All you need to do is say to Dan " gee it would be nice if this worked under Autoconf " and it 'll be done in a day . phd d:  phd e: Right ? grad h: Actually you know I should check because he did port it to SPRACHcore phd e: Right . professor b: So grad h: I 'll check at that professor b: But it would what would serve would serve both purposes , is if you contact him and ask him if he 's already done it . phd e: What I phd f: How does it play ? grad h: Yeah , right . professor b: If he has then you learn , if he hasn't then he 'll do it . So , and I 've been corresponding with Dan and also with   , SoftSound guy ,  postdoc a: It 's amazing . professor b: Tony Robinson ? phd f: Tony Robinson ? grad h: Do  Tony ? I guess I do . grad h: Which one do  ? phd e: Steve Renals is not SoftSound , is he ? professor b: No . grad h: so that 's all going pretty well , professor b: Assuming we 're phd e: Right . phd f: What about issues of playing sound files @ @ between the two platforms ? grad h: I think we 'll be OK with that . phd e: Why don't you try and merge Transcriber and THISL IR ? They 're both Tcl interfaces . grad h: Well this is one of the reasons This is the one of the reasons that I 'm gonna have  Dave Gelbart Gelbart Having him volunteer to work on it is a really good thing because he 's worked on the Transcriber stuff phd e: Right . But the point is that the Transcriber uses Snack and then you can but you can use a a lot of the same functionality and it 's grad h: Yeah , yeah , I I think THISL THISL GUI probably uses Snack . Well my thought was is that it would be nice it would be nice to have the running transcripts  eh you know , from speaker to speaker . grad h: And if it doesn't phd e: Right ? Do you have you have , you know , a speaker mark here and a speaker mark here ? grad h: Right , we 'll have to figure out a user interface for that , so . It might be fairly difficult to get that to work in the little short segments we 'd be talking about and having the search tools and so on . grad h: but professor b: The thing I was asking about with , free BSD is that it might be easier to get PowerPoint shows running in free BSD than to get this other package running in grad h: Yeah ,  we have to I have to sit down and try it before I make too many judgments , professor b: Yeah . grad h: so   My experience with the Gnu compatibility library is really it 's just as hard and just as easy to port to any system . grad h: It 's just , you know , just like all of them , the " include " files are a little different and the function calls are a little different . grad h: So I it might be a little easier but it 's not gonna be a lot easier . So there was that demo , which was one of the main ones , then we talked about  some other stuff which would basically be  showing off the the Transcriber interface itself and as you say , maybe we could even merge those in some sense , but but  ,  and part of that was showing off what the speech - non  nonspeech stuff that Thilo has done s looks like . postdoc a: Can I ask one more thing about THISL ? So with the IR stuff then you end up with a somewhat prioritized  ? grad h:   phd g: So another idea I w t had just now actually for the demo was whether it might be of interest to sh to show some of the prosody  work that Don 's been doing . phd g:  actually show some of the features and then show for instance a task like finding sentence boundaries or finding turn boundaries . professor b: Well I think at at the very least we 're gonna want something illustrative with that phd g: I don't know if that would be of interest or not . professor b: cuz I 'm gonna want to talk about it and so i if there 's something that shows it graphically it 's much better than me just having a bullet point phd g: Yeah . professor b: pointing at something I don't know much about , phd g: you 're looking at this now professor b: so . phd g: Are you looking at Waves or Matlab ? grad c:  yeah I 'm starting to and  Yeah we can probably find some examples of different type of prosodic events going on . phd g: Yeah def professor b: S so when we here were having this demo meeting , what we 're sort of coming up with is that we wanna have all these pieces together , to first order , by the end of the month phd g: I professor b: and then that 'll give us a week or so . The end of phd g: Oh , the end of this month or next month ? Oh , you mean like today ? grad h: This month . professor b:  that 'll that 'll give us that 'll give us a week or so to  to port things over to my laptop and make sure that works , phd e: Exactly . phd g: Yeah if d if Don can sort of talk to whoever 's professor b: Yeah . phd g: cuz we 're doing this anyway as part of our you know , the research , visualizing what these features are doing professor b: Yeah . professor b: so  , you know , to let s the goal is to let them know what it is we 're doing . professor b: So that 's phd g: I don't think anyone has done this on meeting data so it might be neat , you know . So I 've been doing a bunch of XML tools where you we 're sort of moving to XML as the general format for everything and I think that 's definitely the right way to go because there are a lot of tools that let you do extraction and reformatting of XML tools . So yet again we should probably meet to talk about transcription formats in XML because I 'm not particularly happy with what we have now .  it works with Transcriber but it it 's a pain to use it in other tools  because it doesn't mark start and end .  , but I 've installed XML tools of various sorts in various languages and so if people are interested in doing extracting any information from any of these files , either  information on users because the user database is that way I 'm converting the Key files to XML so that you can extract m  various inf  sorted information on individual meetings grad c: Cool . And so l just let me know there it 's mostly Java and Perl but we can get other languages too if if that 's desirable . Is do we have the the seat information ? In in the Key files now ? postdoc a:   grad h: The seat information is on the Key files for the ones which postdoc a: Ah . phd g: Oh in For the new one grad h: it 's been recorded , phd g: OK . phd g: I 'm just trying to figure out , you know , when Morgan 's voice appears on someone 's microphone are they next to him or are they across from him ? professor b: Yeah . phd f: how how w eh where is it in the Key file ? grad h: Right . phd g: Cuz  I haven't been putting it in and in by grad h: You haven't been putting it in . postdoc a: Isn't it always on the digits ? professor b: Some of these are missing . phd g: And professor b: Aren't they ? postdoc a: Isn't it always on the digits forms ? professor b: Some fall out of phd g: Well it grad h: Yeah so we can go back and fill them in for the ones we have . phd g:  they 're on th right , these , but I just hadn't ever been putting it in the Key files . phd f: Yeah I I never phd g: And I don't think Chuck was either phd f: I never knew we were supposed to put it in the Key file . phd g: cuz grad h: I had told you guys about it phd f: Oh really ? phd g: Oh , so we 're both sorry . grad h: but phd g: So grad h:  this is why I wanna use a g a tool to do it rather than the plain text phd g: OK . grad h: I think it 's Edit - key , command Did I show you guys that ? phd d: Yep . phd f: You mentioned it , grad h: I did show it to you , phd f: yeah . grad h: but I think you both said " no , you 'll just use text file " . grad h: Yeah , and so if you don't fill it in , you 're not gonna get it in the meetings . grad c: u grad h: Yeah and then the other thing also that Thilo noticed is , on the microphone , on channel zero it says hand - held mike or Crown mike , phd g: Yeah . phd g: maybe I forgot to d phd f: Takes me no time at all to edit these . grad h: Yeah that 's cuz you kn phd f: I 'm not doing anything . phd g: And I was I was looking at Chuck 's , like , " oh what did Chuck do , OK I 'll do that " . grad h: And then  also in a couple of places instead of filling the participants under " participants " they were filled in under " description " . Oh  also I 'm working on another version of this tool , the the one that shows up here , that will flash yellow if the mike isn't connected . And it 's not quite ready to go yet because  it 's hard to tell whether the mike 's connected or not because the best quality ones , the Crown ones , are about the same level if they 're off and no one 's o off or if they 're on and no one 's talking . So I 'm working on that and it it sorta works and so eventually we will change to that and then you 'll be able to see graphically if your mike is dropping in or out . grad c: Will that also include like batteries dying ? Just a any time the mike 's putting out zeros basically . phd f: But with the screensaver kicking in , it phd d: But grad h: Now phd d: y yeah . grad h:  the other thing is as I 've said before , it is actually on the thing . postdoc a: It would be nice if if these had little light indicators , little L E Ds for grad h:  buzzer . grad h: " Bamp , bamp ! " professor b: Small shocks postdoc a: Yeah . Oh grad h: OK , disk backup , et cetera ?  I spoke with Dave Johnson about putting all the Meeting Recorder stuff on non - backed - up disk to save the overhead of backup and he pretty much said " yeah , you could do that if you want " but he thought it was a bad idea . In fact what he said is doing the manual one , doing  NW archive to copy it is a good idea and we should do that and have it backed up . grad h: And so if a mistake is made and we lose the backup we should have the archive and if then a mistake is made and we lose the archive we should have the backup . professor b: Well I guess it is true that even with something that 's backed up it 's not gonna if it 's stationary it 's not going to go through the increment it 's not gonna burden things in the incremental backups . But he said that that we sh shouldn't worry too much about that , that we 're getting a new backup system and we 're far enough away from saturation on full backups that it 's w probably OK . professor b: Really ? grad h: And  , so the only issue here is the timing between getting more disks and  recording meetings . professor b: So I guess the idea is that we would be reserving the non - backed - up space for things that took less than twenty - four hours to recreate or something like that , right ? grad h: Things that are recreatable easily and also Yeah , basically things that are recreatable . professor b: Well I was allowing someone else to come up with something related that they had  phd e: I thought you guys were gonna burn C Ds ? grad h:  unfortunately we could burn C Ds but first of all it 's a pain . grad h: Because you have to copy it down to the PC and then burn it and that 's a multi - step procedure . And second of all the the write - once burners as opposed to a professional press don't last . grad h: So I think burning them for distribution is fine but burning them for backup is not a good idea . Can I add one top topic ? We have time ? I wanted to ask , I know that  that Thilo you were , bringing the Channeltrans interface onto the Windows machine ? And I wanted to know is th phd d: Yeah it 's it Basically it 's done , postdoc a: It 's all done ? That 's g wonderful . grad h: Yes , since Tcl - TK runs on it , basically things 'll just work . phd d: Yeah it Yeah , it was just a problem with the Snack version and the Transcriber version but it 's solved . Does this mean that the that this could be por  ported to a Think - Pad note or some other type of  phd d: Yeah , basically  I did install it on my laptop and yeah postdoc a: Wonderful . CrossPads ? CrossPads ? grad h:  got an email from  James Landay who basically said " if you 're not using them , could you return them ? " So he said he doesn't need them , he just periodically w at the end of each term sends out email to everyone who was recorded as having them and asks them if they 're still using them . professor b: Once ? grad h: We we used them a couple times , postdoc a:   phd f: Them ? There 's more than one ? grad h: but postdoc a: Yeah . professor b: But grad h: My opinion on it is , first , I never take notes anyway so I 'm not gonna use it ,  and second , it 's another level of infrastructure that we have to deal with . postdoc a: And I have  so my my feeling on it is that I think in principle it 's a really nice idea , and you have the time tags which makes it better tha than just taking ra raw notes . On the other hand , I the down side for me was that I think the pen is really noisy . And I and I don't know if it 's audible on the but I I sort of thought that was a disadvantage . I do take notes , I could be taking notes on these things and I guess the plus with the CrossPads would be the time markings but I don't know . phd d: what is a CrossPad ? professor b: So it 's it 's  it 's a regular pad , just a regular pad of paper but there 's this pen which indicates position . grad h: And then you can download it and they have OCR and searching and all sorts of things . grad h: But I don't take notes , professor b: And one of the reasons that it was brought up originally was because  we were interested in in higher - level things , grad h: so . professor b: not just the , you know , microphone stuff but also summarization and so forth and the question is if you were going to go to some gold standard of what wa what was it that happened in the meeting you know , where would it come from ? And  I think that was one of the things , phd d: Yeah . We 'll have a you know , have a scribe , have somebody  take good notes and then that 's part of the record of the meeting . And then we did it once or twice and we sort of grad h: Yep , and then just sort of died out . professor b: probably chose the wrong scribe but it was It 's  phd g:  grad h: Yeah that 's right . postdoc a: u but I guess the the other thing I 'm thinking is if we wanted that kind of thing I wonder if we 'd lose that much by having someone be a scribe by listening to the tape , to the recording afterwards and taking notes in some other interface . phd f:  we 're transcribing it anyways , why do we need notes ? postdoc a: Oh it 's la it 's useful , grad h: Because that 's summary . phd g: I think there 's also there 's this use that phd f: Summarize it from the transcription . phd g: the Well , what if you 're sitting there and you just wanna make an X and you don't wanna take notes and you 're you just wanna phd f: Doodle . phd g: get the summary of the transcript from this time location like you know , and and then while you 're bored you don't do anything and once in a while , maybe there 's a joke and you put a X and But in in other words you can use that just to highlight times in a very simple way . Also with I was thinking and I know Morgan disagrees with me on this but suppose you have a group in here and you wanna let them note whenever they think there might be something later that they might not wanna distribute in terms of content , they could just sort of make an X near that point or a question mark that sort of alerts them that when they get the transcript back they c could get some red flags in that transcript region and they can then look at it . I know we haven't been using it but I w I can imagine it being useful just for sort of marking time periods grad h: Right . professor b: I guess so , you know , what what makes one think i is maybe we should actually schedule some periods where people go over something later phd g: so . professor b: and and and put some kind of summary or something  you know , some there 'd be some scribe who would actually listen , w who 'd agreed to actually listen to the whole thing , not transcribe it , but just sort of write down things that struck them as important . But then you don't you don't have the time reference  that you 'd have if you had it live . And you don't have a lot of other cues that might be useful , professor b: Yeah . phd f: How do you synchronize the time in the CrossPad and the time of the recording ? phd g: so . grad h:  that was one of the issues we talked about originally and that that 's w part of the difficulty is that we need an infrastructure for using the time the CrossPads and so that means synchronizing the time phd g:  postdoc a:   grad h: You know you want it pretty close and there 's a fair amount of skew because it 's a hand - held unit with a battery postdoc a: Well when when I d grad h: and so you postdoc a: OK . grad h: so you have to synchronize at the beginning of each meeting all the pads that are being used , so that it 's synchronized with the time on that and then you have to download to an application , and then you have to figure out what the data formats are and convert it over if you wanna do anything with this information . phd e: Why grad h: And so there 's a lot of infrastructure which postdoc a: There is an alternative . grad h: unless someone postdoc a: There is an alternative , it 's still , there 's  you know , your point stands about there be needing to be an infrastructure , but it doesn't have to be synchronized with the little clock 's timer on it . You c  , I when I when I did it I synchronized it by voice , by whispering " one , two , three , four " onto the microphone grad h:  . grad h: Well , but then there 's the infrastructure at the other end phd e: Right . grad h: which someone has to listen to that and find that point , postdoc a: Yeah , it 's transcribed . phd g: Well , could we keep one of these things for another year ? Would h  is there a big cau grad h: We can keep all both of them for the whole whole year . phd g: just just in case we grad h: it 's just phd g: even maybe some of the transcribers who might be wanting to annotate  f just there 's a bunch of things that might be neat to do but I it might not be the case that we can actually synchronize them and then do all the infrastructure but we could at least try it out . professor b: Well one thing that we might try  is on some set of meetings , some collection of meetings , maybe EDU is the right one or maybe something else , we we get somebody to buy into the idea of doing this as part of the task . professor b:  part of the reason I think part of the reason that Adam was so interested in  the SpeechCorder sort of f idea from the beginning is he said from the beginning he hated taking notes and grad h: Yep . professor b: and so forth so and and Jane is more into it but eh  you know I don't know if you wanna really do do this all the time so I think the thing is to to get someone to actually buy into it and have at least some series of meetings where we do it . The p the the problem with the the more extended view , all these other you know with  quibbling about particular applications of it is that it looks like it 's hard to get people to   routinely use it ,  it just hasn't happened anyway . But maybe if we can get a person to phd g: Yeah I don't think it has to be part of a what everybody does in a meeting but it might be a useful , neat part of the project that we can , you know , show off as a mechanism for synchronizing events in time that happen that you just wanna make a note of , like what Jane was talking about with some later browsing , just just as a convenience , even if it 's not a full - blown note taking substitute . phd e: Well if you wanted to do that maybe the right architecture for it is to get a PDA with a wireless card . And and that way you can synchronize very easily with the the the meeting because you 'll be synchroni you can synchronize with the the Linux server and  phd g: So what kind of input would you be ? phd e: so so , if you 're not worried about grad h: Buttons . phd g: You 'd just be pressing like a a phd e: Well well you have a PDA and may and you could have the same sort of X interface or whatever , you 'd have to do a little eh a little bit of coding to do it . phd e: if if all you really wanted was you didn't want this secondary note - taking channel but just sort of being able to use m markers of some sort , a PDA with a l a wireless card would be the probably the right way to go . grad h:  for what what you 've been describing buttons would be even more convenient than anything else , phd g: M right . grad h: right ? You have the phd g: I don't have , you know , grandiose ideas in mind but I 'm just sort of thinking well we 've we 're getting into the next year now and we have a lot of these things worked out at in terms of the speech maybe somebody will be interested in this and postdoc a: I like this PDA idea . professor b: where like one one button was "  - oh " and then another button was " that 's great " and another button " that 's f " phd g: Or like this is my " I 'm supposed to do this " kind of button , postdoc a: Yeah . phd g: Yeah something like that or postdoc a: And then grad h:  I think the CrossPad idea is a good one . grad h: It 's just a question of getting people to use it and getting the infrastructure set up in such a way that it 's not a lot of extra work .  that 's part of the reason why it hasn't happened is that it 's been a lot of extra work for me phd g: Yeah . grad h: and postdoc a: But it 's also , it has this problem of having to go from an analog to a d a digital record too , phd g: W postdoc a: doesn't it ?  grad h: Well it 's digital but it 's in a format that is not particularly standard . postdoc a: But  , say , if i if if you 're writing if you 're writing notes in it does it it can't do handwriting recognition , right ? professor b: No , no , but it 's just it 's just storing the pixel informa position information , postdoc a: OK . postdoc a: I I guess what I 'm thinking is that the PDA solution you h you have it already without needing to go from the pixelization to a to a  professor b: Right . You don't have to phd e: The transfer function is less errorful , postdoc a: Oh , nicely put . phd g: Well it also it 's maybe realistic cuz people are supposed to be bringing their P D As to the meeting eventually , right ? That 's why we have this little I don't know what I don't wanna cause more work for anyone but I can imagine some interesting things that you could do with it and so if we don't have to return it and we can keep it for a year I don't know . grad h: Well w we don't we certainly don't have to return it , as I said . All all he said is that if you 're not using it could you return it , if you are using it feel free to keep it . The point is that we haven't used it at all and are we going to ? professor b: So we have no but  by I I would suggest you return one . phd g: We c professor b: We have some aspirations of using them phd g: One would probably be fine . professor b: and phd g: Maybe we could do like a student project , you know , maybe someone who wants to do this as their main like s project for something would be cool .  if we had them out and sitting on the table people might use them a little more professor b: Maybe Jeremy could sit in some meetings and press a button when there when when somebody laughed . grad h: although there is a little phd g: Well , I 'm yeah , that 's not a bad professor b: Yeah , yeah . phd g: Jeremy 's gonna be an he 's a new student starting on modeling brea breath and laughter , actually , which sounds funny but I think it should be cool , professor b: Yeah . phd g: You 're you 're gonna tease me ? grad h: Oh , equipment . grad h: And then at the same time I 'll probably rewire the room as per Jane 's suggestion so that  the first N channels are wireless , eh are the m the close - talking and the next N are far - field . professor b: You know what he means but isn't that funny sounding ? " We ordered more wireless . " It 's like wires are the things so you 're wiring you 're you 're you we 're we ordered more absence of the thing . phd g: I just it 's sort of a anachronism ,  it 's like It 's great . professor b:  there 's there 's all this stuff going on  between  Andreas and and and Dave and Chuck and others with various kinds of runs   recognition runs , trying to figure things out about the features but it 's it 's all sort of in process , so there 's not much to say right now . grad h: So just the transcript number and then the then the phd e: This is Yes , this is number two for me today . professor b: See all you have to do is go away to move way up in the phd e: Oh . grad h: Should we do simultaneous ? phd g: Well , I 'm just thinking , are you gonna try to save the data before this next group comes in ? professor b: Yeah . phd g: You have to plug your ears , by the way  Eric , grad h: Well I have to , phd d: You don't have to <doc-sep>grad a: Alright , so I 'm - I should read all of these numbers ? professor b: OK . professor b: OK , so  i  I don't know whether Ami 's coming or not  but I think we oughta just get started . grad e: Nancy is  currently in Berkeley but not here ? grad c: Nancy 's still stick ? professor b: Don't know . Anyway , so my idea f for today and we can  decide that that isn't the right thing to do was to at spend at least part of the time trying to eh build the influence links , you know which sets of things are  relevant to which decisions and actually I had  specific s suggestion to start first with the path ones . The database ones being in some sense less interesting to us although probably have to be done and so to do that so there 's and the idea was we were gonna do two things grad c: Is your mike on ? professor b: Ah . We were gonna do two things one of which is just lay out the influence structure of what we think influences what grad d: That 's funny . professor b: and then as a  separate but related task  particularly Bhaskara and I were going to try to decide what kinds of belief nodes are needed in order to  do what we what we need to do . Once so but du we should sort of have all of the  basic design of what influences what done before we decide exactly how to compute it . So I didn't did you get a chance to look at all yet ? grad d: Yeah , I looked at some of that stuff . OK so let 's start with the  belief - nets , the general influence stuff and then we 'll then we 'll also at some point break and talk about the techy stuff . grad e: Well I think one could go there 's I think we can di discuss everything . First of all this I added , I knew from sort of basically this has to be there right ?  professor b: Oh are you gonna go there or not ? Yeah , so one i grad e: Given given   not transverse the castle , the decision is does the person want to go there or is it just professor b: Right , true . And I 'm sure we 'll find more as we go that grad e: And  ? So Go - there in the first place or not is definitely  one of the basic ones .  Is this basically true or false or maybe we 'll get professor b: Well grad d: Which one ? grad e: what ? grad a: " Go there " . professor b: so there is this question about grad e: Here we we actually get just probabilities , professor b: Yeah . professor b: the the reason it might not be true or false is that we did have this idea of when so it 's , you know  current @ @ and so forth and so on or not at all , grad e:   professor b: right ? And so that a decision would be do we want that so you could two different things you could do , you could have all those values for Go - there or you could have Go - there be binary and given that you 're going there when . grad e:  ? grad a:  it seems that you could   it seems that those things would be logically independent like you would wanna have them separate or binary , Go - there and then the the possibilities of how to go there because professor b: OK , that 's let 's start that way . grad a: because , you know it might be easy to figure out that this person is going to need more film eventually from their utterance but it 's much more complex to query when would be the most appropriate time . And so I 've tried to come up with some initial things one could observe so who is the user ? Everything that has user comes from the user model everything that has situation comes from the situation model - A . But when it comes to sort of writing down when you when you do these things is it here ? You sort of have to a write the values this can take . grad e: And here I was really  in some s sometimes I was really sort of standing in front of a wall feeling very stupid because  this case it 's pretty simple , but as we will see the other ones  for example if it 's a running budget so what are the discrete values of a running budget ? So maybe my understanding there is too impoverished . professor b: No  grad e: How can I write here that this is something , a number that cr keeps on changing ? But OK . professor b: You 've s have you seen this before at all Keith , these belief - net things ? grad a: no , but I think I 'm following it . grad e: So here is the the we had that the user 's budget may influence the outcome of decisions . grad d: Is this like a number that represents how much money they have left to spend ? OK , h well  how is it different from user finance ? grad e:  the finance is sort of here thought of as as the financial policy a person carries out in his life , he is he cheap , average , or spendy ? grad d: Alright . grad e: And  I didn't come  maybe a user I don't know , I didn't want to write greediness , but grad a: Yeah . So Keith w what 's behind this is actually a program that will once you fill all this in actually s solve your belief - nets for you and stuff . professor b: So this is not just a display , this is actually a GUI to a simulator that will if we tell it all the right things we 'll wind up with a functioning belief - net at the other end . grad e: OK , so here was OK , I can think of  people being cheap , average , or spendy or we can even have a a finer scale moderately cheap , professor b: Doesn't matter . grad d: Well ,  you 've written in you 've written in what  seems to be required like what else is is do you want ? grad e: If that 's permissible then I 'm happy . So here 's here 's what 's permissible is that you can arrange so that the  the value of that is gonna have to be updated and n it 's not a belief update , right ? It 's you took some actions , you spent money and stuff , so the update of that is gonna have to be essentially external to the belief - net . And if it does influence anything then you 're gonna need something that converts from the the number here to something that 's relevant to the decision there . So it could be ra they create different ranges that are relevant for different decisions or whatever but for the moment this is just a node that is conditioned externally and might influence various things . grad e: OK , and so this , oh that grad d: The other thing is that  every time that 's updated beliefs will have to be propagated but then the question is do you do we wanna propagate beliefs every single time it 's updated or only when we need to ? professor b: Yeah , that 's a good question . grad d:  Well , in Srini 's thing there was this thing there was this  option like proper inferences which suggests that  doesn't happen , automatically . Yeah someone has to track that down , but I but  And and and I think actually  grad e: I just accidentally Oops . professor b: one of the we w items for the  user home base  should be  essentially non - local . I they 're only there for the day and they don't have a place that they 're staying . grad d: Well grad e: Oh just  accidentally erased this , I I just had values here such as   is he s we had in our list we had " Is he staying in our hotel ? " , " Is he staying with friends ? " , and so forth professor b: Yeah . So my suggestion is we just pick  grad e: Something down here ? professor b: one , you know one  particular one of the  well let 's do the first first one let 's do the one that we sort of already think we did so w that was the of the endpoint ? grad e:   grad d: Is  grad e: Ah , grad d: So it 's true or false ? professor b: No , that 's that 's a grad e: OK . grad c: What 's the difference between mode and endpoint ? grad d: I thought mode , yeah . professor b: although that grad e:  mode was  professor b: Well , that 's grad d: Mode of transportation ? grad e: Yeah . professor b: No , he has he hasn't filled them in yet , is what 's true . grad e: climb , emerge professor b: No no no , these are ju that 's just a point , grad c:  grad d: Well some of those are subsumed by approach . professor b: this is ju grad c: Would it be an endpoint if you were crossing over it ? grad a: The Charles Bridge , you know . You know , you y you go first go the town square grad c: Well I eh grad a: No , if you go to re you know if you go to Prague or whatever one of your your key points that you have to do is cross the Charles Bridge and doesn't really matter which way you cross which where you end up at the end but the part the good part is walking over it , so . Anyway so let 's just leave it three with three for now grad e:  -  , mmm . professor b: and let 's see if we can get it linked up just to get ourselves started . grad e: OK , we professor b: You 'll see it you 'll see something comes up immediately , that the reason I wanna do this . grad e: w well the  user was  definitely more likely to enter if he 's a local professor b: Right . grad e: more likely to view if he 's a tourist  and then of course we had the fact that given the fact that he 's thrifty and there will be admission then we get all these cross  professor b: We did , but the three things w that that it contributed to this in fact , the other two aren't up there . so one was the ontology grad e: We 'll d what type of building is it ? professor b: Yeah . professor b: OK , so this is w Right , so what w I what we seem to need here , this is why it starts getting into the technical stuff grad a:  -  professor b: the way we had been designing this , there were three intermediate nodes  which were the endpoint decision as seen from the  user model as seen from the ontology and as seen from the discourse . So each of those the way we had it designed , now we can change the design , but the design we had was there was a decision with the same three outcomes  based on the th those three separate considerations grad a:  -  professor b: so if we wanted to do that would have to put in  three intermediate nodes grad e:  we can load it up it you know very simple . grad a: So professor b: and then what you and I have to talk about is , OK if we 're doing that and they get combined somehow  how do they get combined ? But the they 're they 're undoubtedly gonna be more things to worry about . grad e: So that 's w w in our  in in Johno 's sort of pictogram everything that could contribute to whether a person wants to enter , view , or approach something . professor b: Oh , it was called mode , so this this is m mode here means the same as endpoint . professor b: OK , why don't we ch can we change that ? grad e: We can just rename that , yeah . You know , but that was actually , yeah unfortunately that was a  kind of an intermediate versio that 's I don't think what we would currently do . grad a: Can I ask about " slurred " and " angry " as inputs to this ? professor b: That 's a grad a: What why ? grad d: Like they 're either true or false grad e: The prosody ? grad a: OK . grad c: If the if the person talking is angry or slurs their speech they might be tired or , you know grad a:   grad d: Therefore grad c: And , you know , possibly  grad a: Less likely to enter . So so my advice to do is is get this down to what we think is actually likely to to be a a strong influence . professor b: So let 's think about this this question of how do we wanna handle so there 're two separate things . One is how do we want to handle the notion of the ontology now what we talked about , and this is another technical thing Bhaskara , is  can we arrange so that I think we can so that the belief - net itself has properties and the properties are filled in  from on ontology items . So the let 's take the case of the  this endpoint thing , the notion was that if you had a few key properties like is this a tourist site , you know some kind of landmark is it a place of business  is it something you physically could enter grad a:   So that there 'd be certain properties that would fit into the decision node and then again as part of the ou outer controlling conditioning of this thing those would be set , so that some somehow someone would find this word , look it up in the ontology , pull out these properties , put it into the belief - net , and then the decision would flow . professor b: Now grad e: Seems to me that we 've sort of e em embedded a lot , em embedded a lot of these  things we had in there previously in in in some of the other final decisions done here , for example if we would know that this thing is exhibiting something  professor b: Right . grad e: if it is exhibiting pictures or sculptures and stuff like this , then it 's more likely to be entered . professor b: I  that 's I think that 's completely right and  I think that 's good , right ? So what what that says is that we might be able to  take and in particular so so the ones we talked about were  exhibiting and selling grad e: Accessibility . professor b: no , accessibility meant grad e: If it 's closed one probably won't enter . Or if it 's not accessible to a tourist ever the likelihood of that person actually wanting to enter it , professor b: OK . grad e: Yeah , all all of these if it 's fixing things selling things , or servicing things professor b: Right . is that you want an intermediate structure which i  is essentially the or of  for this purpose of of  selling , f fixing , or servicing . So that it  that is , for certain purposes , it becomes important but for this kind of purpose  one of these places is quite like the other . Does that seem right ? So we di grad c: Basic you 're basically just merging those for just the sake of endpoint decision ? professor b: if we Yes . professor b: So if well it may be more than endpoint decisions , so the idea would be that you might wanna merge those three grad e: These three ? professor b: Yeah . grad d: What ex  and so either those is true f or false ? professor b:   well it it i here 's where it gets a little tricky . grad d: So professor b:  from the belief - net point of view it is from another point of view of course it 's interest it 's it 's important to know what it 's selling or servicing and so forth . professor b: That that if any of those things is true then it 's the kind of place that you  grad e:  more likely to enter . grad d: So you just wanna have them all pointing to a summary thing ? professor b: You could , yeah . professor b: That 's the p part of the idea , is grad e:  is is that the object type node ? professor b: I d grad e: So are they the is it the kind of object that sells , fixes , or services things ? professor b: Well , o open up object type and let 's see what its values are . professor b: Oh , well OK first of all it 's not objects , we called them entities , right ? grad e: Yeah . grad e: Yeah , I w I was just gonna commercial action inside where people p professor b: Well couldn't I do let 's do commercial  landmark and grad e: And where was the accessible , yeah . professor b: Well accessible I think is different cuz that 's tempor that that varies temporally , grad e: Yeah . grad c: What would a hotel fall under ? professor b: I would call that a service , but but I don't know . grad c: Well  in terms of entity type ? professor b: Say w w well it 's co I would s a a again for this purpose I think it 's commercial . grad d:  what does the underscore - T at the end of each of those things signify ? grad e:  things . grad a: So we 're deriving  this the this feature of whether the the main action at this place happens inside or outside or what we 're deriving that from what kind of activity is done there ? Couldn't you have it as just a primitive feature of the entity ? professor b: Well you could , that 's a that 's a choice . professor b: So  grad a:  it seems like that 's much more reliable cuz you could have outdoor places that sell things and you know indoor places that do something else professor b: Yeah , the problem with it is that it sort of putting in a feature just for one decision , grad a: and  . professor b: now w we may wind up having to do that this i anyway , this i grad a: OK . professor b: at a mental level that 's what we we 're gonna have to sort out . professor b: So , you know what does this look like , what are what are  intermediate things that are worth computing , what are the features we need in order to make all these decisions grad a:   professor b: and what 's the best way to organize this so that  it 's clean and and consistent and all that sort of stuff . I 'm just thinking about how people , human beings who know about places and places to go and so on would store this and it would probably you wouldn't just sort of remember that they sell stuff and then deduce from that that it must be going on inside or something . grad e: Well I think an entity maybe should be regard as a vector of several possible things , it can either em do s do sell things , fix things , service things , exhibit things , it can be a landmark at the same time as doing these things , grad a:   grad e: it 's not either or mmm certainly a place can be a hotel and a famous site . IE a a castle or can be a landmark a or not accessible , some statue grad a:    which is to get rid get rid of that l long link between who the user and the endpoint . grad e: Could we just move it like this ? professor b: No no , I don't want the link there at all . professor b: Because what we 're gonna want is an intermediate thing which is  the endpoint decisi the endpoint decision based o on the user models , so what we we what we talked about is three separate endpoint decisions , so let 's make a new node grad e: Yeah . grad c: Just as a suggestion maybe you could " save as " to keep your old one nice and clean and so you can mess with this one . The old one was not that not that important , I think but grad c: OK , well , not a big deal then . grad c: Well the Isn't there a " save as " inside of java base ? grad e: But I can just take this grad c: OK . This was user something professor b: Well this was grad e: or professor b:  let 's p put it this let 's do endpoint underbar - U . grad e: end point ? professor b: i endpoint , e end poi this is sa grad e: Ah . professor b: let 's say underbar - U , so that 's the endpoint decision  as seen through the grad c: As related from the user model . So let 's let 's actually yeah so lin you can link that up to the grad e: Should I rename this too ? professor b:  yeah , so that , I guess that 's endpoint  grad e:  It 's underscore - E . Or ? Can I ? Where ? What ? professor b: Oh I d eh I don't know . grad c: I think you have to be in move mode before grad e:  - huh . And  maybe th maybe it 's just one who is the user , I don't know , maybe maybe there 's more . grad e: Well if he 's usi if he 's in a car right now what was that people with Harry drove the car into the cafe professor b: Never mind . Now but the now so so but then the question is  so and and we assume that some of these properties would come indirectly through an ontology , but then we had this third idea of input from the discourse . grad e: Well let 's should we finish this , professor b: Sure , grad e:  but surely the user interests professor b: OK . grad e: yeah , yeah professor b: Well , maybe , I again , I d well , OK , put em in but what we 're gonna wanna do is actually  grad c: Well is grad e: Here this was one of my problems we have the user interest is a is a vector of five hundred values , so  That 's from the user model , grad d: Oh you mean level of interest ? grad e:  -  , no not levels of interest but things you can be interested in . grad d: Oh I see , grad e: Gothic churches versus Baroque townhouses versus grad d: right . So why is it oh it , so it 's like a vector of five hundred one 's or zero 's ? grad e: Yea - n is that grad d: Like for each thing are we are you interested in it or not ? grad e: yeah  I I think grad d: I see . But the other thing you could do is have an and this will give you the flavor of the of what you could have a node that 's that was a measure of the match between the object 's feature , you know , the match between the object the entity , I 'm sorry and the user . professor b: So you could have a k a " fit " node and again that would have to be computed by someone else grad e:   professor b: but  so that  grad e: Just as a mental note  professor b: Yeah , that 's all . And and should we say that this interests eh affects the likelihood of of entering ? professor b: Yeah . " Do I have time to go in and climb all the way to the top of the Koelner Dome or do I just have to " " time to take a picture of the outside ? " grad e: Schedule ? professor b: Right . grad c: It seems like everything in a user model a affects professor b: Well that 's what we don't wanna do , see that se cuz then we get into huge combinatorics and stuff like that grad c: Yeah . professor b: an grad c: Cuz if the , and if the user is tired , the user state , grad d: Well grad c: right , it would affect stuff , but I can't see why e anything w everything in the model wouldn't be professor b: Well , but grad d: Right . professor b: Well , that that 's we can't do that , so we we 're gonna have to grad c: Yeah . professor b: but this is a good discussion , we 're gonna have to somehow figure out  some way to encapsulate that  so if there 's some general notion of for example the  relation to the time to do this to the amount of time the guy has or something like that is is the  compatibility with his current state , so that 's what you 'd have to do , you 'd have to get it down to something which  was itself relatively compact , so it could be compatibility with his current state which would include his money and his time and and his energy grad c: Yeah , just seems like it 'd push the problem back a level . grad c: Yeah , but grad d: No but , it 's more than that , like the the more sort of you break it up like because if you have everything pointing to one node it 's like exponential whereas if you like keep breaking it up more and more it 's not exponential anymore . That 's tha there 's one technical one grad c: Sh - sh yeah , professor b: and the other is it it gets used grad c: S so we 'd basically be doing subgrouping ? Subgrouping , basically into mo grad d: Yeah . But it there 's two advantages , one is the technical one that you don't wind up with such big exponential  CBT 's , grad e: Bhaskara ? professor b: the other is it can be it presumably can be used for multiple decisions . professor b: So that if you have this idea of the compatibility with the requirements of an action to the state of the user one could well imagine that that was u grad d: Right . professor b: not only is it sim is it cleaner to compute it separately but it could be that it 's used in multiple places . professor b: OK , you 've got a signal , a d set of decisions  how do we do this ? grad e: What do I have under user state anyhow cuz I named that already something . grad c: I guess the the question  is It 's hard for me to imagine how everything wouldn't just contribute to user state again . professor b: Oh but the thing is that we   we had some things that  grad e: That don't . professor b: that don't grad e: The user interests and the user who who who the user is are completely apart from the fact whether he is tired broke grad c: Sure , but other I thought though the node we 're creating right now is user compatibility to the current action , right ? professor b: the right grad c: Seems like everything in the user model would contribute to whether or not the user was compatible with something .  the that 's the the issue is  would Even if it was true in some abstract general sense it might not be true in terms of the information we actually had and can make use of . And anyway we 're gonna have to find some way to cl  get this sufficiently simple to make it feasible . grad e: Maybe  if we look at the if we split it up again into sort of  if we look at the  the endpoint again we we said that for each of these things there are certain preconditions so you can only enter a place if you are not too tired to do so and also eh have the money to do so if it costs something so if you can afford it and perform it is preconditions . professor b: W w but that eh viewing it without ent yeah view w with our definition of view it 's free cuz you grad e: And so is approaching .  are there are there large things that you would have to pay to get up close to like ,  never mind , not in the current professor b: No we have to enter the park .  So let me suggest we switch to another one ,  clearly there 's more work to be done on this grad e:   professor b: but I think it 's gonna be more instructive to to think about  other decisions that we need to make in path land . grad c: So you can save this one as and open up the old one , right and and then everything would be clean . professor b: Why , I think it 's worth saving this one but I think I 'd I 'd like to keep this one grad d: Yeah . professor b: cuz I wanna see if if we 're gonna reuse any of this stuff . grad e:  so this might be What next ? professor b: Well you tell me , so in terms of the  planner what 's what 's a good one to do ? grad e: Well let 's th this go there or not I think is a good one . So what makes things more likely that professor b: Well the fir see the first thing is , getting back to thing we left out of the other is the actual discourse . grad e: So professor b: So Keith this is gonna get into your world because  we 're gonna want to know you know , which constructions indicate various of these properties grad a:   professor b: s and so I I don't yet know how to do this , I guess we 're gonna wind up pulling out  discourse properties like we have object properties and we don't know what they are yet . professor b: So that that the Go - there decision will have a node from  discourse , and I guess why don't we just stick a discourse thing up there to be as a placeholder for grad e: We we also had discourse features of course for the endpoint . grad e: and so again re that 's completely correct , we have the user model , the situation model here , we don't have the discourse model here yet . professor b: Well the ontology we sort of said we would pull these various kinds of properties from the ontology like exhibiting , selling , and so forth .  This be specific for second year ?  And and we probably will have  something like a discourse for endpoint . grad e:  ? professor b: It 'll have the EVA values if if we have it . OK just for starters and here discourse  professor b: For Go - there , probably is true and false , let 's say . grad e:  well , I think  we 're looking at the the little data that we have , so people say how do I get to the castle and this usually means they wanna go there . grad e: however people also sometimes say how do I get there in order to find out how to get there without wanting to go there . grad e: because they wanna know where it is but in most cases they probably professor b: Yeah , but that doesn't change the fact that you 're you want these two values . So this is sort of some external thing that takes all the discourse stuff and then says here it 's either yep , yay , A , or nay . OK ? professor b: And they 'll be a y  , a user Go - there and maybe that 's all , I don't know . grad d: Situation Go - there , because it 's whether it 's open or not . grad d:  the that interacts with the  EVA thing if they just wanna view it then it 's fine to go there when it 's closed whereas if they want to  professor b: Right . grad d: so professor b: Right , so that 's that 's where it starts getting to be   essentially more interesting , so what  Bhaskara says which is completely right is if you know that they 're only going to view it then it doesn't matter whether it 's closed or not grad a:   So yeah , that 's what I said just having one situational node may not be enough because this that node by itself wouldn't distinguish professor b: Well i i it can have di various values . grad d: Yeah , see I 'm I 'm thinking that any node that begins with " Go - there " is either gonna be true or false . grad a: Also , that node ,  the Go - there s S node would just be fed by separate ones for grad e:   grad a: you know , there 's different things , the strikes and the professor b: Could be . So so now the other thing that Bhaskara eh pointed out is what this says is that  there sh should be a link , and this is where things are gonna get very messy from the endpoint  decision grad d: I guess the final professor b: maybe the t they 're final re and , I guess the very bottom endpoint decision  to the Go - there node . grad d: Maybe we could  have intermediate node that just the Endpoint and the Go - there S node sort of fed into ? professor b: Could be , yeah . Well the Go - there , actually the Endpoint node could feed feed into the Go - there S That 's right , grad d: Yeah , right . professor b: and again we 'll have to do layout at some point , but something like that .  really important in in the belief worl net world not to have loops grad e: I was just gonna professor b:  grad d: Yes . grad e: How long does it take you to to compute  professor b: No it 's much worse than that . It if i loo it it it it it 's not def i it 's not well defined if you 're there are loops , grad d: It things don't converge , yeah . grad e:  R recursive action ? professor b: you just you have to there are all sorts of ways of breaking it up so that there isn't  OK . grad e:  but this isn't , this is this line is just coming from over here . professor b: Yeah , no it 's not a loop yet , I 'm just saying we we , in no , in grad d: Yeah .  what 's what 's another decision you like ? grad e: OK , these have no parents yet , but I guess that sort of doesn't matter . Right ? professor b: Well , the idea is that you go there , you go comes from something about the user from something about the situation and the  the discourse is is a mystery . grad a:  actually professor b: And then also the discourse endpoint , I I guess endpoint sub - D is if you wanna make it consistent . grad a:  actually is this the the right way to have it where  go there from the user and go there from the situation just sort of don't know about each other but they both feed the go there decision because isn't the ,  professor b: I think so . But that still allows for the possibility of the of the user model affecting our decision about whether a strike is the sort of thing which is going to keep this user away from professor b: Maybe not , a Right . grad a: But  OK I was just thinking I guess maybe I 'm conflating that user node with possible possible asking of the user professor b: Yeah . grad a: you know hey there 's a strike on ,  does that affect whether or not you wanna go or something professor b: Ah . Good point , I don't I don't know how we 're going to t  grad a: or Yeah , so that might not come out of a user model but , you know , directly out of interaction . professor b:  My current idea on that would be that each of these decision nodes has questions associated with it . professor b: And the question wouldn't itself be one of these conditional things grad a: OK . professor b: you know , given that you know there 's a strike do you still wanna go ? grad a: Yeah . professor b: But  if you told him a bunch of stuff , then you would ask him do you wanna go ? grad a:   professor b: Alright , but let me let let 's stay with this a minute grad e: But professor b: because I want to do a little bit of organization . The organization is going to be that  the flavor of what 's going on is going to be that  as we s e sort of going to this detail indeed Keith is going to to worry about the various constructions that people might use grad a:   professor b: and Johno has committed himself to being the parser wizard , grad a: Alright . professor b: so what 's going to happen is that eventually like by the time he graduates , OK  they 'll be some sort of system which is able to take the discourse in context and have outputs that can feed the rest of belief - net . I j wa I I assume everybody knows that , I just wanna you know , get closure that that 'll be the game then , grad a:   professor b: so the semantics that you 'll get out of the discourse will be of values that go into the various discourse - based decision nodes . And now some of those will get fancier like mode of transportation and stuff so it isn't by any means  necessarily a simple thing that you want out . So  if there is an and there is mode of transportation grad e: And it there 's a sort of also a split if you loo if you blow this up and look at it in more detail there 's something that comes from the discourse in terms of what was actually just said what 's the utterance go giving us professor b: Yeah . professor b: Yeah , well that , well , we 'll have to decide  how much of th where that goes .  it could be those are two separate things , it could be that the discourse gadget itself integrates em as which would be my guess that you 'd have to do see in order to do reference and stuff like that  you 've gotta have both the current discourse and the context to say I wanna go back there , grad a:   professor b: wow , what does that mean and  grad e:  -  grad a: Now . So grad e: But is th is this picture that 's emerging here just my wish that you have noticed already for symmetry or is it that we get for each each decision on the very bottom we sort of get the sub - E , sub - D , sub - U and maybe a sub - O " O " for " ontology "  meta node professor b: I don't know . grad e: could be professor b: This is this is getting into the thing I wanna talk about next , grad e: so this professor b: which is s if that 's true  how do we wanna combine those ? O or when it 's true ? grad e: but this eh w wou wou would be nice though that , you know , we only have at most four at the moment  arrows going f to each of the  bottom decisions . grad e: It 's too much ? professor b: Well i i it see i if it 's fou if it 's four things and each of them has four values it turns out to be a big CPT , it 's not s completely impossi  it 's it 's not beyond what the system could solve but it 's probably beyond what we could actually  write down . grad c: is that what that professor b: Yeah ,  it 's and I don't think it 's gonna g e I don't think it 'll get worse than that by the way , so le that 's a that 's a good grad d: Mmm yeah . grad e: But but four didn't we decide that all of these had true or false ? So is it 's four professor b:  for go there , but not f but not for the other one 's three values for endpoint already . grad d: Yeah ,  you need actually three to the five because  well  if if it has four inputs and then it itself has three values grad c: Right . grad e:  for endpoint ? No it 's it 's sh professor b: EV - it 's the EVA . grad d: No it still has three , professor b: Since ta they will still have three . professor b: Each so you 're   from each point of view you 're making the same decision . professor b: So from the point of view of the ob of the entity grad e: Want to view that , yeah yeah . grad e: yeah grad d: This and also , the other places where , like for example consider endpoint view , it has inputs coming from user budget , user thrift professor b: Right . S so we 're we 're gonna have to use some t care in the knowledge engineering to not have this explode . And in fact I think it doesn't in the sense that  Read it , you know actually with the underlying semantics and stuff I think it isn't like you have two hundred and fifty - six different  ways of of thinking about whether this user wants to go to some place . But  What I was gonna suggest next is maybe we wanna work on this a little longer but I do want to also talk about the thing that we started into now of  well it 's all fine to say all these arrows come into the si same place what rule of combination is used there . professor b: how do they affect it ? And belief - nets have their own beliefs about  what are good ways to do that . So is it it 's it 's clearer n clear enough what the issue is , grad d: Right . professor b: right ? So do we wanna switch that now or we wanna do some more of this ? grad e: R basically w we just need to sort of in order to get some closure on this figure out how we 're gonna get this picture sort of  completely messy . professor b: Well , here he here 's one of the things that that I th you sh you no , I don't know how easy it is to do this in the interface but you it would be great if you could actually just display at a given time  all the things that you pick up , you click on " endpoint " , OK and everything else fades grad e:   And I does anybody remember the GUI on this ? grad c:  d I would almost say the other way to do that would be to open u or make you know N - many belief - nets and then open them every time you wanted to look at a different one grad e:   grad c: vers cuz  grad e: It 's probably pretty easy do it to do it in HTML , just grad c: Yeah , but grad e:  grad d: HTML ? grad e: Yeah I have each of these thing each of the end belief - nets be be a page and then you click on the thing and then li consider that it 's respective , professor b: Yeah the well the b grad d: OK . grad e: but professor b: anyway so  it clear that even with this if we put in all the arrows nobody is gonna be able to read the diagram . professor b: Alright , so e we have to figure out some eh eh  basically display hack or something to do this because anyway I I let me consi suggest that 's a s not a first - order consideration , we have two first - order considerations which is what are the  influences A , A , and B how do they get combined mathematically , how do we display them is an issue , but  grad c: I don't , yeah I just don't think this has been designed to support something like that . Yeah , I  , it might soon , if this is gonna be used in a serious way like java base then it might soon be necessary to  start modifying it for our purposes . Yeah , and  I that seems like a perfectly feasible thing to get into , but  we have to know what we want first . OK , so why don't you tell us a little bit about decision nodes and what what the choices might be for these ? grad d: So Ah , sorry . So  recall the basic problem which is that  you have a belief - net and you have like a lot of different nodes all contributing to one node . Right ? So as we discussed specifying this kind of thing is a big pain and it 's so will take a long time to write down because for example if these S have three possibilities each and this has three possibilities then you know you have two hundred and forty - three possibilities which is already a lot of numbers to write down . So what  helps us in our situation is that these all have values in the same set , right ? These are all like saying EV or A , right ? So it 's not just a generalized situation like  basically we wanna just take a combination of we wanna view each of these as experts ea who are each of them is making a decision based on some factors and we wanna sort of combine their decisions and create you know ,  sorta weighted combination . So the problem is to specify the  so the conditional property of this given all those , right ? That 's the way belief - nets are defined , like each node given its parents , right ? So  that 's what we want , we want for example P of  let 's call this guy Y and let 's call these X - one , X - two XN , right . So we want probability that Y equals , you know , for example  E given that these guys are I 'll just refer to this as like X  hat or something ,  the co like all of them ? Given that for example the data says you know , A , V , A , E , or something right ? professor b: Yep . professor b: Alright , so  Is that  I yeah , I just wanna make sure everybody is with us before he goes on . professor b: It 's it 's cl e is is it clear what he wants to compute ? grad a:   So Basically  what we don't wanna do is to for every single combination of E and V and A and every single letter E , s give a number grad a:   What we wanna do is find some principled way of  saying what each of these is and we want it to be a valid probability distribution , so we want it to  add up to one , right ? grad a:  . So what  I guess , what Jerry suggested earlier was basically that we , you know view these guys as voting and we just take the  we essentially take  averages , right ? So for example here two people have voted for A , one has voted for V , and one has voted for E , so we could say that the probabilities are , you know , probability of being E is one over four , because one person voted for E out of four and similarly , probability of so this is probability of E s and then probability of A given all that is  two out of four and probability of V is one out of four . Right ? So that 's step that 's the  yeah that 's the that 's the basic  thing . grad d: Is that all OK ? grad e: And that one outcome , that 's professor b: What ? grad e: it 's X X - one voted for A X - two voted for V grad a:   professor b: S so this assumes symmetry and equal weights and all this sort of things , which may or may not be a good assumption , grad e: That 's the outcome . So we 've assumed equal weights whereas it might turn out that you know , some w be that for example , what the  the actual the  verbal content of what the person said , like what  what might be  somehow more  important than the  grad c: X - one matters more i than X - two or grad d: Right . Sure , so we don't wanna like give them all equal weight so currently we 've been giving them all weight one fourth so we could replace this by  W - one , W - two , W - three , and W - four grad a:  . grad d: right ? And in order for this to be a valid probability distribution for each  X - hat , we just need that the W 's sum to one . So they can be for example , you know you you could have point one , point three , point two , and point four , say . And  grad c: So I jus just to make sure I understand this , so in this case  we would still compute the average ? grad d: You 'd compute the weighted average , so the probability of E would be  grad c: OK , so so it 'd be so in this case the probability that Y equals A would be  W one times grad a: Point three . grad c: or A or let 's see , one full quarter times point one grad d: Not one quarter , grad a: No . grad d: so these numbers have been replaced with point one , point three , point two , and point four . So the next possibility is that  we 've given just a single weight to each expert , right , whereas it might be the case that  in certain situations one of the experts is more  reliable and in certain situations the other expert is more reliable . So the way this is handled is by what 's called a mixture of experts , so what you can have is you augment these diagrams like this so you have a new thing called " H " , OK ? This is a hidden variable . And what this is is it gets its input from X - one , X - two , X - three , and X - four , and what it does is it decides which of the experts is to be trusted in this particular situation . So what 's going on is that  this H node looks at these four values of those guys and it decides in given these values which of these isn't likely to be more reliable or most reliable . So H produces some you know , it produces a number , either one , two , three , or four , in our situation , right ? Now this guy he looks at the value of H say it 's two , and then he just selects the  thing . grad a: So so the function of the thing that comes out of H is very different from the function of the other inputs . grad c: So H passes a vector on to the next node ? grad d: It could . grad c: It could ? A vector of the weights as the se grad d: Yeah , it could grad c: oh . grad d: Sorry ? grad a: Well a vector with three zero 's and one one , right ? grad c: Oh it 's basically to tell the bottom node which one of the situations that it 's in or which one of the weighting systems grad d: Right , so  the way you desc grad c: W I was just , if you wanted to pay attention to more than one you could pass a w a weighting s system though too , couldn't you ? OK . grad a:  Does H have to have another input to tell it alpha , beta , whatever , or is the that 's determined by what the experts are saying , like the type of situ OK .  It it just seems that like without that that outside input that you 've got a situation where , you know , like if if  X - one says no , you know , a low value coming out of X - on or i if X - one says no then ignore X - one , you know ,  that seems like that 'd be weird , grad d: Yeah , well could be things like if X - two and X - three say yes then i ignore X - one also . grad c: Oh The situations that H has , are they built into the net or OK , so they they could either be hand coded or learned or OK . grad c: So you specify one of these things for every one of those possi possible situations .  Well ,  to learn them we need data , where are we gonna get data ? Well  we need data with people intentions , right ? grad a: Right , right . But what 's the data about like , are we able to get these nodes from the data ? grad a: Like how thrifty the user is , or do we have access to that ?   grad d: Yeah , but that 's my question , like how do we  , how do we have data about something like   endpoint sub - E , or endpoint sub  you know s S ? grad c: Well , basically you would say , based on in this dialogue that we have which one of the things that they said eh whether it was the entity relations or whatever was the thing that determined what mode it was , grad d: Mmm . I don't think , well you have a can you bring up the function thing ?  w where is the thing that allows you to sort of grad c: That 's on the added variable , isn't it ? grad d: Oh function properties , is that it ?  , I guess not . And  it so e either it 'll allow us to do everything which I think is unlikely , I think more likely it 'll allow us to do very few of these things and in that case we 'll have to  just write up little things that allow you to  create such CPU 's on your own in the java base format . Yeah , I was assuming that 's what we 'd always do because yeah I was assuming that 's what we 'd always do , it 's Right . Well in terms of java base I think it 's basically what you see is what you get in I don't yeah , I would be surprised if it supports anything more than what we have right here . By the way   just talking about  about that general end of things  is there gonna be data soon from what people say when they 're interacting with the system and so on ? Like , what kind of questions are being given being asked ? Cuz OK . I 'm just wondering , because in terms of , you know ,   w the figure I was thinking about this figure that we talked about , fifty constructions or whatever that 's  that 's a whole lot of constructions and  you know ,  one might be f fairly pleased with getting a really good analysis of five maybe ten in a summer so ,  I know we 're going for sort of a rough and ready .  , I I I I was  I was talking about the , you know , if you wanted to do it really in detail and we don't really need all the detail for what we 're doing right now but anyway in terms of just narrowing that task you know which fifty do I do , I wanna see what people are using , so Well , it will inspire me <doc-sep>phd a: Why is it so cold in here ? professor b: so , we haven't sent around the agenda . So , i  , any agenda items anybody has , wants to talk about , what 's going on ? postdoc g: I c I could talk about the meeting .  phd a: Well , I had a just a quick question but I know there was discussion of it at a previous meeting that I missed , but just about the the wish list item of getting good quality close - talking mikes on every speaker . professor b: OK , so let 's let 's So let 's just do agenda building right now . You were gonna starting to say something ? postdoc g: Well , you you , already know about the meeting that 's coming up and I don't know if if this is appropriate for this . phd e: What meeting ? professor b: We can so we can ta so n NIST is NIST folks are coming by next week postdoc g: OK . professor b: I think phd e: Who 's coming ? professor b: John Fiscus postdoc g: An - anything else , strike anybody ? phd a: we started running recognition on one conversation but it 's the r isn't working yet . phd e: Wha phd a: the main thing would be if anyone has , knowledge about ways to , post - process the wave forms that would give us better recognition , that would be helpful to know about . professor b: Yeah , so ,  phd e: What about , is there anything new with the speech , nonspeech stuff ? phd c: Yeah , we 're working more on it but , it 's not finished . postdoc g: I had thought under my topic that I would mention the , four items that I I , put out for being on the agenda f on that meeting , which includes like the pre - segmentation and the and the developments in multitrans . professor b: Alright , why don't we start off with this , u u I guess the order we brought them up seems fine . So the one issue was that the the , lapel mike , isn't as good as you would like . Right ? phd a: Ri -  , professor b: Is that is that basically the point ? phd a: yeah , the And actually in addition to that , that the the close talking mikes are worn in such a way as to best capture the signal . And the reason here is just that for the people doing work not on microphones but on sort of like dialogue and so forth ,  or and even on prosody , which Don is gonna be working on soon , it adds this extra , you know , vari variable for each speaker to to deal with when the microphones aren't similar . phd a: So And I also talked to Mari this morning and she also had a strong preference for doing that . And in fact she said that that 's useful for them to know in starting to collect their data too . Right , so one th grad h: Well , so professor b: well one thing I was gonna say was that , i we could get more , of the head mounted microphones even beyond the number of radio channels we have because I think whether it 's radio or wire is probably second - order . grad h: So it 's towards the corner of your mouth so that breath sounds don't get on it . grad h: And then just sort of about , a thumb or a thumb and a half away from your from your mouth . phd a: But we have more than one type of professor b: How am I d phd a: for instance , you 're phd c: Yeah . phd a: But if we could actually standardize , you know , the the microphones , as much as possible that would be really helpful . professor b: Well ,  it doesn't hurt to have a few extra microphones around , phd d: Yeah . professor b: so why don't we just go out and and get an order of of if this microphone seems OK to people , I 'd just get a half dozen of these things . grad h: Well the onl the only problem with that is right now , some of the Jimlets aren't working . grad h: And so , w  , I 've only been able to find three jacks that are working . phd e: Can we get these , wireless ? grad h: So professor b: No , but my point is phd a: But y we could just record these signals separately and time align them with the start of the meeting . professor b: R r right grad h: I I 'm not sure I 'm follow . Say that again ? professor b: Right now , we 've got , two microphones in the room , that are not quote - unquote standard . professor b: Also what we 've talked before about getting another , radio , grad h: Right . professor b: So ,  so we should go out to our full complement of whatever we can do , but have them all be the same mike . I think the original reason that it was done the other way was because , it w it was sort of an experimental thing and I don't think anybody knew whether people would rather have more variety or or , more uniformity , phd a: Right . phd a: Well , for short term research it 's just there 's just so much effort that would have to be done up front n  , professor b: Yeah . phd e: Is it because You you 're saying the for dialogue purposes , so that means that the transcribers are having trouble with those mikes ? Is that what you mean ? phd a: Well Jane would know more about the transcribers . postdoc g: a couple times , so , yeah , the transcribers notice And in fact there 're some where ,  ugh well ,  there 's it 's the double thing . postdoc g: And he 's always they always they just rave about how wonderful Adam 's Adam 's channel is .  , " Baaah ! " phd a: Even if if you 're talking on someone else 's mike it 's still you w postdoc g: Yeah , but  it 's not just that , it 's also you know you professor b: Yeah . postdoc g: It 's also like n no breathing , no You know , it 's like it 's it 's  , professor b: Yeah . postdoc g: it 's really it makes a big difference from the transcribers ' point of view grad h: Yeah , it 's an advantage when you don't breath . professor b: When we 're doing grad h: Yeah , I think that the point of doing the close talking mike is to get a good quality signal . professor b: Yeah , probably yeah , to the store we talked about and that grad h: Yep . postdoc g: And there was some talk about , maybe the h headphones that are uncomfortable for people , to grad h: Yep . So , as as I said , we 'll do a field trip and see if we can get all of the same mike that 's more comfortable than than these things , which I think are horrible . grad h: And , you know , we 're researchers , so we all have big heads . OK , so , Jonathan Fiscus is coming on the second of February and I 've spoken with , u u a lot of people here , not everyone .  , and , he expressed an interest in seeing the room and in , seeing a demonstration of the modified multitrans , which I 'll mention in a second , and also , he was interested in the pre - segmentation and then he 's also interested in the transcription conventions . postdoc g: And ,  So , it seems to me in terms of like , i i it wou You know , OK . So the room , it 's things like the audio and c and audi audio and acoustic acoustic properties of the room and how it how the recordings are done , and that kind of thing . OK , in terms of the multi - trans , well that that 's being modified by Dave Gelbart to , handle multi - channel recording . grad h: Oh , I should 've I was just thinking I should have invited him to this meeting . I I , @ @ didn't didn't see it , yesterday but I 'm going to see it today . And , that 's that will enable us to do nice  , tight time marking of the beginning and ending of overlapping segments . In terms of , like , pre - segmentation , that that continues to be , a terrific asset to the to the transcribers . What what I 'm doing right now is I 'm trying to include some information about which channel , there 's some speech in . I 'm just trying to do this by comparing energies ,  normalizing energies and comparing energies of the different channels . phd c: And so to to give the transcribers some information in which channel there 's there 's speech in addition to to the thing we we did now which is just , speech - nonspeech detection on the mixed file . So I 'm I 'm relying on on the segmentation of the mixed file postdoc g: This is good . phd c: but I 'm I 'm trying to subdivide the speech portions into different portions if there is some activity in in different channels . postdoc g: Excellent , so this 'd be like w e providing also speaker ID potentially . professor b: something I guess I didn't put in the list but , on that , same day later on in or maybe it 's No , actually it 's this week , Dave Gelbart and I will be , visiting with John Canny who i you know , is a CS professor , postdoc g: Oh . You know , maybe they 'd wanna stick an array mike here when we 're doing things phd e: That would be cool . professor b: or or maybe it 's it 's not a specific array microphone they want phd d: Yeah . professor b: but they might wanna just , you know , you could imagine them taking the four signals from these these table mikes and trying to do something with them  , I also had a discussion So , w  , we 'll be over over there talking with him , after class on Friday . I had a , discussion today with , Birger Kollmeier who 's a , a German , scientist who 's got a fair sized group doing a range of things . But but , he does stuff with auditory models and he 's very interested in directionality , and location , and and , head models and microphone things . And so , he 's he and possibly a student , there w there 's , a student of his who gave a talk here last year , may come here , in the fall for , sort of a five month , sabbatical . phd e: That that reminds me , I had a a thought of an interesting project that somebody could try to do with the data from here , either using , you know , the the mikes on the table or using signal energies from the head worn mikes , phd d:   phd e: and that is to try to construct a map of where people were sitting , professor b: Right . phd e: And so you could plot out who was sitting next to who professor b: A little bit , phd e: and professor b: he didn't do a very extreme thing but just it was just sort of phd d: Yeah , yeah . professor b: e e given that , the the the block of wood with the the the two mikes on either side , grad h:   professor b: if I 'm speaking , or if you 're speaking , or someone over there is speaking , it if you look at cross - correlation functions , you end up with a phd d: Yeah . professor b: if if someone who was on the axis between the two is talking , then you you get a big peak there . And if if someone 's talking on on on , one side or the other , it goes the other way . professor b: And then , it it it even looks different if th t if the two two people on either side are talking than if one in the middle . Well I was just thinking , you know , as I was sitting here next to Thilo that  , when he 's talking , my mike probably picks it up better than your guys 's mikes . phd e: So if you just looked at grad h: Oh , that 's another cl cue , phd d: Yeah . phd e: yeah , looked at the energy on my mike and you could get an idea about who 's closest to who . professor b: Yeah , well you have to the appropriate normalizations are tricky , and and and are probably the key . phd a: You just search for Adam 's voice on each individual microphone , you pretty much know where everybody 's sitting . postdoc g: Can I ask one thing ?  , so , Jonathan Fiscus expressed an interest in , microphone arrays . postdoc g: is there  b And I also want to say , his he can't stay all day . He needs to  , leave for  , from here to make a two forty - five flight grad h: Oh , so just morning . postdoc g: So it makes the scheduling a little bit tight but do you think that ,  that , i John Canny should be involved in this somehow or not . professor b: Probably not but I I 'll I 'll I 'll know better after I see him this Friday what what kind of level he wants to get involved . professor b: he might be excited to and it might be very appropriate for him to , or he might have no interest whatsoever . grad h: Is he involved in Ach ! I 'm blanking on the name of the project . NIST has has done a big meeting room instrumented meeting room with video and microphone arrays , and very elaborate software . Is is he the one working on that ? professor b: Well that 's what they 're starting up . professor b: well I think they 've instrumented a room but I don't think they they haven't started recordings yet . They don't have the phd e: Are they going to do video as well ? grad h:  . grad h: Oh , cuz what what I had read was , they had a  very large amount of software infrastructure for coordinating all this , both in terms of recording and also live room where you 're interacting the participants are interacting with the computer , and with the video , and lots of other stuff . professor b: All all I know is that they 've been talking to me about a project that they 're going to start up recording people meet in meetings . professor b: And one one notable difference u u actually I can't remember whether they were going to routinely collect video or not , but one one , difference from the audio side was that they are interested in using array mikes . The reason I didn't go for that here was because , the focus , both of my interest and of Adam 's interest was  , in impromptu situations . And we 're not recording a bunch of impromptu situations but that 's because it 's different to get data for research than to actually apply it . professor b: And so , for scientific reasons we thought it was good to instrument this room as we wanted it . But the thing we ultimately wanted to aim at was a situation where you were talking with , one or more other people i  , in in an p impromptu way , where you didn't didn't actually know what the situation was going to be . And therefore it would not it 'd be highly unlikely that room would be outfitted with with some very carefully designed array of microphones . It was just , you know , yet another piece of research and it seemed like we had enough troubles just phd e: So there 's no like portable array of mikes ? professor b: No . So there 's there 's  , there 's a whole range of things there 's a whole array of things , that people do on this . professor b: So , the ,  the big arrays , places , like  , Rutgers , and Brown , and other other places , they have , big arrays with , I don't know , a hundred hundred mikes or something . professor b: And it 's and , in fact at one point we had a a proposal in with Rutgers where we were gonna do some of the sort of per channel signal - processing and they were gonna do the multi - channel stuff , but it d it d we ended up not doing it . grad h: And then they have little ones too professor b: And then they had the little ones , yeah . grad h: but  but they don't have our block of wood , right ? professor b: Yeah , our block of wood is unique . professor b: But the But the No , there are these commercial things now you can buy that have four mikes or something phd a:   professor b: and and ,  So , yeah , there 's there 's there 's a range of things that people do . professor b: so if we connected up with somebody who was interested in doing that sort of thing that 's that 's a good thing to do .  , whenever I 've described this to other people who are interested on the with the acoustic side that 's invariably the question they ask . Just like someone who is interested in the general dialogue thing will always ask "  , are you recording video ? " phd a: Right , professor b: right ? phd a: right . professor b: And and the acoustic people will always say , " well are you doing , array microphones ? " So it 's it 's a good thing to do , but it doesn't solve the problem of how do you solve things when there 's one mike or at best two mikes in in this imagined PDA that we have .  , I know that having an array of  , I would imagine it would be more expensive to have a an array of microphones . But couldn't you kind of approximate the natural sis situation by just shutting off  , channels when you 're later on ?  , it seems like if the microphones don't effect each other then couldn't you just , you know , record them with an array and then just not use all the data ? grad h: It 's it 's just a lot of infrastructure that for our particular purpose we felt we didn't need to set up . professor b: Yeah , if ninety - nine percent of what you 're doing is c is shutting off most of the mikes , then going through the postdoc g: OK . professor b: But if you get somebody who 's who who has that as a primary interest then that put then that drives it in that direction . grad h: That 's right ,  if someone if someone came in and said we really want to do it , phd a: Right . That would be fine , phd e: So to save that data you You have to have one channel recording per mike in the array ? grad h: Buy more disk space . professor b: But then , you know , there 's it there 's phd e: What you save , if you 're going to do research with it . yeah professor b: There 's I I don't know what they 're going to do and I don't know how big their array is . Obviously if you were gonna save all of those channels for later research you 'd use up a lot of space . grad h: Well their software infrastructure had a very elaborate design for plugging in filters , and mixers , and all sorts of processing . But I think in practical situations you would have some engine of some sort doing some processing to reduce this to some to the equivalent of a single microphone that was very directional . professor b: Right ? phd a: it seems professor b: So phd e: Sort of saving the result of the beam - forming . phd a: it seems to me that there 's you know , there are good political reasons for for doing this , just getting the data , because there 's a number of sites like right now SRI is probably gonna invest a lot of internal funding into recording meetings also , which is good , but they 'll be recording with video and they 'll be You know , it 'd be nice if we can have at least , make use of the data that we 're recording as we go since it 's sort of this is the first site that has really collected these really impromptu meetings , and just have this other information available . So , if we can get the investment in just for the infra infrastructure and then , I don't know , save it out or have whoever 's interested save that data out , transfer it there , it 'd be g it 'd be good to have have the recording . grad h: You mean to to actually get a microphone array and do that ? phd a: Well , if Even if we 're not grad h: And video and phd a: I 'm not sure about video . That 's sort of an video has a little different nature since right n right now we 're all being recorded but we 're not being taped .  , but it definitely in the case of microphone arrays , since if there was a community interested in this , then grad h: Well , but I think we need a researcher here who 's interested in it . professor b: See the problem is it it took , it took at least six months for Dan to get together the hardware and the software , and debug stuff in in the microphones , and in the boxes . And so I think we could get a microphone array in here pretty easily and , have it mixed to to one channel of some sort . professor b: But , e I think for  , how we 're gonna decide For for maximum flexibility later you really don't want to end up with just one channel that 's pointed in the direction of the the the p the person with the maximum energy or something like that .  , you you want actually to you want actually to have multiple channels being recorded so that you can And to do that , it we 're going to end up greatly increasing the disk space that we use up , we also only have boards that will take up to sixteen channels and in this meeting , we 've got eight people and and six mikes . phd a: Well if there 's a way to say time to sort of solve each of these f those professor b: Yeah . phd a: So suppose you can get an array in because there 's some person at Berkeley who 's interested and has some equipment , and suppose we can as we save it we can , you know , transfer it off to some other place that that holds this this data , who 's interested , and even if ICSI it itself isn't .  , and it it seems like as long as we can time align the beginning , do we need to mix it with the rest ? I don't know . phd a:  it 's just it 's worth considering as sort of grad h: And y it 'd certainly gets skew . phd a: once you make the up front investment and can sort of save it out each time , and and not have to worry about the disk space factor , then it mi it might be worth having the data . I mentioned that , b as a practical matter , grad h: Just professor b: but the real issue is that , there is no way to do a recording extended to what we have now with low skew . professor b: which would mean that the sampling times and so forth would be all over the place compared to this . So it would depend on the level of pr processing you were doing later , but if you 're d i the kind of person who 's doing array processing you actually care about funny little times . And and so you actually wou would want to have a completely different set up than we have , phd a: I see . But what we could do is if there was someone else who 's interested they could have a separate set up which they wouldn't be trying to synch with ours which might be useful for for them . phd a: Right ,  at least they 'd have the data and the transcripts , professor b: And then we can offer up the room , phd a: and Right . professor b: Yeah , we can o offer the meetings , and the physical space , and and yeah , the transcripts , and so on . phd a: But it 's if it 's impossible or if it 's a lot of effort then you have to just balance the two , professor b: Well I thi phd a: so professor b: yeah , the thing will be , u u in in again , in talking to these other people to see what you know , what what we can do . phd e: Is there an interest in getting video recordings for these meetings ?  professor b: Right , so we have we phd e:  grad h: Yes , absolutely . But it 's exactly the same problem , that you have an infrastructure problem , you have a problem with people not wanting to be video taped , and you have the problem that no one who 's currently involved in the project is really hot to do it . Internally , but I know there is interest from other places that are interested in looking at meeting data and having the video . So it 's just postdoc g: Yeah , w although I m I I have to u u mention the human subjects problems , that i increase with video . There 's the fact that then  , if i I I 've heard comments about this before , " why don't you just put on a video camera ? " But you know , it 's sort of like saying , "  , well we 're primarily interested in in some dialogue things , but , why don't we just throw a microphone out there . "  , the thing is , once you actually have serious interest in any of these things then you actually have to put a lot of effort in . professor b: So I think NIST or LDC , or somebody like that I think is much better shape to do all that . postdoc g: I it it occurred to me , has Don signed a human subject 's form ? grad h: Oh ! Probably not . postdoc g: A permission form ? grad h: Has Don have you s did you si I thought you did actually . grad h: Didn't you read a digit string ? phd e: You were here at a meeting before . postdoc g: Did you sign a form ? grad f: Did I ? I don't know . professor b: Yeah , we we postdoc g: But I just grad f: Can I verbally consent ? postdoc g: you know . You 're on recor you 're being recorded postdoc g: o grad f: Yeah . phd a: and professor b: we don't we don't perform electro - shock during these meetings , grad f: I don't care . phd e: How big is the data set ? postdoc g: Oh , it 's what  is one meeting . I didn't want to have any conflicts of , you know , of of when to stop transcribing this one or So I wanted to keep it clear whose data were whose , and and and so phd e:  - huh . postdoc g: And , meetings , you know , I think that they 're they go as long as a almost two hours in some in some cases . So , you know , that means you know , if we 've got two already finished and they 're working on  , right now all eight of them have differe  , additional data sets . postdoc g: And , also Dan Ellis 's innovation of the ,  the multi - channel to here really helped a r a lot in terms of clearing clearing up h hearings that involve overlaps . But , just out of curiosity I asked one of them how long it was taking her , one of these two who has already finished her data set .  , these still , when they 're finished , that means that they 're finished with their pass through . They still need to be edited and all but But it 's word level , speaker change , the things that were mentioned . OK , now I wanted to mention the , teleconference I had with , Jonathan Fiscus . postdoc g: He , he in indicated to me that they 've that he 's been , looking , spending a lot of time with I 'm not quite sure the connection , but spending a lot of time with the ATLAS system . But it looks to me like that 's the name that has developed for the system that Bird and Liberman developed for the annotated graphs approach . postdoc g: So what he wants me to do and what we what we will do and  , is to provide them with the u already transcribed meeting for him to be able to experiment with in this ATLAS System . And they do have some sort of software , at least that 's my impression , related to ATLAS and that he wants to experiment with taking our data and putting them in that format , and see how that works out . I I I explained to him in in detail the , conventions that we 're using here in this in this word level transcript . And , you know , I I explained , you know , the reasons that that we were not coding more elaborately and and the focus on reliability . He 's he 's very  , independently he asked , " well what about reliability ? " So , he 's interested in the consistency of the encoding and that sort of thing . OK ,  phd a: Sorry , can you explain what the ATLAS I 'm not familiar with this ATLAS system . postdoc g: Well , you know , at this point I think  , well Adam 's read more in more detail than I have on this . But , there there is a way of viewing  , whenever you have coding categories , and you 're dealing with  , a taxonomy , then you can have branches that that have alternative , choices that you could use for each each of them . grad h: Is is Is ATLAS the his annotated transcription graph stuff ? I don't remember the acronym . The the one the what I think you 're referring to , they they have this concept of an an annotated transcription graph representation . grad h: And that 's basically what I based the format that I did I based it on their work almost directly , in combination with the TEI stuff . And so it 's it 's a data representation and a set of tools for manipulating transcription graphs of various types . phd e: Is this the project that 's sort of , between , NIST and and , a couple of other places ? postdoc g:   And I looked through them and they mainly had to do with this , this , tree structure , annotated tree diagram thing . postdoc g: So ,  and , you know , in terms of like the conventions that I 'm a that I 've adopted , it there there 's no conflict at all . And , " oh , and how 'd you handle this ? " And I said , " well , you know , this way " and And and we had a really nice conversation .  , OK , now I also wanted to say in a different a different direction is , Brian Kingsbury . I told him he could SSH on and use multi - trans , and have a look at the already done , transcription . And what he said was that , what they 'll be providing is will not be as fine grained in terms of the time information . And , that 's ,  You know , I need to get back to him and and , you know , explore that a little bit more and see what they 'll be giving us in specific , phd a:  . phd e: The p the people postdoc g: but I just haven't had time yet . phd e: The the folks that they 're , subcontracting out the transcription to , are they like court reporters postdoc g: Sorry , what ? Yes . phd e: or postdoc g: Apparently Well , I get the sense they 're kind of like that . Like it 's like a pool of of somewhat  , secretarial I don't think that they 're court reporters . Like medical transcriptionist type people grad h: Nu - it 's mostly it 's for their speech recognition products , phd e: But aren't they 're postdoc g: Yep . grad h: Well they they do send it out but my understanding is that that 's all this company does is transcriptions for IBM for their speech product . grad h: So most of it 's ViaVoice , people reading their training material for that . postdoc g: Up to now it 's been monologues , as far my understood . postdoc g: Brian himself downloaded So So , Adam sent them a CD and Brian himself downloaded  , cuz , you know , we wanted to have it so that they were in familiar f terms with what they wanted to do . postdoc g: And then what he 's going to do is check it , a before they go be beyond the first one . professor b: So if they hear something off in the distance they don't they just go phd e: OK . grad h: Well , but that 's OK , because , you know , you 'll do all them and then combine them . phd e: Well if you 're tran if you got that channel right there postdoc g: d  , in my case phd c: Yeah . postdoc g: Yeah , I I would think that it would be kind of hard to come out with Yeah . phd a: I I think it 's sort of hard just playing the you know , just having played the individual files . phd a: there are a lot of words that are so reduced phonetically that make sense when you know what the person was saying before . phd e: Yeah , that 's phd a: it sort of depends where you are in postdoc g: And especially since a lot of these phd d: Yeah . grad h: And the answer is we don't actually know the answer because we haven't tried both ways . postdoc g: Well , except I can say that my transcribers use the mixed signal mostly grad h: So . postdoc g: unless there 's a huge disparity in terms of the volume on on the mix . In which case , you know , they they wouldn't be able to catch anything except the prominent channel , grad h: Right . postdoc g: Yeah , well phd a: Actually , are th so are they giving any time markings ? grad h: Right . phd a: In other words , if postdoc g: Well , I have to ask him . postdoc g: But but the ,  I did want to say that it 's hard to follow one channel of a conversation even if you know the people , and if you 're dealing furthermore with highly abstract network concepts you 've never heard of So , you know , one of these people was was transcribing the , networks group talk and she said , " I don't really know what a lot of these abbreviations are , " " but I just put them in parentheses cuz that 's the that 's the convention and I just " Cuz you know , if you don't know grad h: Oh , I 'd be curious to to look at that . grad h: The networks group meetings are all phd e: Given all of the effort that is going on here in transcribing why do we have I B M doing it ? Why not just do it all ourselves ? professor b: it 's historical .  , some point ago we thought that  , it " boy , we 'd really have to ramp up to do that " , phd c:  - huh . phd d: No , just professor b: you know , like we just did , and , here 's , a a , collaborating institution that 's volunteered to do it . professor b: And it still might be a good thing phd e: I 'm just wondering now phd a: Actu yeah , Mar - Mari asked me the same question as sort of professor b: but phd e: Well , I 'm I 'm wondering now if it 's grad h: Well we can talk about more details later . Let 's see what comes out of it , and and , you know , have some more discussions with them . It 's very a real benefit having Brian involved because of his knowledge of what the how the data need to be used and so what 's useful to have in the format . grad h: So , Liz , with with the SRI recognizer , can it make use of some time marks ? phd a: OK , so this is a , grad h: I I guess I don't know what that means . phd a: and actually I should say this is what Don has b  , he 's already been really helpful in , chopping up these So so first of all you  , for the SRI front - end , we really need to chop things up into pieces that are f not too huge .  , but second of all ,  in general because some of these channels , I 'd say , like , I don't know , at least half of them probably on average are g are ha are have a lot of cross - ta sorry , some of the segments have a lot of cross - talk .  , it 's good to get sort of short segments if you 're gonna do recognition , especially forced alignment . So , Don has been taking a first stab actually using Jane 's first the fir the meeting that Jane transcribed which we did have some problems with , and Thilo , I think told me why this was , but that people were switching microphones around in the very beginning , so the SRI re phd c: No , th Yeah . They they were not switching them but what they were they were adjusting them , phd a: and they They were not phd c: so . phd a: So we have to sort of normalize the front - end and so forth , and have these small segments . phd a: So we 've taken that and chopped it into pieces based always on your your , cuts that you made on the mixed signal . Right ? phd a: the problem is if we have no time marks , then for forced alignment we actually don't know where you know , in the signal the transcriber heard that word . And so grad h: Oh , I see , phd a: if if it 's a whole conversation and we get a long , you know , par paragraph of of talk , grad h: it 's for the length . phd a: And , I think with phd e: Well you would need to like a forced alignment before you did the chopping , right ? phd a: No , we used the fact that So when Jane transcribes them the way she has transcribers doing this , whether it 's with the pre - segmentation or not , grad h: It 's already chunked . And maybe they choose the chunk or now they use a pre - segmentation and then correct it if necessary . That 's great , cuz the recognizer can grad h: it 's all pretty good sized for the recognizer also . phd a: Right , and it it helps that it 's made based on sort of heuristics and human ear I think . phd a: Th - but there 's going to be a real problem , even if we chop up based on speech silence these , the transcripts from I B M , we don't actually know where the words were , which segment they belonged to . phd e: Why not do a a a forced alignment ? grad h: That 's what she 's saying , is that you can't . phd a: If you do a forced alignment on something really grad h: Got  six sixty minutes of phd a: well even if you do it on something really long you need to know you can always chop it up but you need to have a reference of which words went with which , chop . postdoc g: Now wasn't I thought that one of the proposals was that IBM was going to do an initial forced alignment , phd a: So postdoc g: after they grad h: Yeah , but professor b: I I think that they are , grad h: We 'll have to talk to Brian . professor b: yeah , I 'm sure they will and so we we have to have a dialogue with them about it . professor b: it sounds like Liz has some concerns phd a: Maybe they have some you know , maybe actually there is some , even if they 're not fine grained , maybe the transcribers professor b: and phd a: I don't know , maybe it 's saved out in pieces or or something . postdoc g: I just you know , it 's like I got over - taxed with the timing . But the it is true that the segments I haven't tried the segments that Thilo gave you but the segments that in your first meeting are great . phd a: Right , cuz postdoc g: Well , I I was thinking it would be fun to to  , if if you wouldn't mind , to give us a pre - segmentation . postdoc g: maybe you have one already of that first m of the meeting that  , the first transcribed meeting , the one that I transcribed . phd c: I 'm sure I have some postdoc g: Do you have a could you generate a pre - segmentation ? grad h: February sixteenth I think . phd c: but but that 's the one where we 're , trai training on , so that 's a little bit grad h: Oh . phd c: It 's a little bit at odd to postdoc g: Oh , darn . phd a: And actually as you get transcripts just , for new meetings , we can try postdoc g:  - huh . So it 'd be good for just to know as transcriptions are coming through the pipeline from the transcribers , just to sort of we 're playing around with sort of  , parameters f on the recognizer , postdoc g:   phd a: The first meeting had I think just four people , phd c: Four speakers , yeah . postdoc g: Yeah , Liz and I spoke d w at some length on Tuesday and and I and I was planning to do just a a preliminary look over of the two that are finished and then give them to you . I guess the other thing , I I can't remember if we discussed this in the meeting but , I know you and I talked about this a little bit , there was an issue of , suppose we get in the , I guess it 's enviable position although maybe it 's just saying where the weak link is in the chain , where we we , we have all the data transcribed and we have these transcribers and we were we 're the we 're still a bit slow on feeding at that point we 've caught up and the the the , the weak link is is recording meetings . OK , two questions come , is you know what how how do we  , it 's not really a problem at the moment cuz we haven't reached that point but how do we step out the recorded meetings ? And the other one is , is there some good use that we can make of the transcribers to do other things ? So , I I can't remember how much we talked about this in this meeting but there was grad h: We had spoken with them about it . postdoc g: And there is one use that that also we discussed which was when , Dave finishes the and maybe it 's already finished the the modification to multi - trans which will allow fine grained encoding of overlaps .  , then it would be very these people would be very good to shift over to finer grain encoding of overlaps . It 's just a matter of , you know , providing So if right now you have two overlapping segments in the same time bin , well with with the improvement in the database in in the , sorry , in the interface , it 'd be possible to , you know , just do a click and drag thing , and get the  , the specific place of each of those , the time tag associated with the beginning and end of of each segment . professor b: One one was  , we had s had some discussion in the past about some very high level labelings , postdoc g: Yeah . The types of overlaps professor b: types of overlaps , and so forth that that someone could do . And the third one is is , just a completely wild hair brained idea that I have which is that , if ,  if we have time and people are able to do it , to take some subset of the data and do some very fine grained analysis of the speech . For instance , marking in some overlapping potentially overlapping fashion , the value of , ar articulatory features . professor b: You know , just sort of say , OK , it 's voiced from here to here , there 's it 's nasal from here to here , and so forth .  , as opposed to doing phonetic  , you know , phonemic and the phonetic analysis , grad h:  professor b: and , assuming , articulatory feature values for those those things . postdoc g: Also if you 're dealing with consonants that would be easier than vowels , wouldn't it ?  , I would think that that , being able to code that there 's a a fricative extending from here to here would be a lot easier than classifying precisely which vowel that was . professor b: but I think also it 's just the issue that that when you look at the u w u u when you look at Switchboard for instance very close up there are places where whether it 's a consonant or a vowel you still have trouble calling it a particular phone postdoc g:   grad h: Yeah , but but just saying what the professor b: because it 's you know , there 's this movement from here to here postdoc g: Yeah , I 'm sure . professor b: and and and it 's so I phd e: You 're saying r sort of remove the high level constraints and go bottom - up . Maybe there 's there 's even a better way to do it but it but but that 's , you know , sort of a traditional way of describing these things , phd e:   professor b: and  , actually this might be a g neat thing to talk to postdoc g: That 's nice . professor b: some sort of categories but but something that allows for overlapping change of these things and then this would give some more ground work for people who were building statistical models that allowed for overlapping changes , different timing changes as opposed to just " click , you 're now in this state , which corresponds to this speech sound " and so on . phd a: So this is like gestural  , these g professor b: Yeah , something like that . professor b: actually if we get into that it might be good to , haul John Ohala into this phd a: Right . phd a: But is is the goal there to have this on meeting data , postdoc g: Excellent . phd a: like so that you can do far field studies of those gestures or  , or is it because you think there 's a different kind of actual production in meetings that people use ? Or ? professor b: No , I think I think it 's for for for that purpose I 'm just viewing meetings as being a a neat way to get people talking naturally . And then you have i and then and then it 's natural in all senses , phd e: Just a source of data ? phd a: I see . professor b: in the sense that you have microphones that are at a distance that you know , one might have , and you have the close mikes , and you have people talking naturally . And the overlap is just indicative of the fact that people are talking naturally , phd a:  - huh . professor b: right ? So so I think that given that it 's that kind of corpus , phd d: Yeah . professor b: if it 's gonna be a very useful corpus  , if you say w OK , we 've limited the use by some of our , censored choices , we don't have the video , we don't and so forth , but there 's a lot of use that we could make of it by expanding the annotation choices . professor b: And , most of the things we 've talked about have been fairly high level , and being kind of a bottom - up person I thought maybe we 'd , do some of the others . professor b: people have made a lot of use of of TIMIT and ,  w due to its markings , and then the Switchboard transcription thing , well I think has been very useful for a lot of people . professor b: So phd a: I guess I wanted to , sort of make a pitch for trying to collect more meetings . phd a: I actually I talked to Chuck Fillmore and I think they 've what , vehemently said no before but this time he wasn't vehement and he said you know , " well , Liz , come to the meeting tomorrow professor b: Yeah . Go to their meeting tomorrow and see if we can try , to convince them postdoc g:   professor b: Cuz they have something like three or four different meetings , phd a: because they have And they have very interesting meetings from the point of view of a very different type of of talk than we have here professor b: right ? postdoc g:    phd e: You mean in terms of the topic topics ? phd a: Well , yes and in terms of the the fact that they 're describing abstract things and , just dialogue - wise , professor b:   And then the other thing is , I don't know if this is at all useful , but I asked Lila if I can maybe go around and talk to the different departments in this building to see if there 's any groups that , for a free lunch , professor b: Yes . grad h: You mean non - ICSI ? phd a: non - ICSI , non - academic , grad h: Yeah , I guess you you can try phd a: you know , like government people , grad h: but phd a: I don't know .  , it seems like we we had this idea before of having like linguistics students brought down for free lunches grad h: Well , tha I think that 's her point . phd a: Right , and then we could also we might try advertising again because I think it 'd be good if if we can get a few different sort of non - internal types of meetings postdoc g: Yeah . phd e: Does does John Ohala have weekly phonetics lab meetings ? postdoc g:   grad h: And I think , if we could get phd a: So I actually wrote to him and he answered , " great , that sounds really interesting " . grad h: But , it would be nice if we got someone other than me who knew how to set it up and could do the recording phd a: So grad h: so u I didn't have to do it each time . phd a: and I was thinking professor b: He - he 's supposed he 's supposed to be trained to do it .  , the other thing is that there was a number of things at the transcription side that , transcribers can do , like dialogue act tagging , grad h: It 's not that hard . phd a: disfluency tagging , things that are in the speech that are actually something we 're y working on for language modeling . So if you wanna process a utterance and the first thing they say is , " well " , and that " well " is coded as some kind of interrupt u tag .  , and things like that , th postdoc g: Of course some of that can be li done lexically . phd a: A lot of it can be done postdoc g: And I also they are doing disfluency tagging to some degree already . phd a: I think there 's a second pass and I don't really know what would exist in it . But there 's definitely a second pass worth doing to maybe encode some kinds of , you know , is it a question or not , postdoc g:   postdoc g: I wanted to whi while we 're  , so , to return just briefly to this question of more meeting data ,  I have two questions . One of them is , Jerry Feldman 's group , they they , are they I know that they recorded one meeting . professor b: I think there 's we should go beyond , ICSI but , there 's a lot of stuff happening at ICSI that we 're not getting now that we could . So th there was the thing in Fillmore 's group but even there he hadn't What he 'd said " no " to was for the main meeting . phd e: Well , and and the other thing too is when they originally said " no " they didn't know about this post - editing capability thing . professor b: there 's there 's , the networks group , I don't Do they still meeting regularly or ? grad h: Well , I don't know if they meet regularly or not but they are no longer recording . professor b: But  , ha ha have they said they don't want to anymore or ? grad h: ugh , what was his name ? professor b: i i postdoc g: Joe Sokol ? grad h: Yeah . professor b: OK , so they 're down to three or four people grad h: They and they stopped Yeah . postdoc g: We might be able to get the administration grad h: Well he was sort of my contact , so I just need to find out who 's running it now . phd a: Yeah , it One thing that would be nice postdoc g: I don't know phd a: and this it sounds bizarre but , I 'd really like to look at to get some meetings where there 's a little bit of heated discussion , like ar arguments and or emotion , and things like that . Some group , " yes , we must " grad h: Who 's willing to get recorded and distributed ? phd a: Well , you know , something phd c: Yeah . grad f: Yeah , I don't think the more political argumentative ones would be willing to phd a:  professor b: Yeah , with with with potential use from the defense department . phd a: No , but maybe stu student , groups or , film - makers , or som Something a little bit colorful . Yeah , th there 's a problem there in terms of , the  commercial value of of st  , postdoc g: Yeah , of course there is this problem though , that if we give them the chance to excise later we e might end up with like five minutes out of a f of m one hour phd d: Film - maker . phd a: And I don't mean that they 're angry phd d: Is postdoc g: of Yes . phd a: but just something with some more variation in prosodic contours and so forth would be neat . So if anyone has ideas , I 'm willing to do the leg work to go try to talk to people but I don't really know which groups are worth pursuing . postdoc g: Well there was this K P F A grad h: No that 's postdoc g: but OK . phd a: Or postdoc g: And I had one other one other aspect of this which is , Jonathan Fiscus expressed primar  y a major interest in having meetings which were all English speakers . phd e: Did he mean ,  did he mean and non - British ? grad h: Well phd c: The all native . phd e: He said British was OK ? postdoc g: But but Sure , sure , sure . professor b: Why ? grad h: British is English ? phd c:  postdoc g: Yeah . professor b: Well , I don't I don't I don't think if he didn't say that postdoc g: Native speaking . grad h: So , why would he care ? phd e: Knowing the application phd a: That 's professor b: I remember wh I I remember a study phd a: I was thinking , knowing the , n National Institute of Standards , it is all professor b: I remember a study that BBN did where they trained on this was in Wall Street Journal days or something , they trained on American English and then they tested on , different native speakers from different areas . professor b: it was Swiss w Yeah , so it 's so I think , you know , if he 's if he 's thinking in terms of recognition kind of technology I I I think he would probably want ,  American English , postdoc g: All America , OK . It it yeah , unless we 're gonna train with a whole bunch of postdoc g: I think that the Feldman 's meetings tend to be more that way , aren't they ?  , I sort of feel like they have professor b: I think so , grad h: Maybe . grad h: And maybe there are a few of with us where it was professor b: Yeah . grad h: you know , Dan wasn't there and before Jose started coming , professor b: Yeah . professor b: So , what about what about people who involved in some artistic endeavor ? phd d: Yeah . phd a: Exactly , that 's what I was professor b: You 'd think like they would be phd d: A film - maker . phd a: something where there there is actually discussion where there 's no right or wrong answer but but it 's a matter of opinion kind of thing . phd a: Yeah , we could phd e: A any department that calls itself science phd d: Department . phd d: Computer sci grad h: That postdoc g: We could get Julia Child . phd a: I 'm I 'm actually serious grad h: That 's phd a: because , you know , we have the set up here grad h: Got a ticket . grad f: I know grad h: I could phd a: Not not professor b: Put a little ad up saying , " come here and argue " . grad h: and they they they expressed willingness back when they thought they would be doing more stuff with speech . grad h: But when they lost interest in speech they also stopped answering my email about other stuff , so . phd a: Or people who are really h professor b: They could have a discussion about te grad f: I grad h: We should probably bleep that out . grad f: I heard that at Cal Tech they have a special room someone said that they had a special room to get all your frustrations out that you can go to and like throw things and break things . professor b: Yeah , now that is not actually what we grad f: So we can like post a grad h: Th - that 's not what we want . grad f: No , not to that extent phd a: Well , far field mikes can pick up where they threw stuff on the wall . professor b: Yeah , but we don't want them to throw the far field mikes is the thing . postdoc g: It 'd be fun to get like a a p visit from the grad h: There was a dorm room at Tech that , someone had coated the walls and the ceiling , and , the floor with mattresses . professor b: What did we mean by that ? Remember @ @ ? grad h: Liz wanted to talk about methods of improving accuracy by doing pre - processing . phd a: Well I think that that was just sort of I I already asked Thilo professor b: Oh , you already did that . phd a: but that , it would be helpful if I can stay in the loop somehow with , people who are doing any kind of post - processing , whether it 's to separate speakers or to improve the signal - to - noise ratio , or both , that we can sort of try out as we 're running recognition .  , so , i is that Who else is work I guess Dan Ellis and you phd c: Dan , yeah . professor b: he 's he 's interested in in fact we 're look starting to look at some echo cancellation kind of things . professor b: Which  grad h: I am not sure how much that 's an issue with the close talking mikes , professor b:  ? grad h: but who knows ? professor b: Well , let 's w i isn't that what what you want phd a: I don't know . I 'm bad professor b: t No , so No , i w wha what you what you want when you 're saying improving the wave form you want the close talking microphone to be better . professor b: And the question is to w to what extent is it getting hurt by ,  by any room acoustics or is it just  , given that it 's close it 's not a problem ? phd a: It doesn't seem like big room acoustics problems to my ear professor b:  phd a: but I 'm not an expert . grad h: e I bet with the lapel mike there 's plenty , room acoustic phd a: That that may be true . phd a: But I don't know how good it can get either by those the those methods grad h: Yeah . grad h: So I I think it 's just , phd a: Oh , I don't know . phd a: All I meant is just that as sort of as this pipeline of research is going on we 're also experimenting with different ASR , techniques . phd e: So the problem is like , on the microphone of somebody who 's not talking they 're picking up signals from other people and that 's causing problems ? phd a: R right , although if they 're not talking , using the the inhouse transcriptions , were sort of O K because the t no one transcribed any words there and we throw it out . phd a: But if they 're talking at all and they 're not talking the whole time , so you get some speech and then a "  -  " , and some more speech , so that whole thing is one chunk . And the person in the middle who said only a little bit is picking up the speech around it , that 's where it 's a big problem . postdoc g: You know , this does like seem like it would relate to some of what Jose 's been working on as well , the encoding of the phd d: Yeah . postdoc g: And and he also , he was phd a: The energy , phd d: Yeah , phd a: right . postdoc g: I was t I was trying to remember , you have this interface where you i you ha you showed us one time on your laptop that you you had different visual displays as speech and nonspeech events . Because , eh , it 's possible , eh , eh , in a simp sample view , to , nnn , to compare with c with the segment , the the kind of assessment what happened with the the different parameters . And only with a different bands of color for the , few situation , eh , I consider for acoustic event is enough to @ @ . phd d: I I I see that , eh , you are considering now , eh , a very sophisticated , eh , ehm , eh , @ @ set of , eh , graphic s eh , eh , ehm , si symbols to to transcribe . No ? Because , before , you you are talking about the the possibility to include in the Transcriber program eh , a set of symbols , of graphic symbol to t to mark the different situations during the transcription postdoc g: Oh , I w  - huh . No ? postdoc g: Well , you 're saying So , symbols for differences between laugh , and sigh , and and and slam the door and stuff ? phd d: Yeah . postdoc g: Or some other kind of thing ? phd d: No ? To to mark postdoc g: Well , I wouldn't say symbols so much . The the main change that I that I see in the interface is is just that we 'll be able to more finely c  , time things . postdoc g: But I I also st there was another aspect of your work that I was thinking about when I was talking to you phd a:  . postdoc g: which is that it sounded to me , Liz , as though you and , maybe I didn't q understand this , but it sounded to me as though part of the analysis that you 're doing involves taking segments which are of a particular type and putting them together . postdoc g: And th so if you have like a p a s you know , speech from one speaker , then you cut out the part that 's not that speaker , phd a:   postdoc g: and you combine segments from that same speaker to and run them through the recognizer . Is that right ? phd a: Well we try to find as close of start and end time of as we can to the speech from an individual speaker , postdoc g:   phd a: because then we we 're more guaranteed that the recognizer will for the forced alignment which is just to give us the time boundaries , because from those time boundaries then the plan is to compute prosodic features . phd a: And the sort of more space you have that isn't the thing you 're trying to align the more errors we have .  , so , you know , that that it would help to have either pre - processing of a signal that creates very good signal - to - noise ratio , postdoc g:   phd a: which I don't know how possible this is for the lapel , or to have very to have closer , time you know , synch times , basically , around the speech that gets transcribed in it , or both . So I just wanted to see , you know , on the transcribing end from here things look good . And then the issue of like global processing of some signal and then , you know , before we chop it up is is yet another way we can improve things in that . You can ,   The problem is just that the acoustic when the signal - to - noise ratio is too low , you you 'll get , a  an alignment with the wrong duration pattern or it phd e: Oh , so that 's the problem , is the the signal - to - noise ratio . It 's not the fact that you have like  , what he did is allow you to have , words that were in another segment move over to the at the edges of of segmentations . phd a: Right , things things near the boundaries where if you got your alignment wrong phd e:   It 's that there are problems even in inside the alignments , because of the fact that there 's enough acoustic signal there t for the recognizer to to eat , as part of a word . So , yeah , bottom bottom line is just I wanted to make sure I can be aware of whoever 's working on these signal - processing techniques for , detecting energies , phd d: Yeah . professor b: O K ,  tea has started out there I suggest we c run through our digits and , postdoc g: OK <doc-sep>grad d: and  pick out the ones that have problems , and either correct them or have them re - read . So we probably have like four or five more forms to be read , to be once through the set . I extracted out about an hour 's worth which are the f digits with for which whose speaker have speaker forms , have filled out speaker forms . So I extracted one for speakers who have speaker forms and for meetings in which the " key " file and the transcript files are parsable . Some of the early key files , it looks like , were done by hand , and so they 're not automatically parsable and I have to go back and fix those .  , Liz professor f: So you think two you think two hours is the is the total that we have ? grad d: Yep , yeah . professor f: And you think we th  , I I didn't quite catch all these different things that are not quite right , but you think we 'll be able to retrieve the other hour , reasonably ? grad d: Yes , absolutely . grad d: So it 's just a question of a little hand - editing of some files and then waiting for more people to turn in their speaker forms . I have this web - based speaker form , and I sent mail to everyone who hadn't filled out a speaker form , and they 're slowly s trickling in . professor f: So the relevance of the speaker form here , s grad d: It 's for labeling the extracted audio files . grad d: No , I spoke with Jane about that and we sort of decided that it 's probably not an issue that We edit out any of the errors anyway . So the other topic with digits is  , Liz would like to elicit different prosodics , and so we tried last week with them written out in English . So in conversations with Liz and  Jane we decided that if you wrote them out as numbers instead of words it would elicit more phone number , social security number - like readings . When I look at this , that first line is " sixty one , sixty two , eighteen , eighty six , ten . "  , and so the question is does anyone care ?  , I 've already spoken with Liz and she feels that , correct me if I 'm wrong , that for her , connected numbers is fine , postdoc e: I think two hours is probably fine for a test set , but it may be a little short if we actually wanna do training and adaptation and all that other stuff . professor f: Yeah  , do  you want different prosodics , so if you always had the same groupings you wouldn't like that ? Is that correct ? phd g: Well , we actually figured out a way to grad d: Yeah , the the phd g: the the groupings are randomly generated . professor f: No but , I was asking if that was something you really cared about because if it wasn't , it seems to me if you made it really specifically telephone groupings that maybe people wouldn't , go and do numbers so much . You know if it if it 's phd a:  phd g: I think they may still do it , professor f: Maybe some , but I probably not so much . phd b: What about putting a hyphen between the numbers in the group ? phd g: And professor f: Right ? So if you if if you have  grad d: Six dash one , you mean ? professor f: if you go six six six  dash  two nine three one . phd g: I well OK I it might help , I would like to g get away from having only one specific grouping . phd g: but  it seems to me that , at least for us , we can learn to read them as digits postdoc e: Yeah . phd g: and it seems like that might be better for you guys since then you 'll have just more digit data , grad d: Right . phd g: It 's a little bit better for me too because the digits are easier to recognize . Right , read them as single digits , so sixty - one w is read as six one , postdoc e:   phd g: and if people make a mistake we grad d: How about " O " versus " zero " ? professor f: the other thing is we could just bag it because it 's it 's it 's - I 'm not worrying about it  , because we do have digits training data that we have from  from OGI . I 'm sorry , digits numbers training that we have from OGI , we 've done lots and lots of studies with that . phd g: But it 's nice to get it in this room with the acous professor f: Yeah . phd g:  for it 's professor f: No , no , I guess what I 'm saying is that grad d: Just let them read it how they read it . professor f: to some extent maybe we could just read them have them read how how they read it and it just means that we have to expand our our vocabulary out to stuff that we already have . Well that 's fine with me as long as It 's just that I didn't want to cause the people who would have been collecting digits the other way to not have the digits . phd g: So professor f:  we s we we 've We can do this for awhile phd g: OK . Do yo  , do you want do you want this Do you need training data or adaptation data out of this ? phd g: OK . professor f: How much of this do you need ? with  the phd g: It 's actually unclear right now . I just thought well we 're if we 're collec collecting digits , and Adam had said we were running out of the TI forms , I thought it 'd be nice to have them in groups , and probably , all else being equal , it 'd be better for me to just have single digits professor f: OK . phd g: since it 's , you know , a recognizer 's gonna do better on those anyway , and it 's more predictable . So we can know from the transcript what the person said and the transcriber , in general . professor f: OK , well if you pre phd g: But if they make mistakes , it 's no big deal if the people say a hundred instead of " one OO " . and also w maybe we can just let them choose " zero " versus " O " as they as they like because even the same person c sometimes says " O " and sometimes says " zero " in different context , professor f: Yeah . So I don't have a Specific need cuz if I did I 'd probably try to collect it , you know , without bothering this group , but If we can try it grad d: OK so so I can just add to the instructions to read it as digits not as connected numbers . phd g: Right , and you can give an example like , you know , " six sixty - one would be read as six one " . And is the spacing alright or do you think there should be more space between digits and groups ? professor f: OK . phd g: and grad d: Or is that alright ? phd g:  what do other people think cuz you guys are reading them . postdoc e: I it it to me it looks like you 've got the func the idea of grouping and you have the grou the idea of separation phd g: OK . postdoc e: and , you know , it 's just a matter of u i the instructions , that 's all . grad d: And I think there are about ten different gouping patterns professor f: Let 's try it . phd g: Righ - right , and you just they 're randomly generated and randomly assigned to digits . professor f: Sorry , I I was just gonna say , so we have in the vicinity of forty hours of of recordings now . And you 're saying two hours , is digits , so that 's roughly the ratio then , grad d: Yep . professor f: yeah like you say , I think a couple hours for a for a for a test test set 's OK . It 'd be nice to get , you know , more later because we 'll we might use use this up , in some sense , postdoc e:   professor f: but but  postdoc e: Yeah , I also would like to argue for that cuz it it seems to me that , there 's a real strength in having the same test replicated in a whole bunch of times and adding to that basic test bank . postdoc e:  ? Cuz then you have , you know , more and more , u chances to get away from random errors . And I think , the other thing too is that right now we have sort of a stratified sample with reference to dialect groups , and it might be there might be an argument to be made for having  f for replicating all of the digits that we 've done , which were done by non - native speakers so that we have a core that totally replicates the original data set , which is totally American speakers , and then we have these stratified additional language groups overlapping certain aspects of the database . I think that  trying to duplicate , spending too much effort trying to duplicate the existing TI - digits probably isn't too worthwhile because the recording situation is so different . postdoc e: Except that if you have the stimuli comparable , then it says something about the the contribution of setting professor f: No it 's it 's not the same . postdoc e: and professor f: A little bit , but the other differences are so major . professor f: They 're such major sources of variance that it 's it 's it 's  postdoc e: What 's an example of a of m some of the other differences ? Any other a difference ? professor f: Well i i individual human glottis is going to be different for each one , postdoc e: OK .  , we 're collecting it in a read digit in a particular list , and I 'm sure that they 're doing more specific stuff . grad d: Was it read ? professor f: Yeah , I think the reading zipcode stuff you 're thinking of would be OGI . professor f: Yeah , no TI - digits was read in th in read in the studio I believe . grad d: But but regardless it 's gonna it 's hard to compare cross - corpus . professor f: And they 're different circumstances with different recording environment and so forth , so it 's it 's it 's really pretty different . But I think the idea of using a set thing was just to give you some sort of framework , so that even though you couldn't do exact comparisons , it wouldn't be s valid scientifically at least it 'd give you some kind of  frame of reference .  , you know it 's not phd b: Hey Liz , What what do the groupings represent ? postdoc e: OK . phd b: You said there 's like ten different groupings ? phd g: Right , just groupings in terms of number of groups in a line , and number of digits in a group , and the pattern of groupings . Are the patterns like are they based on anything or phd g: I I just roughly looked at what kinds of digit strings are out there , and they 're usually grouped into either two , three , or four , four digits at a time . In the old days you probably only had three sequences , and telephone numbers were less , and so forth . So , there 's between ,  Well if you look at it , there are between like three and five groups , and each one has between two and four groupings and I purposely didn't want them to look like they were in any kind of pattern . phd g: So grad d: And which group appears is picked randomly , and what the numbers are are picked randomly . grad d: So unlike the previous one , which I d simply replicated TI - digits , this is generated randomly . phd g: But I think it 'd be great i to be able to compare digits , whether it 's these digits or TI - digits , to speakers , and compare that to their spontaneous speech , and then we do need you know a fair amount of of digit data because you might be wearing a different microphone grad d:   phd g: and ,  so it 's it 's nice to have the digits you know , replicated many times . phd g: so we have a problem with acoustic adaptation , and we 're not using the digit data now , but you know grad d: Oh , you 're not . v W we 're not we were running adaptation only on the data that we ran recognition on and I 'd As soon as someone started to read transcript number , that 's read speech and I thought " well , we 're gonna do better on that , grad d: Oh I see . So those speakers who are very quiet , shy grad d: That would be interesting to see whether that helps . phd g: r Right phd b: Like Adam ? grad d: Do you think that would help adapting on Yeah . phd g: Well , it sh  it 's the same micropho see the nice thing is we have that in the in the same meeting , grad d: Right . phd g: and so you don't get grad d: same microphone , phd a: Yeah . professor f: Yeah  , for the for the  acoustic research , for the signal - processing , farfield stuff , I see it as as as the place that we start . But , th  , it 'd be nice to have twenty hours of digits data , but but  the truth is I 'm hoping that we we through the the stuff that that you guys have been doing as you continue that , we get , the best we can do on the spontaneous stuff  ,  nearfield , and then  , we do a lot of the testing of the algorithms on the digits for the farfield , and at some point when we feel it 's mature and we understand what 's going on with it then we we have to move on to the spontaneous data with the farfield . phd g: The only thing that we don't have , I know this sounds weird , and maybe it 's completely stupid , but we don't have any overlapping digits . phd g: An - yea I know it 's weird , but  phd a: Overlapping digits ! grad d: The the problem I see with trying to do overlapping digits is the cognitive load . grad d: No it 's it 's not stupid , it 's just  , try to do it . phd g: I 'm just talkin for the stuff that like Dan Ellis is gonna try , grad d: here , let 's try it . phd g: OK So but grad d: So so you read the last line , I 'll read the first line . professor f: No , I 'll p phd g: So you plu you plug your ears . grad d: Oh I guess if you plug you 're ears you could do it , but then you don't get the the same effects . phd g: Well , what  is actually no not the overlaps that are well - governed linguistically , but the actual fact that there is speech coming from two people grad d: Yeah . phd g: and the beam - forming stuf all the acoustic stuff that like Dan Ellis and and company want to do . phd g: It would take one around amount of ti phd b: It 's the P - make of digit reading . I I mea I 'm I was sort of serious , but I really , I 'm I don't feel strongly enough that it 's a good idea , professor f: See , y grad d: You do the last line , I 'll do the first line . I 'm mean I think it 's doable , grad d: The poor transcribers phd g: I 'm just grad d: they 're gonna hate us . phd g: So , we we could have a round like where you do two at a time , and then the next person picks up when the first guy 's done , or something . professor f: Oh like a round , yeah , like in a a phd g: Like a , phd a: Yeah , just pairwise , professor f: yeah . phd g: I 'm actually sort of serious if it would help people do that kind o but the people who wanna work on it we should talk to them . professor f: I don't think we 're gonna collect vast amounts of data that way , phd g: So . professor f: but I think having a little bit might at least be fun for somebody like Dan to play around with , phd g: OK . grad d: I think maybe if we wanted to do that we would do it as a separate session , professor f: yeah . grad d: something like that rather than doing it during a real meeting and you know , do two people at a time then three people at a time and things like that . postdoc e: c c Can I can I have an another another question w about this ? grad d: Oh well . postdoc e: So , there are these digits , which are detached digits , but there are other words that contain the same general phon phoneme sequences . Like " wonderful " has " one " in it and and Victor Borge had a had a piece on this where he inflated the digits . Well , I wonder if there 's , an if there would be a value in having digits that are in essence embedded in real words to compare in terms of like the articulation of " one " in " wonderful " versus " one " as a digit being read . grad d: Oh ! grad c: It only sounds w good when you scream it , though . postdoc e: Well , I just wanted to offer that as a possible task professor f: Yes . postdoc e: because , you know , if we were to each read his embedded numbers words in sent in sentences cuz it 's like an entire sketch he does and I wouldn't take the inflated version . So he talks about the woman being " two - derful " , and and a But , you know , if it were to be deflated , just the normal word , it would be like a little story that we could read . postdoc e: I don't know if it would be useful for comparison , but it 's embedded numbers . grad d: I think for something like that we 'd be better off doing like  TIMIT . Well I think the question is what the research is , so  , I presume that the reason that you wanted to have these digits this way is because you wanted to actually do some research looking at the prosodic form here . professor f: So if somebody wanted to do that , if they wanted to look at the the the difference of the  phones in the digits in the context of a word versus  the digits a a non - digit word versus in digit word ,  that would be a good thing to do , but I think someone would have to express interest in that . professor f: I think , to  if you were interested in it then we could do it , for instance . grad d: We have ASR results from Liz , transcript status from Jane , and disk space and storage formats from Don . Does do we have any prefer preference on which way we wanna we wanna go ? phd g: Well I was actually gonna skip the ASR results part , in favor of getting the transcription stuff talked about grad d:   phd g: since I think that 's more important to moving forward , but  Morgan has this paper copy and if people have questions , it 's pretty preliminary in terms of ASR results because we didn't do anything fancy , but I think e just having the results there , and pointing out some main conclusions like it 's not the speaking style that differs , it 's the fact that there 's overlap that causes recognition errors . And then , the fact that it 's almost all insertion errors , which you would expect but you might also think that in the overlapped regions you would get substitutions and so forth , leads us to believe that doing a better segmentation , like your channel - based segmentation , or some kind of  , echo cancellation to get basically back down to the individual speaker utterances would be probably all that we would need to be able to do good recognition on the on the close - talking mikes . phd a: So these grad d: why don't you , if you have a hard copy , why don't you email it to the list . professor f: Yeah , so it 's the same thing ? phd g:  he he professor f: It 's the same thing I mailed to every everybody that w where it was , phd g: it it 's that paper . phd g: and it 's Let 's see , th I guess the other neat thing is it shows for sure w that the lapel , you know within speaker is bad . grad d: Horrible ? phd g: And it 's bad because it picks up the overlapping speech . phd a: So , your your ASR results were run on the channels synchronized , phd g: Yes , cuz that 's all that w had been transcribed at the time , phd a: OK . phd g: the closer t that would be very interesting for us phd b: So if phd g: because we professor f: Yeah , that 's that 's why I only used the part from use phd a: Yeah . phd b: So if there was a segment of speech this long phd g: cuz phd a: Yeah . phd b: and oh and someone said " oh , " the whole thing was passed to the recognizer ? grad d: And someone said " oh " in the front in the middle . In fact I I pulled out a couple classic examples in case you wanna u use them in your talk of phd b: That 's why there 's so many insertion errors ? grad c:   phd g: Chuck on the lapel , so Chuck wore the lapel three out of four times . phd g: yeah , and I wore the lapel once , and for me the lapel was OK . phd g: Right , but when Chuck wore the lapel and Morgan was talking there 're a couple really long utterances where Chuck is saying a few things inside , and it 's picking up all of Morgan 's words pretty well and so the rec you know , there 're error rates because of insertion Insertions aren't bounded , so with a one - word utterance and ten insertions you know you got huge error rate . So I this is sort of what we expected , but it 's nice to be able to to show it . phd g: And also I just wanted to mention briefly that ,  Andreas and I called up Dan Ellis who 's still stuck in Switzerland , and we were gonna ask him if if there 're you know , what 's out there in terms of echo cancellation and things like that . " phd g: And he We 've given him the data we have so far , so these sychronous cases where there are overlap . phd g: And he 's gonna look into trying to run some things that are out there and see how well it can do phd b: So phd g: because right now we 're not able to actually report on recognition in a real paper , like a Eurospeech paper , because it would look sort of premature . phd b: So So the idea is that you would take this big hunk where somebody 's only speaking a small amount in it , and then try to figure out where they 're speaking based on the other peopl phd g: Right . Or who 's At any point in time who 's the foreground speaker , who 's the background speaker . phd g: So there 's like grad d: But how would you do that automatically ? phd g: Well ther there 's phd a: I 've actually done some experiments with cross - correlation phd b: Right . phd a: and it seems to work pretty well to to get rid of those those overlaps , grad d:  that that 's the sort of thing that you would do . Exactly , so it 's it 's a phd b: So why do you want to do echo cancellation ? phd g: it would be techniques used from adaptive adaptive echo cancellation which I don't know enough about to talk about . phd g: But , right , and that would be similar to what you 're also trying to do , but using  , you know , more than energy phd a: Yeah . phd b: So it would be phd g: So the idea is to basically run this on the whole meeting . and get the locations , which gives you also the time boundaries of the individual speak phd b: OK . professor f: Yeah , Dave Dave  is , also gonna be doin usin playing around with echo cancellation for the nearfield farfield stuff , phd g: So . professor f: so we 'll be phd g: And I guess Espen ? This is  is he here too ? professor f: Yeah . phd g: May also be working So it would just be ver that 's really the next step because we can't do too much , you know , on term in terms of recognition results knowing that this is a big problem phd b:   phd b: I think this also ties into one of the things that Jane is gonna talk about too . grad d: I also wanted to say I have done all this chopping up of digits , postdoc e:   Definitely grad d: And and I have scripts that will extract it out from " key " files phd g: and Don should grad d: and and do all the naming automatically , phd g: OK . grad c: You 've compiled the list of , speaker names ? phd g: So that that 's it for the grad d:   grad d: and it does all sorts of matches because the way people filled out names is different on every single file so it does a very fuzzy sort of match . phd g: So at this point we can sort of finalize the naming , and so forth , grad c:   phd g: and we 're gonna basically re rewrite out these waveforms that we did because as you notice in the paper your " M O in one meeting and " M O - two " in another meeting and it 's we just need to standardize the grad c: Yeah . phd g: no it 's it 's professor f: No , I didn't notice that actually . So th I now have a script that you can just say basically look up Morgan , phd g: So professor f: Yeah . Is that something we need to talk about at the meeting , or should you just talk with Chuck at some other time ? grad c: I had some general questions just about the compression algorithms of shortening waveforms and I don't know exactly who to ask .  , yeah , which is grad d: And I assume half of it is scratch and half of it is ? grad c: I 'm not exactly sure how they partitioned it . grad c: yeah , I don't know what 's typical here , but  , it 's local though , so grad d: That doesn't matter . How do you do that ? professor f: In fact , this is an eighteen gig drive , or is it a thirty six gig drive with eighteen grad d: N grad c: Eighteen . It was a spare that Dave had around grad d: Slash N slash machine name , slash X A in all likelihood . grad d: so the the only question is how much of it The distinction between scratch and non - scratch is whether it 's backed up or not . grad d: So what you wanna do is use the scratch for stuff that you can regenerate . grad d: So , the stuff that isn't backed up is not a big deal because disks don't crash very frequently , grad c: Right .  all of this stuff can be regenerated , phd g: Yeah it 's grad c: it 's just a question grad d: Then put it all on scratch phd g: Well the grad d: because we 're ICSI is is bottlenecked by backup . grad d: So we wanna put phd g: Well I 'd leave all the All the transcript stuff shouldn't should be backed up , postdoc e:   phd g: but all the waveform Sound files should not be backed up , grad c: Yeah , I guess Right . So , I guess th the other question was then , should we shorten them , downsample them , or keep them in their original form ?  grad d: It just depends on your tools .  , because it 's not backed up and it 's just on scratch , if your sc tools can't take shortened format , I would leave them expanded , grad c: Right . grad d: so you don't have to unshorten them every single time you wanna do anything . phd g: We can downsample them , grad c: Do you think that 'd be OK ? phd g: so . phd g:  the r the front - end on the SRI recognizer just downsamples them on the fly , grad c: Yeah , I guess the only argument against downsampling is to preserve just the original files in case we want to experiment with different filtering techniques . phd g: so So that 's professor f: I I I 'm sorry phd g: Yeah , if professor f: Yeah , l  over all our data , we we want to not downsample . phd g: So we 're what we 're doing is we 're writing out  , this is just a question . We 're writing out these individual segments , that wherever there 's a time boundary from Thilo , or or Jane 's transcribers , you know , we we chop it there . phd g: And the reason is so that we can feed it to the recognizer , professor f:   grad d: Yeah , as I said , since that 's it 's regeneratable , what I would do is take downsample it , phd g: So Yeah . grad d: and compress it however you 're e the SRI recognizer wants to take it in . professor f: ye phd g: So we can't shorten them , grad c: Right . professor f: As yeah , as long as there is a a form that we can come from again , that is not downsampled , then , grad c: r Yeah . That that 's why we need more disk space professor f: uuu phd g: cuz we 're basically duplicating the originals ,  professor f: Yeah . But for for fu future research we 'll be doing it with different microphone positions and so on phd g: Oh yeah . professor f: we would like to phd b: So the SRI front - end won't take a  an an a large audio file name and then a a list of segments to chop out from that large audio file ? professor f: Yeah . phd b: They actually have to be chopped out already ? phd g: it 's better if they 're chopped out , phd b:  - huh . phd g: and and it it will be yeah , y we could probably write something to do that , but it 's actually convenient to have them chopped out cuz you can run them , you know , in different orders . grad d: And that 's the whole point about the naming conventions phd g: you can get rid of grad d: is that you could run all the English speaking , phd g: Yeah , it it 's a lot faster . grad d: all the native speakers , and all the non - native speakers , phd g: Right . You can grab everything with the word " the " in it , grad d: and all the men , and all the women . phd g: and it 's That 's a lot quicker than actually trying to access the wavefile each time , find the time boundaries and So in principle , yeah , you could do that , phd b: I don't I don't think that 's really right . phd g: but it 's but it 's  grad d: " That 's just not right , man . " The the point phd g: These are long These are long grad d: So so s For example , what if you wanted to run run all the native speakers . grad d: Right , so if if you did it that way you would have to generate a program that looks in the database somewhere , extracts out the language , finds the time - marks for that particular one , do it that way . The way they 're doing it , you have that already extracted and it 's embedded in the file name . And so , you know , you just say phd g: We - yeah that 's so that 's part of it grad d: y so you just say you know " asterisk E asterisk dot wave " , and you get what you want . And the other part is just that once they 're written out it it is a lot faster to to process them . Otherwise , you 're just accessing grad d: This is all just temporary access , so I don't I think it 's all just It 's fine . Two gig ? phd g: So we 're also looking at these in Waves like for the alignments and so forth . phd g: You need to s have these small files , and in fact , even for the Transcriber program  grad d: Yes you can . phd g: Yeah , if you try to load s really long waveform into X Waves , you 'll be waiting there for phd b: No , I I 'm not suggesting you load a long wave file , phd g: Oh phd b: I 'm just saying you give it a start and an end time . grad d: I th w The transcribers didn't have any problem with that did they Jane ? postdoc e: What 's th u w in what respect ? phd g: Loading the long phd a: No , with the Transcriber tool , it 's no problem . phd g: It takes a very long ti phd a: Yeah just to load a transcription postdoc e: In the in   grad d: Are you talking about Transcriber or X Waves ? phd g: Huh . phd g: Actually , you 're talking about Transcriber , right ? phd a: Yeah . grad d: And they were loading the full mixed files then , postdoc e: Yeah . Well we we have a problem with that , you know , time - wise on a It - it 's a lot slower to load in a long file , grad d:  . phd g: and also to check the file , so if you have a transcript , grad d: Well regardless , it 's professor f: Yeah . phd g:  it 's I I think overall you could get everything to work by accessing the same waveform and trying to find two you know , the begin and end times .  , but I think it 's more efficient , if we have the storage space , to have the small ones . grad d: and , it 's no problem , right ? phd g: Yeah , it 's grad d: Because it 's not backed up . grad d: So we just phd g: It 's it 's just grad d: If we don't have a spare disk sitting around we go out and we buy ourselves an eighty gigabyte drive and make it all scratch space . phd g: Yeah , so these wouldn't be backed up , the postdoc e: Yeah . grad d: So remind me afterward phd g: And grad d: and I 'll and we 'll look at your disk and see where to put stuff .  , I could just u do a DU on it right ? And just see which how much is on each So . Alright so , first of all , there was a an interest in the transcribe transcription , checking procedures and and I can tell you first , to go through the steps although you 've probably seen them .  , as you might imagine , when you 're dealing with , r really c a fair number of words , and  , @ @ natural speech which means s self - repairs and all these other factors , that there 're lots of things to be , s standardized and streamlined and checked on . And , so , I did a bunch of checks , and the first thing I did was obviously a spell - check . And at that point I discovered certain things like , " accommodate " with one " M " , that kind of thing . And then , in addition to that , I did an exhaustive listing of the forms in the data file , which included n detecting things like f faulty punctuation and things phd b: I 'm I 'm sorry to interrupt postdoc e: Yeah ? phd b: you could could I just back up a little bit postdoc e: Sure , please , phd b: and postdoc e: yeah , please , please . phd b: So you 're doing these So the whole process is that the transcribers get the conversation postdoc e: Yeah , yeah , yeah . phd b: And then when they 're finished with it , it comes to you , postdoc e: That 's right . And so , I do a an exhaustive listing of the forms Actually , I will go through this in in order , so if if we could maybe wait and stick keep that for a second cuz we 're not ready for that . grad d: So on the fifth page , seven down postdoc e: Yeah , yeah , yeah , yeah . Exactly ! Exactly ! Alright so , a spelling check first then an exhaustive listing of the ,  all the forms in the data with the punctuation attached and at that point I pick up things like , oh , you know , word followed by two commas . But there 's w And that that 's done with the assumption that pronunciation variants can be handled . So for things like " and " , the fact that someone doesn't say the " D " ,  that 's not important enough to capture in the transcription because a a good pronunciation , you know , model would be able to handle that . However , things like " cuz " where you 're lacking an entire very prominent first syllable , and furthermore , it 's a form that 's specific to spoken language , those are r reasons f for those reasons I I kept that separate , and used the convention of using " CUZ " for that form , however , glossing it so that it 's possible with the script to plug in the full orthographic form for that one , and a couple of others , not many . So " wanna " is another one , " going "  , " gonna " is another one , with just the assumption , again , that this th these are things which it 's not really fair to a c consider expect that a pronunciation model , to handle . And Chuck , you in you indicated that " cuz " is is one of those that 's handled in a different way also , didn't you ? Did I phd b: I don't remember . postdoc e: but someone told me that in fact " cuz " is treated differently in , i u in this context because of that r reason that , it 's a little bit farther than a pronunciation variant . OK , so after that , let 's see , phd b: So that was part of the spell - check , or was that that was after the spell - check ? postdoc e:  . Well so when I get the exhau So the spell - check picks up those words because they 're not in the dictionary . postdoc e: So it gets " cuz " and " wanna " and that grad d: And then you gloss them ? postdoc e: Yeah ,   Run it through I have a sed You know , so I do sed script saying whenever you see " gonna " you know , " convert it to gonna " , you know , " gloss equals quote going - to quote " , you know . OK , I also wrote a script which will , retrieve anything in curly brackets , or anything which I 've classified as an acronym , and a pronounced acronym . So if it 's " ACL " then it 's " A " underscore " C " underscore " L " . grad d: And so so your list here , are these ones that actually occurred in the meetings ? postdoc e: And the th Yes . phd g: can I ask a question about the glossing ,  before we go on ? postdoc e: Yeah . phd g: So , for a word like " because " is it that it 's always predictably " because " ?  , is " CUZ " always meaning " because " ? postdoc e: Yes , but not the reverse . So sometimes people will say " because " in the meeting , and if if they actually said " because " , then it 's written as " because " with no w " cuz " doesn't even figure into the equation . professor f: But but in our meetings people don't say " hey cuz how you doing ? " phd g: Beca - because Right . phd g: so , I guess So , from the point of view of postdoc e: That 's a good point . phd g: The the only problem is that with for the recognition we we map it to " because " , grad d: Well , phd g: and so if we know that " CUZ " postdoc e: That 's fine . phd g: but , we don't grad d: You have the gloss form so you always replace it . postdoc e: and he 's bee he has a glo he has a script that grad c: I replace the " cuz " with " because " if it 's glossed . grad c: And phd g: But then there are other glosses that we don't replace , right ? Because postdoc e: Yes . postdoc e: on the different on the different types of comments , which we 'll which we 'll see in just a second . postdoc e: So the pronounceable acronyms get underscores , the things in curly brackets are viewed as comments . w And maybe we 'll expand that grad d:  postdoc e: but the but the comments are , of four types mainly right now . grad d: Can ca postdoc e: Another type is ,  grad d: So a are we done with acronyms ? Cuz I had a question on what what this meant . postdoc e: OK so , gloss is things like replacing the full form u with the , more abbreviated one to the left .  , then you have if it 's  , there 're a couple different types of elements that can happen that aren't really properly words , and wo some of them are laughs and breathes , so we have  that 's prepended with a v a tag of " VOC " . phd a: Whew !  postdoc e: And the non - vocal ones are like door - slams and tappings , and that 's prepended with a no non - vocalization . phd b: So then it just an ending curly brace there , or is there something else in there . postdoc e: Oh yeah , so i e this would grad d: A comment , basically . postdoc e: And then the no non - vocalization would be something like a door - slam . And then the third type right now , is m things that fall in the category of comments about what 's happening . So it could be something like , you know , " referring to so - and - so " , " talking about such - and - such " , you know , " looking at so - and - so " . phd b: on the middle t So , in the first case that gloss applies to the word to the left . But in the middle two Th - it 's not applying to anything , right ? postdoc e: Yeah , and this gets substituted here . grad d: Well the " QUAL " can be The " QUAL " is applying to the left . postdoc e: Well , and actually , it is true that , with respect to " laugh " , there 's another one which is " while laughing " , grad d: " While laughing " . postdoc e: and that is , i i An argument could be made for this tur turning that into a qualitative statement because it 's talking about the thing that preceded it , but at present we haven't been , coding the exact scope of laughing , you know , and so to have " while laughing " , you know that it happened somewhere in there which could well mean that it occurred separately and following , or , you know , including some of the utterances to the left . Haven't been awfully precise about that , but I have here , now we 're about to get to the to this now , I have frequencies . But , the very front page deals with this , final c pa  , aspect of the standardization which has to do with the spoken forms like "  -  " and "  -  " and " ha " and "  -  " and all these different types . And , someone pointed out to me , this might have been Chuck , about ,  about how a recognizer , if it 's looking for "  - hmmm " with three M 's , and it 's transcribed with two M 's , that it might  , that it might increase the error rate which is which would really be a shame because  , I p I personally w would not be able to make a claim that those are dr dramatically different items . postdoc e: I I should say grad d: So it 's a small list . So , I 'm gonna I 'm gonna I 'm gonna check grad d: That that 's known as " found data " . I got It was stored in a place I didn't expect , grad c: It 's like the z Zapruder Film . So I 'll I 'll be able to get through that tonight , and then everyth i well , actually later today probably . postdoc e: And I made it so that these are , with a couple exceptions but , things that you wouldn't find in the spell - checker so that they 'll show up really easily . And ,  grad c: Jane , can I ask you a question ? What 's that very last one correspond to ? postdoc e: Sure . grad c: is that like someone 's like burning or some such thing ? postdoc e: So - c I haven't listened to it so I don't know . grad c: Like their hair 's on fire ? postdoc e: I haven't heard it actually . grad d: Ah ! phd a: It 's the Castle of Ah ! phd g: Actually we we gave this to our pronunciation person , grad c: it looks like that . postdoc e: Did she hear the th did she actually hear it ? Cuz I haven't heard it . phd g: No , we just gave her a list of words that , you know , weren't in our dictionary and so of course it picked up stuff like this , and she just didn't listen so she didn't know . Yeah I 'm curious to se hear what it is , but I didn't know wanna change it to something else until I knew . phd g: Maybe it 's " argh " ? postdoc e: Well , sss , you know phd g:  grad c: But that 's not really like postdoc e: Hhh . grad c: No one really says " argh , " you know , postdoc e:  phd g: Yeah . Right , no one say grad c: it 's not professor f: Well , you just did . phd b: Except for now ! grad c: Well , there 's another there 's another word error . grad d: Ah ! postdoc e: phd g: So , Jane , what 's the d grad d: Maybe he died while dictating . phd g: I have one question about the the " EH " versus like the " AH " and the "  " . postdoc e: That 's partly a nonnative - native thing , phd g: OK . phd g:  postdoc e: But it 's mostly non - native phd a: H phd b: That 's " eh " versus " ah " ? phd g: S OK . grad d: Eh ? phd g: " Eh , " yeah right , cuz there were were some speakers that did definite " eh 's " postdoc e:   phd g: but right now we phd b: They were the Canadians , right ? professor f: Canadians , yeah , yeah , yeah . phd g: So , it it 's actually probably good for us to know the difference between the real " eh " and the one that 's just like "  " or transcribed " aaa " postdoc e: Exactly . phd g: cuz in like in Switchboard , you would see e all of these forms , but they all were like "  " . grad d: You mean just the single letter " a " as in the particle ? phd a: The transcription or grad d: Article . phd g: No , no ,  like the the "  " , postdoc e: "  " . grad d: I 'm just these poor transcribers , they 're gonna hate this meeting . phd g: But you 're a native German speaker so it 's not a not a issue for phd a: Yeah . phd g: So it 's like Japanese and Spanish postdoc e: Yeah I I think you 've  - huh , yeah . phd g: and grad d: I didn't get that , postdoc e: That makes sense . postdoc e: Yeah , and so , you know , th th I have there are some , Americans who who are using this " eh " too , and I haven't listened to it systematically , maybe with some of them , they 'd end up being "  's " but , I my spot - checking has made me think that we do have " eh " in also , American e e data represented here . But any case , that 's the this is reduced down from really quite a long a much longer list , phd g: Yeah this is great . grad d: Yeah , it 's good , postdoc e: and this is grad d: yeah . postdoc e: functionally pretty , you know , also It was fascinating , I was listening to some of these , I guess two nights ago , and it 's just hilarious to liste to to do a search for the "  -  's " . postdoc e: Just I wanted to say I w think it would be fun to make a montage of it because there 's a "   postdoc e: All these different vocal tracts , you know , but it 's it 's the same item .  , then the acronyms y and the ones in parentheses are ones which the transcriber wasn't sure of , grad d: Oh I see . postdoc e: and I haven't been able to listen to to to clarify , but you can see that the parenthesis convention makes it very easy to find them grad d: o How about question mark ? postdoc e: cuz it 's the only place where where they 're used . grad d: So they so it 's " PLP ? " postdoc e: Exactly . Sometimes the contrastive stress is showing up , and ,  professor f: I 'm sorry , I I got lost here . What - w what 's the difference between the parenthesized acronym and the non - parenthesized ? postdoc e: The parenthesized is something that the transcriber thought was ANN , but wasn't entirely sure . So I 'd need to go back or someone needs to go back , and say , you know , yes or no , professor f: Ah . postdoc e: But the parentheses are used only in that context in the transcripts , of of noti noticing that there 's something uncertain . grad d: Yeah , P - make is phd g: Yeah  cuz they they have no idea , grad d: That 's a good one . phd g: but it 's professor f: I I don't recognize a lot of these . grad d: I know ! I I was saying that I think a lot of them are the Networks meeting . phd g: and the PTA was in these , topics about children , postdoc e: Yeah . phd g: Is the P - PTA working ? postdoc e: Right and sometimes , you see a couple of these that are actually " OK 's " so it 's it 's may be that they got to the point where  it was low enough understandable understandability that they weren't entirely sure the person said " OK . " You know , so it isn't really necessarily a an undecipherable acronym , grad c: There 's a lot of " OK 's " . This professor f: The number to the left is the number of incidences ? grad d: Count . professor f: So CTS is really big here , grad d: Yeah , I wonder what it is . phd a: So what is the difference between " papers rustling " and " rustling papers " ? professor f: IP , I know what IP is . postdoc e: But , I I 'm a little hesitant to to collapse across categories unless I actually listen to them . OK well professor f: Wh - the self - referential aspect of these these p phd g: I 'm wai grad c: Yeah . phd g: Well this is exactly how people will prove that these meetings do differ because we 're recording , right ? grad d: Yes . phd g: Y no normally you don't go around saying , " Now you 've said it six times . phd g: Now you 've said " postdoc e: But did you notice that there were seven hundred and eighty five instances of " OK " ? phd a: Seven hundred eighty - five instances . grad d: And that 's an underestimate postdoc e: Extra forty one if it 's questioned . grad c: Is this after like did you do some  replacements for all the different form of " OK " to this ? professor f: Yeah . phd b: Wait a minute , w s professor f: So now we 're up to seven hundred and eighty eight . postdoc e: Yeah that 's grad c: Although , what 's there 's one with a slash after it . phd b: Was that somewhere where they were gonna say " new speaker " or something ? postdoc e: No , I looked for that , but that doesn't actually exist . postdoc e: I i it 's the only grad c: There 's postdoc e: it 's the only pattern that has a slash after it , and I think it 's it 's an epiphenomenon . grad d: So I 'll just I was just looking at the bottom of page three there , is that " to be " or " not to be " . phd b: There 's no tilde in front of it , postdoc e: Oh that 's cute . " postdoc e: There is th one Y well , no , that 's r that 's legitimate . So now , comments , you can see they 're listed again , same deal , with exhaustive listing of everything found in everything except for these final th thirty minutes . grad d: OK so , on some of these QUALs , postdoc e: Yeah . grad d: are they really QUALs , or are they glosses ? So like there 's a " QUAL TCL " . professor f: What 's a QUAL ? grad d: Oh I see , I see . grad c: Sh - shouldn't it be " QUAL TICKLE " or something ? grad d: It wasn't said " TCL " . grad c: Like it 's not postdoc e: On the in the actual script in the actual transcript , I s I So this this happens in the very first one . postdoc e: Because we they didn't say " TCL " , they said " tickle " . professor f: I f I forget , what 's QUAL ? postdoc e: Qual - qualifier . grad c: It 's not something you wanna replace with postdoc e: Comment or contextual comment . phd b: So they didn't mean " tickle " as in Elmo , grad c: but phd a: Tickle ? professor f: Yeah . phd g: But at some point  , we probably shoul grad d: We 'll probably add it to the language model . phd b: Add what , Liz ? grad d: We can go on lan lan add it to both dictionary and language model . phd g: it 's in the language model , w yeah , but it so it 's the pronunciation model that has to have a pronunciation of " tickle " . Right ? phd a: " tickle " is pronounced " tickle " ? phd b: What are you saying ? grad d: It 's pronounced the same it 's pronounced the same as the verb . phd g: I 'm sorry ! grad d: So I think it 's the language model that makes it different . What I meant is that there should be a pronunciation " tickle " for TCL as a word . phd g: And that word in the in , you know , it stays in the language model wherever it was . phd g: Yeah you never would put " tickle " in the language model in that form , postdoc e:  grad d: Right . There 's actually a bunch of cases like this with people 's names and phd b: So how w there 'd be a problem for doing the language modeling then with our transcripts the way they are . Yeah so th th there there 's a few cases like that where the  , the word needs to be spelled out in in a consistent way as it would appear in the language , but there 's not very many of these . grad d: And and you 'll ha you 'll have to do it sychronously . grad d: Right , so y so , whoever 's creating the new models , will have to also go through the transcripts and change them synchronously . We have this there is this thing I was gonna talk to you about at some point about , you know , what do we do with the dictionary as we 're up updating the dictionary , these changes have to be consistent with what 's in the Like spelling people 's names and so forth . If we make a spelling correction to their name , like someone had Deborah Tannen 's name mispelled , and since we know who that is , you know , we could correct it , grad d: You can correct it . phd g: but but we need to make sure we have the mispel If it doesn't get corrected we have to have a pronunciation as a mispelled word in the dictionary . postdoc e: Well , of course now the the Tannen corre the spelling c change . So if there 's things that get corrected before we get them , it 's it 's not an issue , postdoc e:   phd g: but if there 's things that  , we change later , then we always have to keep our the dictionary up to date . And then , yeah , in the case of " tickle " I guess we would just have a , you know , word " TCL " which phd b:   phd g: which normally would be an acronym , you know , " TCL " grad d: Right . postdoc e: " ICSI " is is one of those that sometimes people pronounce and sometimes they say " ICSI . postdoc e: So , those that are l are listed in the acronyms , I actually know phd g: Oh yeah . The others , e those really do need to be listened to cuz I haven't been able to go to all the IC ICSI things , phd g: Right , exactly . postdoc e: and and until they 've been listened to they stay as " ICSI " . professor f: Don and I were just noticing , love this one over on page three , " vocal vocal gesture mimicking sound of screwing something into head to hold mike in place . postdoc e: It was ! In fact , it was ! Yeah ! grad d: A lot of these are me the the " beep is said with a high pit high pitch and lengthening . grad d: That was the I was imitating  , beeping out postdoc e: Yeah , that 's it . phd g: Oh there is something spelled out " BEEEEEEP " grad c:  postdoc e: That 's it . Because he was saying , " How many E 's do I have to allow for ? " grad c: You need a lot of grad d: What I meant was " beep " . postdoc e: And those of course get get picked up in the frequency check grad d: glosses . postdoc e: and you know  it gets kicked out in the spelling , and it also gets kicked out in the , freq frequency listing . postdoc e: I have the there 're various things like " breathe " versus " breath " versus " inhale " and , hhh , you know , I don't know . I I think they don't have any implications for anything else so it 's like I 'm tempted to leave them for now an and It 's easy enough to find them when they 're in curly brackets . professor f: " Sings finale - type song " grad c: Yeah , that was in the first meeting . grad d: postdoc e: Yeah , but I don't actually remember what it was . " postdoc e:  Chuck Chuck led to a refinement here which is to add " NUMS " if these are parts of the read numbers . Now you already know i that I had , in places where they hadn't transcribed numbers , I put " numbers " in place of any kind of numbers , but there are places where they , it th this convention came later an and at the very first digits task in some transcripts they actually transcribed numbers . And , d Chuck pointed out that this is read speech , and it 's nice to have the option of ignoring it for certain other prob  p  , things . And that 's why there 's this other tag here which occurs a hundred and five or three hundred and five times right now which is just well n n " NUMS " by itself grad d: " NUMS " , yeah .  , i with the sed command you can really just change it however you want because it 's systematically encoded , you know ? grad d: Yep . postdoc e: Have to think about what 's the best for for the overall purposes , but in any case , " numbers " and " NUMS " are a part of this digits task thing . And actually , th  , the reason I b did it this way was because I initially started out with the other version , you have the numbers and you have the full form and the parentheses , however sometimes people stumble over these numbers they 're saying . And there 's no way of capturing that if you 're putting the numbers off to the side . You can't have the seven and grad d: So what 's to the left of these ? postdoc e: The left is i so example the very first one , grad d:   postdoc e: And then , in here , " NUMS " , so it 's not going to be mistaken as a gloss . grad d: OK now , the other example is , in the glosses right there , postdoc e: Thank you . grad d: What what 's to the left of that ? postdoc e: Well now In that case it 's people saying things like " one one one dash so - and - so " or they 're saying  " two  zero " whatever . postdoc e: And in that case , it 's part of the numbers task , and it 's not gonna be included in the read digits anyway , phd b: So there will be a " NUMS " tag on those lines ? postdoc e: so I m in the  There is . postdoc e: So , so gloss in the same line that would have " gloss quote one one one dash one thirty " , you 'd have a gloss at the end of the line saying , " curly bracket NUMS curly bracket " . postdoc e: So if you if you did a , a " grep minus V nums " phd g: Oh , so you could do " grep minus V nums " . phd g: So there wouldn't be something like i if somebody said something like , " Boy , I 'm really tired , OK . Cuz I was doing the " grep minus V " quick and dirty and looked like that was working OK , postdoc e:   Now why do we what 's the reason for having like the point five have the " NUMS " on it ? Is that just like when they 're talking about their data or something ? postdoc e: This is more because phd g: Or postdoc e: Yeah . Oh these are all these , the " NUMS point " , this all where they 're saying " point " something or other . phd g: These are all like inside the spontaneous postdoc e: And the other thing too is for readability of the transcript .  if you 're trying to follow this while you 're reading it it 's really hard to read , you know eh , " so in the data column five has " , you know , " one point five compared to seventy nine point six " , it 's like when you see the words it 's really hard to follow the argument . And this is just really a a way of someone who would handle th the data in a more discourse - y way to be able to follow what 's being said . postdoc e: So this is where Chuck 's , overall h architecture comes in , phd g: I see .  , there will be scripts that are written to convert it into these t these main two uses and th some scripts will take it down th e into a f a for ta take it to a format that 's usable for the recognizer an  , other scripts will take it to a form that 's usable for the for linguistics an and discourse analysis . These will just be things that are generated , grad d: Right postdoc e: and e by using scripts . postdoc e: When things change then the the script will cham change but the but there won't be stored copies of in different versions of things . phd g: So , I guess I 'd have one request here which is just , maybe to make it more robust , th that the tag , whatever you would choose for this type of " NUMS " where it 's inside the spontaneous speech , is different than the tag that you use for the read speech . phd g: that way w if we make a mistake parsing , or something , we don't see the " point five " , or or it 's not there , then we phd b:   phd g: a Just an And actually for things like " seven eighths " , or people do fractions too I guess , you maybe you want one overall tag for sort of that would be similar to that , postdoc e: Except phd g: or As long as they 're sep as they 're different strings that we that 'll make our p sort of processing more robust . postdoc e: Well phd g: Cuz we really will get rid of everything that has the " NUMS " string in it . phd b: I suppose what you could do is just make sure that you get rid of everything that has " curly brace NUMS curly brace " . postdoc e: You know , as I said I was considering changing it to " digits " . And , it just i you know , it 's just a matter of deciding on whatever it is , and being sure the scripts know . phd g: It would probably be safer , if you 're willing , to have a separate tag just because  , then we know for sure .  , phd b: Yeah , and it makes it I guess the thing about phd g: but it it 's probably not hard for a person to tell the difference phd b: Yeah . phd g: because one 's in the context of a you know , a transcribed word string , phd b: Right . postdoc e: The other thing is you can get really so minute with these things phd g: and So postdoc e: and increase the size of the files and the re and decrease the readability to such an extent by simply something like " percent " . Now I I could have adopted a similar convention for " percent " , but somehow percent is not so hard , you know ? grad d:  . postdoc e: i It 's just when you have these points and you 're trying to figure out where the decimal places are And we could always add it later . And you 'll find both of those in one of these meetings , where he 's saying " well the first point I wanna make is so - and - so " and he goes through four points , and also has all these decimals . phd b: what does the SRI recognizer output for things like that ? " seven point five " . phd b: Right , the word " seven " ? grad d: Well , the numbers ? phd b: The number " seven " ? phd g: The word . professor f: So I 'd so " I 'd like I 'd like to talk about point five " . phd g: it 's the same point , actually , the the p you know , the word " to " and the word y th " going to " and " to go to " those are two different " to 's " and so there 's no distinction there . phd g: It 's just just the word " point " has Yeah , every word has only one , yeah e one version even if even if it 's A actually even like the word " read " and " read " Those are two different words . phd g: So , yeah , I I like the idea of having this in there , I just I was a little bit worried that , the tag for removing the read speech because i What if we have like " read letters " or , I don't know , grad d: We might wanna just a separate tag that says it 's read . OK ? Are we done ? postdoc e: Well I wanted to say also regarding the channelized data , grad d: Oh , I guess we 're not done . postdoc e: that , Thilo requested , that we ge get some segments done by hand to e e s reduce the size of the time bins wh like was Chuc - Chuck was mentioning earlier that , that , if you if you said , " Oh " and it was in part of a really long , s complex , overlapping segment , that the same start and end times would be held for that one grad d: Well postdoc e: as for the longer utterances , grad d: We did that for one meeting , right , postdoc e: and grad d: so you have that data don't you ? phd a: Yeah , that 's the training data . postdoc e: And he requested that there be , similar , samples done for five minute stretches c involving a variety of speakers and overlapping secti sections . postdoc e: He gave me he did the very nice , he he did some shopping through the data and found segments that would be useful . In addition the I 've I have the transcribers expanding the amount that they 're doing actually . postdoc e: So right now , I know that as of today we got an extra fifteen minutes of that type , and I 'm having them expand the realm on either side of these places where they 've already started . postdoc e: But if if you know , and I and he 's gonna give me some more sections that that he thinks would be useful for this purpose . postdoc e: Because it 's true , if we could do the the more fine grained tuning of this , using an algorithm , that would be so much more efficient . phd a: So I I thought we we sh we sh perhaps we should try to to start with those channelized versions just to just to try it . Give it Give one tr transcriber the the channelized version of of my speech - nonspeech detection and look if if that 's helpful for them , or just let them try if if that 's better or If they if they can postdoc e: You mean to start from scratch f in a brand new transcript ? phd a: Yeah . As it stands we 're still in the phase of sort of , cleaning up the existing data getting things , in i m more tight tightly time  , aligned . I also wanna tell  , I also wanted to r raise the issue that OK so , there 's this idea we 're gonna have this master copy of the transcript , it 's gonna be modified by scripts t into these two different functions . postdoc e: So right now we 've taken this i initial one , it was a single channel basically the way it was input . And now , thanks to the advances made in the interface , we can from now on use the channelized part , and , any changes that are made get made in the channelized version kind of thing . But I wanted to get all the finished all the checks phd b: Yeah , so that has implications for your script . So , have those e e the vis the ten hours that have been transcribed already , have those been channelized ? And I know I 've seen @ @ I 've seen they 've been channelized , postdoc e: Yes , they have . grad d: All ten hours ? grad c: but postdoc e: Except for the missing thirty minutes . grad c: have they  have they been has the time have the time markings been adjusted , p on a per channel grad d: Great . postdoc e: for for a total of like twenty m f for a total of Let 's see , four times total of about an thirty minutes . grad c: I guess , I don't know if we should talk about this now , or not , but I grad d: Well it 's just we 're missing tea . grad c: No , but  my question is like should I wait until all of those are processed , and channelized , like the time markings are adjusted before I do all the processing , and we start like branching off into the into the our layer of  transcripts . postdoc e: Well , you know the problem the problem is that some some of the adjustments that they 're making are to bring are to combine bins that were time bins which were previously separate . postdoc e: And so , i i i it 's true that it 's likely to be adjusted in the way that the words are more complete . postdoc e: so I it 's gonna be a more reliable thing and I 'm not sure grad c:  I 'm sure about that , postdoc e: Yeah . grad c: but do you have like a time frame when you can expect like all of it to be done , or when you expect them to finish it , or postdoc e: Well partly it depends on how  , how e effective it will be to apply an algorithm because i this takes time , grad c: Yeah . phd b: So right now the what you 're doing is you 're taking the  , the o original version and you 're sort of channelizing yourself , right ? grad c: Yeah .  i if the time markings aren't different across channels , like the channelized version really doesn't have any more information . grad c: So , I was just  , originally I had done before like the channelized versions were coming out . grad c: phd b: So I I th I think probably the way it 'll go is that , you know , when we make this first general version and then start working on the script , that script @ @ that will be ma you know primarily come from what you 've done , we 'll need to work on a channelized version of those originals . phd b: And so it should be pretty much identical to what you have t except for the one that they 've already tightened the boundaries on . phd b: So postdoc e: Yeah ,  phd b: and then probably what will happen is as the transcribers finish tightening more and more , you know , that original version will get updated postdoc e: yeah . phd b: But the I guess the ef the effect for you guys , because you 're pulling out the little wave forms into separate ones , that would mean these boundaries are constantly changing you 'd have to constantly re rerun that , grad c: I know . phd g: I I think the harder part is making sure that the transc the transcription phd b: OK . phd g: So if you b merge two things , then you know that it 's the sum of the transcripts , but if you split inside something , you don't where the word which words moved . phd g: And that 's wh that 's where it becomes a little bit  , having to rerun the processing . grad c: You know , if I just have to run three scripts to extract it all and let it run on my computer for an hour and a half , or however long it takes to parse and create all the reference file , that 's not a problem . And I know exactly like what the steps will work what 's going on , in the editing process , phd b: Yeah . postdoc e: So that 's I  I could there were other checks that I did , but it 's I think that we 've unless you think there 's anything else , I think that I 've covered it <doc-sep>professor g: Alright ! grad a: So , sorry about not professor g: We 're not crashing . postdoc f: I don't know if you 're professor g: Yeah , that 's right . postdoc f: maybe raise the issue of microphone ,  procedures with reference to the cleanliness of the recordings . professor g: OK , transcription , microphone issues postdoc f: And then maybe ask , th  , these guys . The we have great great , p steps forward in terms of the nonspeech - speech pre - segmenting of the signal . phd d: use grad a: but I 'm not sure if that 's of general interest or not . professor g:  phd b: Since ,  since I have to leave as usual at three - thirty , can we do the interesting stuff first ? postdoc f: I beg your pardon ? professor g: Well phd c: Which is ? grad a: What 's the interesting stuff ? postdoc f: I beg your pardon ? phd d: Yeah . professor g: But phd b: Well , I guess the work that 's been done on segmentation would be most phd c: Yeah .  , and , the other thing , which I 'll just say very briefly that maybe relates to that a little bit , which is that , one of the suggestions that came up in a brief meeting I had the other day when I was in Spain with , Manolo Pardo and Javier , Ferreiros , who was here before , was , why not start with what they had before but add in the non - silence boundaries . So , in what Javier did before when they were doing ,  h he was looking for , speaker change points . As a simplification , he originally did this only using silence as , a putative , speaker change point . professor g: And , he did not , say , look at points where you were changing broad sp  , phonetic class , for instance . professor g: And , so one of the things that they were pushing in d in discussing with me is , w why are you spending so much time , on the , feature issue , when perhaps if you sort of deal with what you were using before phd d:  - huh . professor g: and then just broadened it a bit , instead of just ta using silence as putative change point also ? phd d: Nnn , yeah . professor g: So then you 've got you already have the super - structure with Gaussians and H - you know , simple H M Ms and so forth . And you you might So there was a there was a little bit of a a a a difference of opinion because I I thought that it was it 's interesting to look at what features are useful . professor g: But , on the other hand I saw that the they had a good point that , if we had something that worked for many cases before , maybe starting from there a little bit Because ultimately we 're gonna end up with some s su kind of structure like that , phd d: Yeah . professor g: where you have some kind of simple  and you 're testing the hypothesis that , there is a change . Yeah , why don't we do the speech - nonspeech discussion ? postdoc f: Yeah . Do I I hear you you didn't phd c: Speech - nonspeech ? OK . phd c: so , what we basically did so far was using the mixed file to to detect s speech or nonspeech portions in that . phd c: And what I did so far is I just used our old Munich system , which is an  - ba based system with Gaussian mixtures for s speech and nonspeech . And it was a system which used only one Gaussian for silence and one Gaussian for speech . phd c: Adam , Dave , and I , we did , for that dialogue and I trained it on that . They they can use it or ? postdoc f: they they think it 's a terrific improvement . postdoc f: And , y you also did some something in addition which was , for those in which there was , quiet speakers in the mix . phd c: And so I did two mixtures , one for the loud speakers and one for the quiet speakers . grad a: And did you hand - label who was loud and who was quiet , or did you just ? phd c: I did that for for five minutes of one dialogue grad a: Right . phd c: It 's just our our old Munich , loudness - based spectrum on mel scale twenty twenty critical bands and then loudness . phd c: And four additional features , which is energy , loudness , modified loudness , and zero crossing rate . And so I did some some modifications in those parameters , basically changing the minimum minimum length for s for silence to have , er to have ,  yeah to have more or less , silence portions in inserted . grad a: But for overlap I imagine that doesn't work at all , phd c: Yeah . But it it saves so much time the the transcribers professor g:  grad a: Yep . W w we originally we did that professor g: Just phd c: but we saw , when we used it , f for our close - talking microphone , which yeah , for our for our recognizer in Munich we saw that w it 's it 's not it 's not so necessary . professor g: Yeah , I don't think it 's a big deal for this application , phd c: Yeah . But then there 's another thing that also Thilo 's involved with , which is ,  OK , and and also Da - Dave Gelbart .  we ,  regarding the representation of overlaps , because at present , because of the limitations of th the interface we 're using , overlaps are , not being encoded by the transcribers in as complete and , detailed a way as it might be , and as might be desired I think would be desired in the corpus ultimately . postdoc f: So we don't have start and end points at each point where there 's an overlap . So @ @ the limits of the over of of the interface are such that we were at this meeting we were entertaining how we might either expand the the interface or find other tools which already do what would be useful . Because what would ultimately be , ideal in my my view and I think  , I had the sense that it was consensus , is that , a thorough - going musical score notation would be the best way to go . Because you can have multiple channels , there 's a single time - line , it 's very clear , flexible , and all those nice things . So , I spoke I had a meeting with Dave Gelbart on on and he had , excellent ideas on how the interface could be modified to to do this kind of representation . But , he in the meantime you were checking into the existence of already , existing interfaces which might already have these properties .  , I talked with , Munich guys from from Ludwi - Ludwig Maximilians University , who do a lot of transcribing and transliterations . phd c: And they basically said they have they have , a tool they developed themselves and they can't give away , f it 's too error - prone , and had it 's not supported , a a a and professor g: Yeah . phd c: But , Susanne Bur - Burger , who is at se CMU , he wa who was formally at in Munich and w and is now at with CMU , she said she has something which she uses to do eight channels , trans transliterations , eight channels simultaneously , professor g: Excuse me . grad a: Well , maybe we should get it and if it 's good enough we 'll arrange Windows machines to be available . postdoc f: I also wanted to be sure  , I 've I 've seen the this this is called Praat , PRAAT , which I guess means spee speech in Dutch or something . phd c: Yeah , but then I 'm not sure that 's the right thing for us . professor g: The other thing , to keep in mind , we 've been very concerned to get all this rolling so that we would actually have data , postdoc f: Mmm , yeah . professor g: but , I think our outside sponsor is actually gonna kick in postdoc f:   So I don't know if we have a long - term need to do lots and lots of transcribing . I think we had a very quick need to get something out and we 'd like to be able to do some later because just it 's inter it 's interesting . But as far a you know , with with any luck we 'll be able to wind down the larger project . phd b: But you s grad a: What our decision was is that we 'll go ahead with what we have with a not very fine time scale on the overlaps . postdoc f: And and I was just thinking that , if it were possible to bring that in , like , you know , this week , then when they 're encoding the overlaps it would be nice for them to be able to specify when you know , the start points and end points of overlaps . postdoc f: And , so my my goal was w m my charge was to get eleven hours by the end of the month . And it 'll be I 'm I 'm I 'm clear that we 'll be able to do that . postdoc f: I sent it to ,  who did I send that to ? I sent it to a list and I thought I sent it to the e to the local list . postdoc f: You saw that ? So Brian did tell me that in fact what you said , that ,  that our that they are making progress and that he 's going that they 're going he 's gonna check the f the output of the first transcription and and professor g: basically it 's it 's all the difference in the world .  , basically it 's just saying that one of our one of our best people is on it , postdoc f: Yeah . So phd b: But about the need for transcription , postdoc f: Isn't that great ? phd b: don't we didn't we previously decide that the IBM transcripts would have to be checked anyway and possibly augmented ? professor g: So . grad a: Yeah , and Dave Gelbart did volunteer , postdoc f: Good . grad a: and since he 's not here , I 'll repeat it to at least modify Transcriber , which , if we don't have something else that works , I think that 's a pretty good way of going . My approach originally , and I 've already hacked on it a little bit it was too slow because I was trying to display all the waveforms . grad a: That if you just display the mix waveform and then have a user interface for editing the different channels , that 's perfectly sufficient . And and , Dan Ellis 's hack already allows them to be able to display different waveforms to clarify overlaps and things , grad a: No . They can only display one , postdoc f: so that 's already grad a: but they can listen to different ones . postdoc f: Oh , yes , but Well , yes , but what  is that , from the transcriber 's perspective , those two functions are separate . And Dan Ellis 's hack handles the , choice the ability to choose different waveforms from moment to moment . postdoc f:  grad a: The waveform you 're looking at doesn't change . postdoc f: Yeah , but that 's that 's OK , cuz they 're they 're , you know , they 're focused on the ear anyway . postdoc f: the hack to preserve the overlaps better would be one which creates different output files for each channel , grad a: Right . postdoc f: which then would also serve Liz 's request of having , you know , a single channel , separable , cleanly , easily separable , professor g:   Well , holidays may have interrupted things , cuz in in in They seem to want to get absolutely clear on standards for transcription standards and so forth with with us . I just think I told them to contact Jane that , if they had a grad a: Oh , OK . So is it cuz with any luck there 'll actually be a a there 'll be collections at Columbia , collections at at UW  Dan Dan is very interested in doing some other things , grad a: Right . grad a: Well , I think it 's important both for the notation and the machine representation to be the same . postdoc f: N there was also this , email from Dan regarding the speech - non nonspeech segmentation thing . postdoc f: I don't know if , we wanna ,  and Dan Gel - and Dave Gelbart is interested in pursuing the aspect of using amplitude as a a a as a basis for the separation . He was talking he was talking  , we he had postdoc f: Cross professor g: Yeah , cross - correlation . phd c: Cross professor g: I had mentioned this a couple times before , the c the commercial devices that do , voice ,  you know , active miking , postdoc f:  - huh . professor g: So , by doing that , you know , rather than setting any , absolute threshold , you actually can do pretty good , selection of who who 's talking . professor g:  And those those systems work very well , by the way , so people use them in panel discussions and so forth with sound reinforcement differing in in sort of , phd d:  - huh . professor g:  and , those if Boy , the guy I knew who built them , built them like twenty twenty years ago , grad a:  . Cuz there is one thing that we don't have right now and that is the automatic , channel identifier . postdoc f: That that , you know , that would g help in terms of encoding of overlaps . postdoc f: The the transcribers would have less , disentangling to do if that were available . So I think , you know , basically you can look at some p you have to play around a little bit , to figure out what the right statistic is , postdoc f: But . professor g: but you compare each microphone to some statistic based on the on the overall phd c: Yeah . So that , you cou yo grad a: Yeah , although the the using the close - talking I think would be much better . professor g: I just it 'd be If I was actually working on it , I 'd sit there and and play around with it , and and get a feeling for it .  , the the the ,  But , you certainly wanna use the close - talking , as a at least . professor g: I don't know if the other would would add some other helpful dimension or not . What what are the different , classes to to code , the the overlap , you will use ? postdoc f: to code d phd d: What you you postdoc f: so types of overlap ? phd d: Yeah . postdoc f: so at a meeting that wasn't transcribed , we worked up a a typology . postdoc f: And ,  phd d: Look like , you t you explaining in the blackboard ? The ? Yeah ? Yeah . So it i the it 's basically a two - tiered structure where the first one is whether the person who 's interrupted continues or not . And then below that there 're subcategories , that have more to do with , you know , is it , simply backchannel phd d:   postdoc f: or is it , someone completing someone else 's thought , or is it someone in introducing a new thought . And I hope that if we do a forced alignment with the close - talking mike , that will be enough to recover at least some of the time the time information of when the overlap occurred .  , I I I I 've phd b: So who 's gonna do that ? Who 's gonna do forced alignment ? grad a: Well , u  , IBM was going to . grad a: and I imagine they still plan to but but , you know , I haven't spoken with them about that recently . postdoc f: It occurs to me one of my transcribers t told me today that she 'll be finished with one meeting , by professor g:   postdoc f: well , she said tomorrow but then she said you know , but the , you know let 's let 's just , say professor g:   I know these are er , I could send him that if it would be possible , or a good idea or not , to try to do a s forced alignment on what we 're on the way we 're encoding overlaps now . professor g: you know , basically he 's he just studies , he 's a colleague , a friend , and , postdoc f: Yeah ! professor g: they and and , you know , the the organization always did wanna help us . professor g: It was just a question of getting , you know , the right people connected in , who had the time . professor g: So , eh grad a: Is he on the mailing list ? The Meeting Recorder mailing li ? postdoc f: Oh ! grad a: We should add him . phd e: Did something happen , Morgan , that he got put on this , or was he already on it , grad a: Add him . phd e: or ? professor g: No , I , eh , eh , p It it oc I h it 's Yeah , something happened . So , where are we ? Maybe , brief Well , let 's why don't we talk about microphone issues ? postdoc f: Yeah . professor g: That was that was a grad a: so one thing is that I did look on Sony 's for a replacement for the mikes for the head m head - worn ones cuz they 're so uncomfortable . But I think I need someone who knows more about mikes than I do , because I couldn't find a single other model that seemed like it would fit the connector , which seems really unlikely to me . Does anyone , like , know stores or know about mikes who who would know the right questions to ask ? professor g: Oh , I probably would . phd e: You couldn't you couldn't find the right connector to go into these things ? grad a: Yep . When I looked , i they listed one microphone and that 's it phd e: Huh ! grad a: as having that type of connector . Well , let 's look at it together grad a: it seems it seems really unlikely to me that there 's only one . professor g: and postdoc f: And there 's no adaptor for it ? phd c: Yeah . professor g: Who who are we buying these from ? grad a: professor g: That 'd be grad a: I have it downstairs . grad a: And then ,  just in terms of how you wear them  , I had thought about this before .  , when when when you use a product like DragonDictate , they have a very extensive description about how to wear the microphone and so on . grad a: But I felt that in a real situation we were very seldom gonna get people to really do it and maybe it wasn't worth concentrating on . But professor g: Well , I think that that 's that 's a good back - off position . That 's what I was saying earlier , th that , you know , we are gonna get some recordings that are imperfect and , hey , that 's life . But I I think that it it doesn't hurt , the naturalness of the situation to try to have people wear the microphones properly , if possible , grad a:   professor g: because , the natural situation is really what we have with the microphones on the table . professor g: I think , you know , in the target applications that we 're talking about , people aren't gonna be wearing head - mounted mikes anyway . professor g: So this is just for u these head - mounted mikes are just for use with research . professor g: And , it 's gonna make You know , if if An - Andreas plays around with language modeling , he 's not gonna be m wanna be messed up by people breathing into the microphone . professor g: So it 's it 's ,  grad a: Well , I 'll dig through the documentation to DragonDictate and ste s see if they still have the little form . professor g: Right ?  , and any phd b: It 's interesting , I talked to some IBM guys , last January , I think , I was there . phd b: And they said , the breathing is really a a terrible problem for them , to to not recognize breathing as speech . grad a: Well , that 's the It seemed to me when I was using Dragon that it was really microphone placement helped an in ,  an enormous amount . grad a: So you want it enough to the side so that when you exhale through your nose , it doesn't the wind doesn't hit the mike . professor g: I remember when I was when I I I I used , a prominent laboratory 's , speech recognizer about ,  This was , boy , this was a while ago , this was about twelve twelve years ago or something . And , they were they were perturbed with me because I was breathing in instead of breathing out . And they had models for they they had Markov models for br breathing out but they didn't have them for breathing in . Well , what I wondered is whether it 's possible to have to maybe use the display at the beginning grad a: Yeah . postdoc f: to be able to to judge how how correctly  , have someone do some routine whatever , and and then see if when they 're breathing it 's showing . postdoc f: I don't know if the if it 's professor g: I grad a: You can definitely see it . grad a: And so , you know , I 've I 've sat here and watched sometimes the breathing , professor g: I grad a: and the bar going up and down , and I 'm thinking , I could say something , but professor g: I think grad a: I don't want to make people self - conscious . And you can do some , you know , first - order thing about it , which is to have people move it , a away from being just directly in front of the middle phd d: Yeah . postdoc f: Yeah , i professor g: And then , you know , I think there 's not much Because you can't al you know , interfere w you can't fine tune the meeting that much , I think . It just seems like i if something l simple like that can be tweaked and the quality goes , you know , dramatically up , then it might be worth doing . My my feedback from the transcribers is he is always close to crystal clear and and just fan fantastic to phd c: Yeah . postdoc f: I could say something about about the Well , I don't know what you wanna do . professor g: About what ? postdoc f: About the transcribers or anything or ? I don't know . professor g: Well , the other phd b: But , just to to ,  professor g: why don't we do that ? phd b: One more remark , concerning the SRI recognizer . It is useful to transcribe and then ultimately train models for things like breath , and also laughter is very , very frequent and important to to model . phd b: So , grad a: So , phd b: if you can in your transcripts mark grad a: mark them ? phd b: mark very audible breaths and laughter especially , phd c: Mmm . postdoc f: They 're putting Eh , so in curly brackets they put " inhale " or " breath " . Now they 're they 're not being awfully precise , m So they 're two types of laughter that are not being distinguished . postdoc f: One is when sometimes s someone will start laughing when they 're in the middle of a sentence . postdoc f: And and then the other one is when they finish the sentence and then they laugh . So , I I did s I did some double checking to look through  , you 'd need to have extra  e extra complications , like time tags indicating the beginning and ending of of the laughing through the utterance . phd b: It 's not so I don't think it 's ,  postdoc f: And that and what they 're doing is in both cases just saying " curly brackets laughing " a after the unit . phd b: As as long as there is an indication that there was laughter somewhere between two words I think that 's sufficient , phd c: Yeah . phd b: So as long as you can stick a you know , a t a tag in there that that indicates that there was laughter , grad a: Oh , I didn't know that . grad a: That would be a really interesting prosodic feature , postdoc f: Then phd d: Yeah . postdoc f: So , if they laugh between two words , you you 'd get it in between the two words . postdoc f: But if they laugh across three or four words you you get it after those four words . phd b: Well , the thing that you is hard to deal with is whe when they speak while laughing . phd b: But , that 's not as frequent as just laughing between speaking , postdoc f: OK . grad a: So are do you treat breath and laughter as phonetically , or as word models , or what ? phd b: so professor g:  is it ? phd d: Huh . There was a there 's actually a word for  , it 's not just breathing but all kinds of mouth grad a:   grad a: So train a phone in the neural net ? phd b: Same thing ? Yeah . professor g: No grad a: Oh , it does ? phd b: So the so the the mouth noise , word has just a single phone , that is for that . professor g: Right ?  , you could you could say well , let  we now think that laughter should have three sub sub sub - units in the the three states ,  different states . professor g: And then you would have three  , you know , eh , eh , it 's u grad a: Do whatever you want . phd b: And the the pronun the pronunciations the pronunciations are l are somewhat non - standard . phd b: They actually are  , it 's just a single , s  , you know , a single phone in the pronunciation , but it has a self - loop on it , so it can grad a: To go on forever ? phd b: r can go on forever . grad a: And how do you handle it in the language model ? phd b: It 's just a it 's just a word . We also tried , absorbing these  , both laughter and and actually also noise , and ,  phd d: Yeah . We also tried absorbing that into the pause model  , the the the model that that matches the stuff between words . You you did get me to thinking about I I 'm not really sure which is more frequent , whether f f laughing I think it may be an individual thing . grad a: I was noticing that with Dan in the one that we ,  we hand tran hand - segmented , professor g: Yeah . And and one thing that c that we 're not doing , of course , is we 're not claiming to , get be getting a representation of mankind in these recordings . Why don why don't we just since we 're on this vein , why don't we just continue with , what you were gonna say about the transcriptions postdoc f: OK . professor g: and ? postdoc f: the I I 'm really very for I 'm extremely fortunate with the people who , applied and who are transcribing for us . They are ,  really perceptive and very ,  and I 'm not just saying that cuz they might be hearing this . grad a: Cuz they 're gonna be transcribing it in a few days . So I , e you know , I I brought them in and , trained them in pairs because I think people can raise questions grad a: That 's a good idea . postdoc f: you know , i i the they think about different things and they think of different and  , I trained them to , f on about a minute or two of the one that was already transcribed . This also gives me a sense of You know , I can I can use that later , with reference to inter - coder reliability kind of issues . But the main thing was to get them used to the conventions and , you know , the idea of the th th the size of the unit versus how long it takes to play it back so these th sort of calibration issues . And then , I just set them loose and they 're they all have e a already background in using computers . postdoc f: Well , they they 're very perce they 'll So one of them said " well , you know , he really said " n " , not really " and " , phd d: Yeah . postdoc f: so what what should I do with that ? " grad a: Yeah . If it 's an a noncanonical p " That one , I think we you know , with Eric 's work , I sort of figure we we can just treat that as a variant . But I told them if if there 's an obvious speech error , like I said in one thing , professor g: OK . postdoc f: and I gave my my example , like I said , " microfon " in instead of " microphone " . But it but I thought it 's not worth fixing cuz often when you 're speaking everybody knows what what you mean . postdoc f: But I have a convention that if it 's obviously a noncanonical pronunciation a speech error with you know , wi within the realm of resolution that you can tell in this native English American English speaker , you know that I didn't mean to say " microfon . " Then you 'd put a little tick at the beginning of the word , professor g: Yeah . postdoc f: and that just signals that , this is not standard , and then in curly brackets " pron error " . Well Well , you know , it might be something we 'd wanna do with some , s small subset of the whole thing . Where were they when we needed them ? postdoc f: I think professor g: We certainly wouldn't wanna do it with everything .  , if ,  so I I told them that , we don't know if this will continue past the end of the month professor g:  - huh . postdoc f: and I also m I think they know that the data p source is limited and I may not be able to keep them employed till the end of the month even , although I hope to . professor g: The other thing we could do , actually , is , use them for a more detailed analysis of the overlaps . professor g: Right ? grad a: We could get a very detailed overlap if they were willing to transcribe each meeting four or five times . professor g: But I 've been saying the other thing is just go through it for the overlaps . professor g: Right ? postdoc f: And with the right in interface professor g: Given that y and and do so instead of doing phonetic , transcription for the whole thing , phd d: Yeah . professor g: which we know from the Steve 's experience with the Switchboard transcription is , you know , very , very time - consuming . And and you know , it took them I don't know how many months to do to get four hours . But , the other thing is since we 've been spending so much time thinking about overlaps is is maybe get a much more detailed analysis of the overlaps . professor g: I 'm open to every consideration of what are some other kinds of detailed analysis that would be most useful . professor g: It 's a we have we have due to @ @ variations in funding we have we seem to be doing , very well on m money for this this year , and next year we may have have much less . grad a: Is you mean two thousand one ? professor g: So I don't wanna hire a grad a: Calendar year or ? professor g: calendar year two thousand one . So it 's  , it 's we don't wanna hire a bunch of people , a long - term staff , grad a: Full - time . professor g: because the the funding that we 've gotten is sort of a big chunk for this year . But having temporary people doing some specific thing that we need is actually a perfect match to that kind of , funding . phd e: Are they working full - time now , or ? postdoc f: But Some of them are . But what  is Oh , I shouldn't say it that way because that does sound like forty - hour weeks . I th I I would say they 're probably they don't have o they don't have other things that are taking away their time . professor g: I postdoc f: I I I haven't checked them all , but just spot - checking . grad a: I think it would be professor g: I remember when we were transcribing BeRP , Ron Kay , volunteered to to do some of that . Well , you know , and I also thought , y Liz has this , eh , you know , and I do also , this this interest in the types of overlaps that are involved . These people would be great choices for doing coding of that type if we wanted , grad a: We 'd have to mark them . grad a: I think it would also be interesting to have , a couple of the meetings have more than one transcriber do , professor g: Yeah . postdoc f: You know , there 's also , the e In my mind , I think A An - Andreas was leading to this topic , the idea that , we haven't yet seen the the type of transcript that we get from IBM , and it may just be , you know , pristine . But on the other hand , given the lesser interface Cuz this is , you know we 've got a good interface , we 've got great headphones , m  professor g: It could be that they will  theirs will end up being a kind of fir first pass or something . professor g: Maybe an elaborate one , cuz again they probably are gonna do these alignments , which will also clear things up . Al - although you have to s Don't you have to start with a close enough approximation of the of the verbal part to be able to ? professor g: Well , tha that 's that 's debatable . professor g: Right ?  , so the so the argument is that if your statistical system is good it will in fact , clean things up . professor g: And , so in principle you could start up with something that was kind of rough  , to give an example of , something we used to do , at one point , back back when Chuck was here in early times , is we would take , da take a word and , have a canonical pronunciation and , if there was five phones in a word , you 'd break up the word , into five equal - length pieces which is completely gross . professor g: Right ?  , th the timing is off all over the place in just about any word . You start off with that and the statistical system then aligns things , and eventually you get something that doesn't really look too bad . professor g: So so I think using a a good aligner , actually can can help a lot . If you have a good alignment , it helps the , th the human in in taking less time to correct things . I guess there 's another aspect , too , and I don't know  , this this is very possibly a different , topic . So like in a you know , the topics that are covered during a meeting with reference to the other , uses of the data , professor g:   postdoc f: so being able to find where so - and - so talked about such - and - such , then ,  e  , I I I did sort of a a rough pass on encoding , like , episode - like level things on the , transcribed meeting professor g:   postdoc f: where that i if that 's something that we wanna do with each meeting , sort of like a ,  it 's like a manifest , when you get a box full of stuff , or or if that 's ,  professor g:   postdoc f: i I I don't know what  , level of detail would be most useful . I don't know i if that 's something that I should do when I look over it , or if we want someone else to do , or whatever . professor g: eh , was p Well , you know , the thing I 'm concerned about is we wanted to do these digits postdoc f: Oh , yeah . But I think , do you , maybe , eh ? Did you prepare some whole thing you wanted us just to see ? phd d: Yeah . professor g: how long a ? phd d: I I think it 's it 's fast , because , I have the results , eh , of the study of different energy without the law length . Eh , eh , in the in the measurement , the average , dividing by the by the , variance . phd d: the other ,  the the last w  , meeting eh , I don't know if you remain we have problem to with the with with the parameter with the representations of parameter , because the the valleys and the peaks in the signal , eh , look like , eh , it doesn't follow to the to the energy in the signal . professor g: No , that there 's no point in going through all of that if that 's the bottom line , really . professor g: So , I I think we have to start  , there there 's two suggestions , really , which is ,  what we said before is that , phd d: Mmm , yeah . professor g: it looks like , at least that you haven't found an obvious way to normalize so that the energy is anything like a reliable , indicator of the overlap . professor g: I I 'm I 'm still a little f think that 's a little funny . professor g: but but you don't want to keep ,  keep knocking at it if it 's if you 're not getting any any result with that . But , the other things that we talked about is , pitch - related things and harmonicity - related things , phd d: Yeah .  But , a completely different tack on it wou is the one that was suggested , by your colleagues in Spain , phd d: Yeah . professor g: That is to say , use , you know , as as you 're doing with the speech , nonspeech , use some very general features . professor g: You know , have a have a couple Markov models and and , try to indi try to determine , you know , w when is th when are you in an overlap , when are you not in an overlap . professor g: And let the , statistical system determine what 's the right way to look at the data . professor g: I I , I think it would be interesting to find individual features and put them together . professor g: But given the limitation in time and given the fact that Javier 's system already exists doing this sort of thing , phd d: Yeah . professor g: but , its main limitation is that , again , it 's only looking at silences which would phd d: Yeah . phd d: I I I think that , eh , the possibility , eh , can be that , eh , Thilo , eh , working , eh , with a new class , not only , eh , nonspeech and speech , but , eh , in in in the speech class , professor g:   phd d: dividing , eh , speech , eh , of from a speaker and overlapping , to try to to do , eh , eh , a fast a fast , eh , experiment to to prove that , nnn , this fea eh , general feature , eh , can solve the the the problem , professor g: Yeah . phd d: And I hope the the next week I will have , eh , some results and we we will show we will see , eh , the the parameter the pitch , eh , tracking in with the program . phd d: And , nnn , nnn professor g: Ha - h have you ever looked at the ,  Javier 's , speech segmenter ? phd c: No . professor g: Cuz again the idea is there the limitation there again was that he was he was only using it to look at silence as a as a as a as a p putative split point between speakers . professor g: But if you included , broadened classes then in principle maybe you can cover the overlap cases . phd c: Yeah , but I 'm not too sure if if we can really represent overlap with with the s detector I I I used up to now , phd d: Mmm , yeah . grad a: I think with phd c: the to speech - nonspeech as grad a: That 's right . But I think Javier 's phd c: it 's only speech or it 's it 's it 's nonspeech . professor g: N n grad a: It doesn't have the same Gaus -  , H M M modeling , phd c: Yeah . grad a: But ,  professor g: Well , it 's sort of has a simple one . grad a: Does it ? professor g: Right ? It 's it 's just it 's just a isn't it just a Gaussian phd d: Yeah . Oh , it doesn't have it doesn't have any temporal ,  ? grad a: Maybe I 'm misremembering , but I did not think it had a Markov professor g: I thought it Yeah . phd d: No , Javier di doesn't worked with , a Markov grad a: Yeah , I didn't think so . So he 's just he just computes a Gaussian over potential grad a: Yep . professor g: And and grad a: It 's just , that i it he has the two - pass issue that What he does is , as a first pass he he p he does , a guess at where the divisions might be and he overestimates . And that 's just a data reduction step , so that you 're not trying at every time interval . grad a: And right now he 's doing that with silence and that doesn't work with the Meeting Recorder . professor g: The other thing one could do is Couldn't  , it 's So you have two categories phd c: Yeah . Couldn't you have a third category ? So you have ,  you have , nonspeech , single - person speech , and multiple - person speech ? postdoc f: He has this on his board actually . Don't you have , like those those several different categories on the board ? professor g: Right ? And then you have a Markov model for each ? phd c:  I 'm not sure . But it 's not too easy , I think , the the transition between the different class , to model them in in the system I have now . professor g: the th the reason why , I was suggesting originally that we look at features is because I thought , well , we 're doing something we haven't done before , phd c: Yeah . professor g: It seems like if two people two or more people talk at once , it should get louder , phd c: Yeah . professor g: and , there should be some discontinuity in pitch contours , phd c: I had the impression . professor g: and , there should overall be a , smaller proportion of the total energy that is explained by any particular harmonic sequence in the spectrum . professor g: So far , Jose has has been By the way , I was told I should be calling you Pepe , but phd d: Yeah . professor g: the has has , been exploring , e largely the energy issue and , as with a lot of things , it is not  , like this , it 's not as simple as it sounds . professor g: And then there 's , you know Is it energy ? Is it log energy ? Is it LPC residual energy ? Is it is it is it , delta of those things ?  , what is it no Obviously , just a simple number absolute number isn't gonna work . So it should be with compared to what ? Should there be a long window for the normalizing factor and a short window for what you 're looking at ? phd c: Yeah . professor g: Or , you know , how b short should they be ? So , phd d:  . professor g: th he 's been playing around with a lot of these different things and and so far at least has not come up with any combination that really gave you an indicator . professor g: So I I still have a hunch that there 's it 's in there some place , but it may be given that you have a limited time here , it it just may not be the best thing to to to focus on for the remaining of it . professor g: So pitch - related and harmonic - related , I 'm I 'm somewhat more hopeful for it . professor g: But it seems like if we just wanna get something to work , phd c: Yeah . professor g: that , their suggestion of of Th - they were suggesting going to Markov models , but in addition there 's an expansion of what Javier did . professor g: even if the features that you give it are maybe not ideal for it , it 's just sort of this general filter bank phd c: Yeah . professor g: or or cepstrum or something ,  Eee it 's in there somewhere probably . phd d: But , eh , what did you think about the possibility of using the Javier software ? Eh , the ,  the ,  the BIC criterion , the the t to train the the Gaussian , eh , using the the mark , eh , by hand , eh , eh , to distinguish be mmm , to train overlapping zone and speech zone .  , eh , I I I think that an interesting , eh , experiment , eh , could be , th eh , to prove that , mmm , if s we suppose that , eh , the the first step  , the the classifier what were the classifier from Javier or classifier from Thilo ? W What happen with the second step ? I  , what what happen with the , eh the , clu the ,  the clu the clustering process ? grad a:   grad a: What do you mean ? phd d: I  , that is is enough is enough , eh , to work well , eh , to , eh , separate or to distinguish , eh , between overlapping zone and , eh , speaker zone ? Because th if if we if we , eh , nnn , develop an classifier and the second step doesn't work well , eh , we have another problem . I had tried doing it by hand at one point with a very short sample , phd d: N grad a: and it worked pretty well , but I haven't worked with it a lot . So what I d I d I took a hand - segmented sample phd d: Nnn , yeah . grad a: and I added ten times the amount of numbers at random , phd d: Yeah . But is is if grad a: But this was just very anecdotal sort of thing . phd d: But it 's possible with my segmentation by hand that we have information about the the overlapping , grad a: Right . So if we if we fed the hand - segmentation to Javier 's and it doesn't work , then we know something 's wrong . phd d: this kind o emph emphasises parameter and Gaussian grad a: Yeah . Y do you know where his software is ? Have you used it at all ? phd d: I yeah have 
Summary:
Grad H mentioned that they were working on standardizing data in XML format, but expressed dissatisfaction with the current format. They were also developing tools to extract information from XML files using Java and Perl. The transcriber pool has been meeting expectations in terms of work completed within the allocated time. Grad D explained the importance of combining different experts in the network to adapt to various situations, suggesting dividing the network into parts and allowing it to determine which experts to trust in each situation. The transcriber pool is making good progress and may be utilized for other coding tasks in the future, such as analyzing speaker overlap. They are also coding non-speech gestures like breaths and laughter to improve recognition results. The group reviewed transcription checking procedures and discussed coordinating data channelization and presegmentation with tighter time bins.
What are the current thoughts on using XML tools and the status of transcription?