input: What was the range of errors in the models?
  Professor D: OK. So, uh You can fill those out, uh {pause} after, actually, so So, I got, uh {pause} these results from, uh, Stephane. Also, um, I think that, uh {pause} um {pause} we might hear later today, about other results. I think s that, uh, there were some other very good results that we're gonna wanna compare to. But, {vocalsound} r our results from other {disfmarker} other places, yeah.
  PhD A: I I'm sorry? I didn't
  Professor D: Um, I got this from you
  PhD A: Yeah.
  Professor D: and then I sent a note to Sunil about the {disfmarker} cuz he has been running some other systems
  PhD A: Mm - hmm.
  Professor D: other than the {disfmarker} the ICSI OGI one.
  PhD A: Oh yeah.
  Professor D: So {pause} um, I wan wanna {disfmarker} wanna see what that is. But, uh, you know, so we'll see what it is comparatively later. But {pause} it looks like, um
  PhD A: M yeah.
  Professor D: You know most of the time, even {disfmarker} I mean even though it's true that the overall number for Danish {disfmarker} we didn't improve it If you look at it individually, what it really says is that there's, um, uh Looks like out of the six cases, between the different kinds of, uh, matching conditions {pause} out of the six cases, there's basically, um, a couple where it stays about the same, uh, three where it gets better, and one where it gets worse.
  PhD A: Yeah.
  Professor D: Uh, go ahead.
  PhD A: Y Actually, uh, um, for the Danish, there's still some kind of mystery because, um, um, when we use the straight features, we are not able to get these nice number with the ICSI OGI one, I mean. We don't have this ninety - three seventy - eight, we have eight
  PhD E: Eighty - nine forty - four.
  PhD A: yeah. Uh, so, uh, that's probably something wrong with the features that we get from OGI. Uh, and Sunil is working on {disfmarker} on trying to {disfmarker} to check everything.
  Professor D: Oh, and {disfmarker} and we have a little time on that {disfmarker} and {disfmarker} actually so
  PhD A: Hmm?
  Professor D: We have a little bit of time on that, actually.
  PhD A: Yeah.
  Professor D: We have a day or so, so When {disfmarker} when {disfmarker} when do you folks leave?
  PhD A: Uh, Sunday.
  Professor D: Sunday? So So, uh Yeah, until Saturday midnight, or something, we have W we {disfmarker} we have time, yeah. Well, that would be good. That'd be good.
  PhD A: Yeah.
  Professor D: Yeah. Uh, and, you know, i u when whenever anybody figures it out they should also, for sure, email Hynek because Hynek will be over there {vocalsound} telling people {vocalsound} what we did, so he should know.
  PhD A: Mmm. Yeah.
  Professor D: Good, OK. So, um So, we'll {disfmarker} we'll hold off on that a little bit. I mean, even with these results as they are, it's {disfmarker} it's {disfmarker} it's really not that bad. But {disfmarker} but, uh, um And it looks like the overall result as they are now, even without, you know, any {disfmarker} any bugs being fixed is that, uh, on the {disfmarker} the other tasks, we had this average of, uh, forty uh {disfmarker} nine percent, or so, improvement. And here we have somewhat better than that than the Danish, and somewhat worse than that on the German, but I mean, it sounds like, uh, one way or another, the methods that we're doing can reduce the error rate from {disfmarker} from mel ceptrum {pause} down by, you know {pause} a fourth of them to, uh, a half of them. Somewhere in there, depending on the {pause} exact case. So So that's good. I mean, I think that, uh, one of the things that Hynek was talking about was understanding what was in the other really good proposals and {disfmarker} and trying to see if what should ultimately be proposed is some, uh, combination of things. Um, if, uh {disfmarker} Cuz there's things that they are doing {pause} there that we certainly are not doing. And there's things that we're doing that {pause} they're not doing. And {disfmarker} and they all seem like good things.
  PhD A: Yeah.
  Professor D: So
  PhD E: Mmm, yeah.
  PhD C: How much {disfmarker} how much better was the best system than ours?
  Professor D: So Well, we don't know yet.
  PhD C: Mmm.
  Professor D: Uh, I mean, first place, there's still this thing to {disfmarker} to work out, and second place {disfmarker} second thing is that the only results that we have so far from before were really development set results.
  PhD C: Oh, OK.
  Professor D: So, I think in this community that's of interest. It's not like everything is being pinned on the evaluation set. But, um, for the development set, our best result was a little bit short of fifty percent. And the best result of any system was about fifty - four, where these numbers are the, uh, relative, uh, reduction in, uh, word error rate.
  PhD C: Oh, OK.
  Professor D: And, um, the other systems were, uh, somewhat lower than that. There was actually {disfmarker} there was much less of a huge range than there was in Aurora one. In Aurora one there were {disfmarker} there were systems that ba basically didn't improve things.
  PhD C: Hmm.
  Professor D: And here the {disfmarker} the worst system {pause} still reduced the error rate by thirty - three percent, or something, in development set.
  PhD C: Oh, wow.
  Professor D: So {disfmarker} so, you know, sort of everybody is doing things between, well, roughly a third of the errors, and half the errors being eliminated, {vocalsound} uh, and varying on different test sets and so forth.
  PhD C: Mm - hmm.
  Professor D: So I think Um {pause} It's probably a good time to look at what's really going on and seeing if there's a {disfmarker} there's a way to combine the best ideas while at the same time not blowing up the amount of, uh, resources used, cuz that's {disfmarker} that's critical for this {disfmarker} this test.
  PhD C: Do we know anything about {disfmarker} who {disfmarker} who's was it that had the lowest on the dev set?
  Professor D: Um, uh, the, uh, the there were two systems that were put forth by a combination of {disfmarker} of, uh, French Telecom and Alcatel. And, um they {disfmarker} they differed in some respects, but they e em one was called the French Telecom Alcatel System the other was called the Alcatel French Telecom System, {vocalsound} uh, which is the biggest difference, I think. But {disfmarker} but there're {disfmarker} there're {disfmarker} there're some other differences, too. Uh, and {disfmarker} and, uh, they both did very well,
  PhD C: Uh - huh.
  Professor D: you know? So, {vocalsound} um, my impression is they also did very well on {disfmarker} on the {disfmarker} the, uh, evaluation set, but, um, I {disfmarker} I we haven't seen {disfmarker} you've - you haven't seen any final results for that
  PhD C: And they used {disfmarker} the main thing that {disfmarker} that they used was spectral subtraction?
  Professor D: yeah.
  PhD C: Or
  Professor D: There is a couple pieces to it. There's a spectral subtraction style piece {disfmarker} it was basically, you know, Wiener filtering. And then {disfmarker} then there was some p some modification of the cepstral parameters, where they {disfmarker}
  PhD A: Yeah, actually, something that's close to cepstral mean subtraction. But, uh, the way the mean is adapted {disfmarker} um, it's signal dependent. I'm {disfmarker} I'm, uh So, basically, the mean is adapted during speech and not during silence.
  Professor D: Yeah.
  PhD A: But it's very close to {disfmarker} to cepstral mean subtraction.
  Professor D: But some people have done {vocalsound} {pause} exactly that sort of thing, of {disfmarker} of {disfmarker} and the {disfmarker} I mean it's not {disfmarker} To {disfmarker} to look in {pause} speech only, to try to m to measure these things during speech,
  PhD A: Yeah, yeah.
  Professor D: that's p that's not that uncommon. But i it it {disfmarker} so it looks like they did some {disfmarker} some, uh, reasonable things, uh, and they're not things that we did, precisely. We did unreasonable things, {vocalsound} which {disfmarker} because we like to try strange things, and {disfmarker} and, uh, and our things worked too.
  PhD C: Hmm.
  Professor D: And so, um, uh, it's possible that some combination of these different things that were done would be the best thing to do. But the only caveat to that is that everybody's being real conscious of how much memory and how much CPU they're using
  PhD C: Mm - hmm.
  Professor D: because these, {vocalsound} {vocalsound} {vocalsound} uh, standards are supposed to go on cell phones with m moderate resources in both respects.
  PhD C: Did anybody, uh, do anything with the models as a {disfmarker} an experiment? Or
  Professor D: Uh, they didn't report it, if they did.
  PhD C: N nobody reported it?
  Professor D: Yeah. I think everybody was focused elsewhere. Um, now, one of the things that's nice about what we did is, we do have a {disfmarker} a, uh {disfmarker} a filtering, which leads to a {disfmarker} a, uh {disfmarker} a reduction in the bandwidth in the modulation spectrum, which allows us to downsample. So, uh, as a result of that we have a reduced, um, transmission rate for the bits.
  PhD C: Mm - hmm.
  Professor D: That was misreported the first time out. It {disfmarker} it said the same amount because for convenience sake in the particular way that this is being tested, uh, they were repeating the packets. So it was {disfmarker} they were s they {disfmarker} they had twenty - four hundred bits per second, but they were literally creating forty - eight hundred bits per second, {vocalsound} um, even though y it was just repeated.
  PhD C: Oh. Mm - hmm. Right.
  Professor D: So, uh, in practice
  PhD C: So you could've had a repeat count in there or something.
  Professor D: Well, n I mean, this was just a ph phoney thing just to {disfmarker} to fit into the {disfmarker} the software that was testing the errors {disfmarker} channel errors and so on.
  PhD C: Oh. Oh.
  Professor D: So {disfmarker} so in reality, if you put this {disfmarker} this system in into, uh, the field, it would be twenty - four hundred bits per second, not forty - eight hundred. So, um, so that's a nice feature of what {disfmarker} what we did. Um, but, um, well, we still have to see how it all comes out.
  PhD C: Hmm.
  Professor D: Um, and then there's the whole standards process, which is another thing altogether.
  PhD C: When is the development set {disfmarker} I mean, the, uh, uh, test set results due? Like the day before you leave or something?
  Professor D: Uh, probably the day after they leave, but we'll have to {disfmarker} {vocalsound} we'll have to stop it the day before {comment} we leave.
  PhD A: Yeah, yeah. So
  PhD C: Huh.
  Professor D: I think tha I think the {disfmarker} the meeting is on the thirteenth or something.
  PhD A: Yeah, this Tuesday, yeah.
  Professor D: And, uh, they, uh Right. And the {disfmarker} the, uh, results are due like the day before the meeting or something. So
  PhD A: Yeah, probably, well
  Professor D: I th I think {disfmarker} I I think they are,
  PhD A: Yeah, well
  Professor D: yeah. So {pause} {vocalsound} um, since we have a bit farther to travel than {vocalsound} some of the others, {vocalsound} uh, we'll have to get done a little quicker. But, um, I mean, it's just tracing down these bugs. I mean, just exactly this sort of thing of, you know, why {disfmarker} why these features seem to be behaving differently, uh, in California than in Oregon.
  PhD C: Hmm.
  Professor D: Might have something to do with electricity shortage. Uh, we didn't {disfmarker} we didn't have enough electrons here and Uh, but, um Uh, I think, you know, the main reason for having {disfmarker} I mean, it only takes w to run the {disfmarker} the two test sets in {disfmarker} just in computer time is just a day or so, right?
  PhD A: Yeah,
  Professor D: So
  PhD A: it's very short interval.
  Professor D: yeah. So, I think the who the whole reason for having as long as we have, which was {pause} like a week and a half, is {disfmarker} is because of bugs like that. So Huh So, we're gonna end up with these same kind of sheets that have the {pause} the percentages and so on just for the {disfmarker}
  PhD A: Yeah, so there are two more columns in the sheets,
  Professor D: Oh, I guess it's the same sheets,
  PhD A: two. Yeah, it's the same sheets,
  Professor D: yeah, yeah {disfmarker}
  PhD A: yeah.
  Professor D: just with the missing columns filled in.
  PhD A: Yeah.
  Professor D: Yeah. Well, that'll be good. So, I'll dis I'll disregard these numbers. That's {disfmarker} that's {disfmarker} that's good.
  PhD A: So, Hynek will try to push for trying to combine, uh, different things? Or Hmm?
  Professor D: Uh, well that's {pause} um yeah I mean, I think the question is" Is there {disfmarker} is there some advantage?" I mean, you could just take the best system and say that's the standard. But the thing is that if different systems are getting at good things, um, a again within the constraint of the resources, if there's something simple that you can do Now for instance, uh, it's, I think, very reasonable to have a standard for the terminal's side and then for the server's side say," Here's a number of things that could be done." So, um, everything that we did could probably just be added on to what Alcatel did, and i it'd probably work pretty well with them, too. So, um, uh, that's one {disfmarker} one aspect of it. And then on the terminal's side, I don't know how much, um, memory and {disfmarker} and CPU it takes, but it seems like the filtering {pause} Uh, I mean, the VAD stuff they both had, right? And, um, so {disfmarker} and they both had some kind of on - line normalization, right?
  PhD A: Uh, yeah.
  Professor D: Of sorts, yeah? So {disfmarker} so, it seems like the main different there is the {disfmarker} is the, uh, filtering. And the filtering {disfmarker} I think if you can {disfmarker} shouldn't take a lot of memory to do that Uh, and I also wouldn't think the CPU, uh, would be much either for that part. So, if you can {disfmarker} if you can add those in {pause} um {pause} then, uh, you can cut the data rate in half.
  PhD A: Yeah.
  Professor D: So it seems like the right thing to do is to {disfmarker} on the {disfmarker} on the terminal's side, take what they did, if it {disfmarker} if it does seem to generalize well to German and Danish, uh, take what they did add in a filter, and add in some stuff on the server's side and {disfmarker} and {disfmarker} and that's probably a reasonable standard. Um {pause} Uh
  PhD A: They are working on this already? Because {disfmarker} yeah, Su - Sunil told me that he was trying already to put some kind of, uh, filtering in the {vocalsound} {pause} France Telecom.
  Professor D: Yeah, so that's {disfmarker} that's {disfmarker} that's what That would be ideal {disfmarker} would be is that they could, you know, they could actually show that, in fact, a combination of some sort, {vocalsound} uh, would work even better than what {disfmarker} what any of the systems had. And, um, then it would {disfmarker} it would, uh {pause} be something to {disfmarker} to discuss in the meeting. But, uh, not clear what will go on. Um, I mean, on the one hand, um, sometimes people are just anxious to get a standard out there. I mean, you can always have another standard after that, but {vocalsound} this process has gone on for a while on {disfmarker} already and {disfmarker} and people might just wanna pick something and say," OK, this is it." And then, that's a standard. Uh, standards are always optional. It's just that, uh, if you disobey them, then you risk not being able to sell your product, or {pause} {vocalsound} Uh {pause} um And people often work on new standards while an old standard is in place and so on. So it's not final even if they declared a standard. The other hand, they might just say they just don't know enough yet to {disfmarker} to declare a standard. So you {disfmarker} you {disfmarker} you will be {disfmarker} you will become experts on this and know more {disfmarker} far more than me about the tha this particular standards process once you {disfmarker} you go to this meeting. So, be interested in hearing. So, uh, I'd be, uh, interested in hearing, uh, your thoughts now I mean you're almost done. I mean, you're done in the sense that, um, you may be able to get some new features from Sunil, and we'll re - run it. Uh, but other than that, you're {disfmarker} you're basically done, right? So, uh, I'm interested in hearing {disfmarker} hearing your thoughts about {pause} where you think we should go from this.
  PhD A: Yeah.
  Professor D: I mean, we tried a lot of things in a hurry, and, uh, if we can back off from this now and sort of take our time with something, and not have doing things quickly be quite so much the constraint, what {disfmarker} what you think would be the best thing to do.
  PhD A: Uh, well Hmm Well, first, uh, to really have a look at {disfmarker} at the speech {pause} {vocalsound} from these databases because, well, we tried several thing, but we did not really look {vocalsound} at what what's happening, and {vocalsound} where is the noise, and
  Professor D: OK.
  PhD A: Eh
  Professor D: It's a novel idea. Look at the data. OK.
  PhD A: Yeah.
  Professor D: Or more generally, I guess, what {disfmarker} what is causing the degradation.
  PhD A: Yeah, yeah. Actually, there is one thing that {disfmarker} well {pause} Um, generally we {disfmarker} we think that {vocalsound} most of the errors are within phoneme classes, and so I think it could be interesting to {disfmarker} to see if it {disfmarker} I don't think it's still true when we add noise, and {vocalsound} so we have {disfmarker} I {disfmarker} I guess the confusion ma the confusion matrices are very different when {disfmarker} when we have noise, and when it's clean speech. And probably, there is much more {pause} between classes errors for noisy speech.
  Professor D: Mm - hmm.
  PhD A: And {vocalsound} so, um Yeah, so perhaps we could have a {disfmarker} a large gain, eh, just by looking at improving the, uh, recognition, not of phonemes, but of phoneme classes, simply.
  Professor D: Mm - hmm.
  PhD A: And {vocalsound} which is a s a s a simpler problem, perhaps, but {disfmarker} which is perhaps important for noisy speech.
  Professor D: The other thing that strikes me, just looking at these numbers is, just taking the best cases, I mean, some of these, of course, even with all of our {disfmarker} our wonderful processing, still are horrible kinds of numbers. But just take the best case, the well - matched {pause} uh, German case after {disfmarker} er well - matched Danish after we {disfmarker}
  PhD A: Mm - hmm.
  Professor D: the kind of numbers we're getting are about eight or nine {pause} uh {pause} p percent {pause} error {pause} per digit.
  PhD A: Mm - hmm. Yeah.
  Professor D: This is obviously not usable,
  PhD A: No.
  Professor D: right?
  PhD A: Sure.
  Professor D: I mean, if you have ten digits for a phone number {comment} I mean, every now and then you'll get it right. I mean, it's {disfmarker} it's, uh, {vocalsound} um So, I mean, the other thing is that, uh {disfmarker} And {disfmarker} and {disfmarker} a and {disfmarker} and also, um {pause} part of what's nice about this is that this is, uh, {vocalsound} um {pause} a realistic {disfmarker} almost realistic database. I mean, it's still not people who are really trying to accomplish something, but {disfmarker} but, uh, within the artificial setup, it isn't noise artificially added, you know, simulated, uh, additive noise.
  PhD A: Mm - hmm.
  Professor D: It's real noise condition. And, um, {vocalsound} the {disfmarker} the training {disfmarker} the training, I guess, is always done on the close talking
  PhD A: No, actually {disfmarker} actually the well - matched condition {pause} is {pause} still quite di still quite difficult.
  Professor D: No?
  PhD A: I mean, it's {disfmarker} they have all these data from the close mike and from the distant mike, {vocalsound} from different driving condition, open window, closed window,
  Professor D: Yeah.
  PhD A: and they take all of this and they take seventy percent, I think, for training and thirty percent for testing.
  PhD E: Mm - hmm.
  PhD A: So, training is done {vocalsound} on different conditions and different microphones, and testing also is done {pause} on different microphone and conditions. So, probably if we only take the close microphones, {vocalsound} I guess the results should be much much better than this.
  Professor D: I see.
  PhD A: Mmm.
  Professor D: Oh, OK,
  PhD A: Uh
  Professor D: that explains it partially. Wha - what about i in {disfmarker} so the {disfmarker} the {disfmarker}
  PhD A: Yeah, so {disfmarker} there is this, the mismatched is, um {pause} the same kind of thing,
  Professor D: go ahead.
  PhD A: but {pause} the driving conditions, I mean the speed and the kind of road, is different for training and testing, is that right?
  PhD E: Yeah.
  PhD A: And the last condition is close microphone for training and distant for testing. Yeah.
  Professor D: Uh, OK,
  PhD A: So {disfmarker} {vocalsound} s so {disfmarker}
  Professor D: so I see. So, yeah, so the high {disfmarker} so the {disfmarker} right {disfmarker} so the highly mismatched {vocalsound} case {pause} is in some sense a good model for what we've been, you know, typically talking about when we talk about additive noise in {disfmarker} And so {disfmarker} and i i k it does correspond to a realistic situation in the sense that, {vocalsound} um, people might really be trying to, uh, call out telephone numbers or some or something like that, in {disfmarker} in their cars
  PhD A: Yeah.
  Professor D: and they're trying to connect to something.
  PhD A: Mmm.
  Professor D: Um
  PhD A: Actually, yeah, it's very close to clean speech training because, well, because the close microphone {vocalsound} and noisy speech testing,
  Professor D: Yeah. Yeah.
  PhD A: yeah. Mmm.
  Professor D: Yeah. And the well - matched condition {pause} is what you might imagine that you might be able to approach, if you know that this is the application. You're gonna record a bunch on people in cars and so forth, and do these training. And then, uh, when y you sell it to somebody, they will be a different person with a different car, and so on. So it's {disfmarker} this is a an optim somewhat optimistic view on it, uh, so, you know, the real thing is somewhere in between the two.
  PhD A: Yeah.
  Professor D: Uh, uh, but
  PhD A: But the {disfmarker} I mean, the {pause} th th
  Professor D: Even the optimistic one is
  PhD A: it doesn't work.
  Professor D: Yeah,
  PhD A: It {disfmarker}
  Professor D: right. Right, it doesn't work. So, in a way, that's, you know, that's sort of the dominant thing is that even, say on the development set stuff that we saw, the, uh, the numbers that, uh, that Alcatel was getting when choosing out the best single numbers, {vocalsound} it was just {disfmarker} you know, it wasn't good enough for {disfmarker} for {pause} a {disfmarker} a {disfmarker} for a real system.
  PhD A: Mmm. Mm - hmm.
  Professor D: You {disfmarker} you {disfmarker} you, {vocalsound} um So, uh, we still have stuff to do.
  PhD A: Yeah.
  Professor D: Uh, and, uh I don't know So, looking at the data, where, you know {disfmarker} what's the {disfmarker} what's {disfmarker} what's th what's characteristic i e yeah, I think that's {disfmarker} that's a good thing. Does a any you have any thoughts about what else {vocalsound} y you're thinking that you didn't get to that you would like to do if you had more time? Uh
  PhD E: Oh, f a lot of thing. Because we trying a lot of s {pause} thing, and we doesn't work, {vocalsound} we remove these. Maybe {vocalsound} we trying again with the articulatory feature. I don't know exactly because we tried {disfmarker} we {disfmarker} some {disfmarker} one experiment that doesn't work. Um, forgot it, something {pause} I don't know exactly
  Professor D: Mm - hmm.
  PhD E: because, tsk {comment} {vocalsound} maybe do better some step the general, {vocalsound} eh, diagram.
  Professor D: Mm - hmm.
  PhD E: I don't know exactly s to think what we can improve.
  Professor D: Yeah, cuz a lot of time it's true, there were a lot of times when we've tried something and it didn't work right away, even though we had an intuition that there should be something there. And so then we would just stop it. Um And, uh, one of the things {disfmarker} I don't remember the details on, but I remember at some point, when you were working with a second stream, and you tried a low - pass filtering to cepstrum, in some case you got {disfmarker}
  PhD E: MSG Yeah.
  Professor D: Well, but it was {comment} an MSG - like thing, but it wasn't MSG, right? Uh, you {disfmarker} y I think in some case you got some little improvement, but it was, you know, sort of a small improvement, and it was a {disfmarker} a big added complication, so you dropped it. But, um, that was just sort of one try, right? You just took one filter, threw it there,
  PhD A: Yeah,
  Professor D: right? And it seems to me that, um, if that is an important idea, which, you know, might be, that one could work at it for a while, as you're saying.
  PhD A: Hmm.
  Professor D: And, uh Uh, and you had, you know, you had the multi - band things also, and, you know, there was issue of that.
  PhD A: Yeah,
  Professor D: Um, Barry's going to be, uh, continuing working on multi - band things as well.
  PhD E: Mm - hmm.
  Professor D: We were just talking about, um, {vocalsound} some, uh, some work that we're interested in. Kind of inspired by the stuff by Larry Saul with the, uh {pause} uh, learning articulatory feature in {disfmarker} I think, in the case of his paper {disfmarker} with sonorance based on, uh, multi - band information where you have a {disfmarker} a combination of gradient learning an and, uh, EM.
  PhD A: Mm - hmm.
  Professor D: Um, and {pause} {vocalsound} {vocalsound} Um, so, I think that, you know, this is a, uh {disfmarker} this is a neat data set. Um, and then, uh, as we mentioned before, we also have the {disfmarker} the new, uh, digit set coming up from recordings in this room. So, there's a lot of things to work with. Um and, uh what I like about it, in a way, is that, uh, the results are still so terrible. Uh {pause} {vocalsound} Uh {pause} {vocalsound} I mean, they're much better than they were, you know. We're talking about thirty to sixty percent, uh, error rate reduction. That's {disfmarker} that's really great stuff to {disfmarker} to do that in relatively short time. But even after that it's still, you know, so poor that {disfmarker} that, uh, no one could really use it. So, um I think that's great that {disfmarker} because {disfmarker} and y also because again, it's not something {disfmarker} sometimes we've gotten terrible results by taking some data, and artificially, you know, convolving it with some room response, or something {disfmarker} we take a very {disfmarker} Uh, at one point, uh, Brian and I went downstairs into the {disfmarker} the basement where it was {disfmarker} it was in a hallway where it was very reverberant and we {disfmarker} we made some recordings there. And then we {vocalsound} {disfmarker} we, uh {disfmarker} uh, made a simulation of the {disfmarker} of the room acoustics there and {disfmarker} and applied it to other things,
  PhD A: Mm - hmm.
  Professor D: and uh But it was all pretty artificial, and {disfmarker} and, you know, how often would you really try to have your most crucial conversations in this very reverberant hallway? Um {pause} So, uh {pause} This is what's nice about the Aurora data and the data here, is that {disfmarker} is that it's sort of a realistic room situation {pause} uh, acoustics {disfmarker} acoustic situation, both terms in noise and reflections, and so on and n n And, uh, uh, with something that's still relatively realistic, it's still very very hard to do very well. So Yeah.
  PhD A: Yeah, so d well Actually, this is {disfmarker} tha that's why we {disfmarker} well, it's a different kind of data. We're not {disfmarker} we're not used to work with this kind of data. That's why we should have a loo more closer look at what's going on.
  Professor D: Yeah.
  PhD E: Mm - hmm.
  PhD A: Um Yeah. So this would be the first thing, and then, of course, try to {disfmarker} well, {vocalsound} kind of debug what was wrong, eh, when we do Aurora test on the MSG {pause} particularly, and on the multi - band.
  Professor D: Yeah. Yeah. Yeah.
  PhD A: Uh
  Professor D: Yeah. Yeah. No, I {disfmarker} I think there's lots of {disfmarker} lots of good things to do with this. So Um So let's {disfmarker} I guess {pause} You were gonna say something else? Oh, OK. What do you think?
  PhD C: About
  Professor D: Anything
  PhD C: About other experiments? Uh, now, I'm interested in, um, uh {pause} looking at the experiments where you use, um {pause} uh, data from multiple languages to train the neural net. And I don't know how far, or if you guys even had a chance to try that, but {pause} that would be some it'd be interesting to me.
  PhD A: Yeah, but
  Professor D: S b
  PhD A: Again, it's the kind of {disfmarker} of thing that, uh, we were thin thinking {disfmarker} thinking that it would work, but it didn't work. And, eh, so there is kind of {disfmarker} of {pause} not a bug, but something wrong in what we are doing, perhaps.
  Professor D: Yeah.
  PhD C: Right. Right.
  PhD A: Uh, something wrong, perhaps in the {disfmarker} just in the {disfmarker} the fact that the labels are {disfmarker}
  PhD C: Right.
  PhD A: well
  PhD C: Mm - hmm.
  PhD A: What worked best is the hand - labeled data.
  PhD C: Mm - hmm.
  PhD A: Um Uh, so, yeah. I don't know if we can get some hand - labeled data from other languages.
  PhD C: Yeah.
  PhD A: It's not so easy to find.
  PhD C: Right.
  PhD A: But {pause} that would be something interesting t to {disfmarker} to see.
  PhD C: Yeah, yeah.
  Professor D: Yeah. Also, uh, {vocalsound} I mean, there was just the whole notion of having multiple nets that were trained on different data. So one form of different data was {disfmarker} is from different languages, but the other Well, i in fact, uh, m in those experiments it wasn't so much combining multiple nets, it was a single net that had different
  PhD A: Yeah.
  Professor D: So, first thing is would it be better if they were multiple nets, for some reason? Second thing is, never mind the different languages, just having acoustic conditions rather than training them all up in one, would it be helpful to have different ones? So, um That was a question that was kind of raised by Mike Shire's thesis, and on {disfmarker} in that case in terms of reverberation. Right? That {disfmarker} that sometimes it might be better to do that. But, um, {vocalsound} I don't think we know for sure. So, um Right. So, next week, we, uh, won't meet because you'll be in Europe. Whe - when are you two getting back?
  PhD E: Um, I'm
  PhD A: You on Friday or S on Saturday or {pause}?
  PhD E: Sunday
  PhD A: S oh yeah, Sunday, yeah.
  PhD E: because it's {disfmarker} it's less expensive, the price {disfmarker} the price the ticket.
  PhD C:
  Professor D: Yeah, that's right. You've gotta S have a Saturday overnight, right?
  PhD A: I'll be back on Tuesday.
  Professor D: Tuesday.
  PhD C: Where {disfmarker} where's the meeting?
  Professor D: Uh, Amsterdam, I think, yeah?
  PhD A: Yeah, Amsterdam.
  PhD C: Uh - huh.
  Professor D: Yeah. Yeah, yeah. Yep. Um {pause} So, we'll skip next week, and we'll meet two weeks from now. And, uh, I guess the main topic will be, uh, you telling us what happened.
  PhD A: Yeah.
  PhD E: Yeah.
  Professor D: Uh, so Yeah, well, if we don't have an anything else to discuss, we should, uh, turn off the machine and then say the real nasty things.
  PhD C: Should we do digits first?
  PhD A: Yeah.
  Grad B: Oh, yeah, digits.
  Professor D: Oh yeah, digits! Yeah. Good point. Yeah, good thinking. Why don't you go ahead.
  PhD C: OK. OK.
output: The worst systems were still reducing errors between a third to half of the time. Though, there was a system in Aurora that did not do very well and the Alcatel systems seemed to do better.