WebRTC and speech recognition services with Adherasion
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Pradovera, Luca
In the modern real time communication world, WebRTC is being paired withmultiple channels and services to provide an unparalleled customer experience.Utilizing automated speech recognition allows us to build advanced assistiveservices such as real time captioned phone calls for hearing impaired people,simultaneous translation tools and automated bot assistants. All of that canbe achieved leveraging modern open source software such as Adhearsion, theRuby telephony application framework, FreeSWITCH, Flite, PocketSphinx and(optionally) external APIs such as IBM Watson. Our experience in developingthat kind of service will be presented and discussed, plus code samples and alive demo. Wrapping FreeSWITCH's WebRTC capabilities with Adhearsion acting as the calllogic and the "glue" between parts makes creating rich ASR-enabledapplications easier. Adhearsion allows you to leverage the full power of Rubyto access APIs, send messaging and updates, and use databases. IntegratingFreeSWITCH with third party APIs who do not provide a direct interface, suchas IBM Watson and Wit.ai, through Adhearsion and Ruby. ASR and sentimentanalysis will provide our IVRs with an unparalleled ability to not onlyunderstand what callers say, but also what their desires and intent are.Dialplan applications can be complex to set up to use those APIs, and webapplications are not well suited to that type of application. Adhearsion, theRuby voice application framework, makes it easy and fun!
 The Italian boy, Matthew, of the day. That's a bunch of us. Yeah, that's a bunch of you, four at least. So tell us about speech recognition, the hearsay, where were you seeing all the things? So, when I started preparing my talk for Fasdom, I was thinking about building something that was not usual. We usually work using various APIs, most of them are commercial, stuff like IBM Watson and Google Voice and things like that. And I decided, what if I start my talk by building an application that's based only on open source technology, including the more advanced stuff such as machine learning and stuff like that. So who am I first? My name is Luca Pereira. I'm the new lead at Mojolingo, which is a company that's been managing the addition project and the other stuff. And I'm an addition contributor, and I will tell you all about addition today. And I've been playing with phones since I was eight. My father is also a telephone engineer at Watson and Family. So, demo first. Cool, someone call James Brody while I pull up the demo, please. So, a warning, the demo doesn't actually work because we have no audio. So what I've been trying to do here is get a call into a WebRTC client, which was the part that didn't work essentially, which I touch on my presentation, I did a slide on this after, and get the call through speech recognition using pocket things, so it's free and open source software, running through free Flator on this call, NLU that does the actual bot interpretation, kind of like wheat.AR, their stuff, and then spit out the result. What I did here... Nothing I know here. I need a Chinese restaurant. I'm just piping the event browser, and essentially what we're doing is that we get a Chinese restaurant out of that. This looks simple, it is essentially simple, it's also very not polished, it's horribly complicated. So, what just happened? WebRTC going to Friswitch, check the disclaimer, so demo might not actually contain WebRTC. Friswitch is sending a call to Adirshun for control. Adirshun is using the result from CMU Sphinx to go through Reza and LU and ask Google for a restaurant. F-Lite is what is speaking the voice, F-Lite is an open source TTS engine, that's not too bad. Actually, that's one of the things I found out by building this application, is that all of these products are surprisingly good, and we're skipping over and giving money and time to big corporations. We should probably be working with some of these people in projects. That's the important message you got from building this one. Well, we're using HTTP just to go through. And this camera says, no keyboards have been harmed during the preparation of this demo. That's not actually true, I broke one just because I was so angry. So, moving part, as I said, we have Friswitch with ModVerto, Adirshun, which is the main control layer. Pockets, Sphinx and F-Lite are the voice and Reza and LU. Everybody knows what Friswitch is by now, there's been like three presentations and Giovanni is better than me in explaining what Friswitch is. So, it's a switching platform, you can use it for a variety of things. It's got very good modularity, it's easy to turn off features you don't want. And it's got very good WebRTC support through ModVerto. I'm talking about Friswitch, just because I used Friswitch to build this particular demo. Astros will be Asco, they don't have ModVerto, they just use SIP over WebRTC. And it's pretty much, right now they're feature equal, depending exactly what you need. There might be some differences, but I don't see anything that's really relevant in that. The ear and voice, as I said, are pocket Sphinx. Pocket Sphinx will be tuned for better results for starters. This guy really only understands a few hundred words, just because I'm running with a stock grammar. Just a larger grammar, I didn't have time to install that. It sounds like it works pretty well. And F-Lite gives you good TTS for the price, which is free. The voice is a bit robotic, but then again, it does speak pretty well. The interesting thing I discovered when working on this project was this library I didn't know about, which is Reza and LU. This is a machine learning and network language processing library that aims at providing a service similar to Weith.ai, Lewis.api.ai, so essentially a conversational interpretation of text. You ask it a question, and on the back end you have built a tree of entities and phrases it understands, and it will give you back an intent and subjects. What's an intent? What the person wanted to do or know, and what the subject is, is what the person wanted to do or know about. So I need the Chinese restaurant as an intent of restaurant search, in my configured example, and an object of Chinese, because I said I want the Chinese restaurant. So that's then used to do a search. This is a very, very interesting project also because it's almost batteries included. It didn't take long to set up, and the time you're taking and setting up your model can be reused on any other platform, just because they're compatible. So what did I learn building the app? Principally one thing, we need a better way to set up Friswitch or Asterisk for web or development locally. Working with various security requirements, especially the SSL certificates, is a pain that needs to be fixed, and I'm probably going to start and figure out how to fix it from Monday on. Pocket things is not as bad as the reputation it has. Pocket things is considered a toy. That is wrong, essentially wrong. It does most of what you need in a simple application, including interpreting simple grammars, so you can ask it yes, no questions. You're generally asking that kind of question in a voice application anyway. You don't want people to tell you the story of their lives and then interpret that. You don't really care if they need a counselor or they call a person, not a bot. There's value in running your own brain. Of course, there's an economical value. You're not spending money at all, and essentially the setup time is so tiny. It's probably worth just still looking to using something that you host personally, and you're contributing back to open source, and you're not giving things back to Facebook or Google or whatever. They have enough money. They don't need ours. The other important part, a very important part, is adhesion itself. All these services are disconnected in a sense from each other. Friswitch records the audio. Pocket things interprets it. NLU gives you the conversational representation of what the person said, but you need something to run the call, pick up, answer the recordings, shuttle around messages between the various parts, and that's where adhesion comes into place. So first of all, has anybody here heard about adhesion before? Woo! That's like 300% the usual amount, which is one. That's bad. Here's adhesion. It's great. It's a Ruby voice application framework. It provides third-party call control logic to telephone engines, which means we don't mess with the actual media flow. We just handle things like picking up the call, transferring, answering, recording, playing audio, collecting input, using ASR, et cetera. It connects to the two main open source platforms, which are Friswitch and asterisk, using two different mechanisms. One is RIO for Friswitch and one is AMI for asterisk. We're not on ARI, I know. We should. Version 2 is stable, and it's been stable for a while, too. We just released an update, and version 3 has been at RC1 for quite a while, because there's one bug we need to fix, and we need a sponsor for that, essentially, but we've come up soon enough. And it's backed by a foundation, so the project has guaranteed long-term viability. And so we have a few new things in Edition 3, which make it even easier to build, to use as a glue for building your own complicated applications, whether they're voice-based or under types. Friswitch support is RIO only. It doesn't use the MN socket anymore, in case anybody knows what an MN socket is. Asterisk 1100 plus is required, which is, well, I mean, right now it's okay, I think. We streamlines the internals a lot. It's 30% faster on benchmarks. And it's got a built-in HTTP server, so if you need to build a very simple dashboard to your, for example, PBX, internal telephony exchange for your office, you can have it straight on the server without any other service. And it's got native internalization support, which means you can easily play and send translated text. It's important just because we didn't have it. I know it's forgiven these days, but it wasn't there before. What can you do? So there's plugins. Plugins give you voicemail. CU.TTS. What's CU.TTS? It's a plugin I built that will play dates, numbers, and generally ordinal numbers, so first, second, third, just starting from a string with audio files it carries itself. So you can even run without any kind of TTS, and it will still try and play audio for people. There's platform-specific Fenceanitops against mostly for Asterisk. Asterisk has very good, let's say, primitive support, native support, while Friswitch is only supported through RIO. We have clustering. Login has been unified, so it's easier to deploy to Heroku and whatnot. There are all small changes by far and wide, but in general speaking, means that you can just take an additional application, deploy it to Heroku, point it at your Friswitch server, and you're done. So every phone call is an actor. Probably someone asserted about the actor model. Some people haven't. Has anybody here used Erlang or Ledexer? Same people from before. Yeah, so anyways, the idea is that every call is isolated. If a call crashes, the process doesn't go down. So we're falling back into what Giovanni mentioned earlier. So if a call goes down, is that a big deal, or someone gets your system doesn't go down? So the goal is to have something that always runs. Messages are passed around as events, and each call runs the handling logic and the extra thread, which is a technical term to say if the call dies, only the call dies. The rest stays up. So as I said, controllers group up features. So it's kind of like Rails or any MVC application, really. There's a controller for everything you want people to do. So the demo I run is one controller that asks the person for the restaurant, checks for the, if there's valid input, and then looks for the restaurant and sends the URL back to the browser. Routing, kind of like a web app, controls which call goes to which controller, using, I don't know, could be a phone number, could be time of the day. So the usual time of the day thing when your office is open or closed. But it's far more flexible than using the dial plan because you can, this is a normal Ruby application. You can access anything you want inside the application. There's an event handler to handle as sync messages, and it's generally based on celluloid, which makes it behave similar to what an underlying application is. And there are DSLs for all common operations, so you can really replace your dial plan. Quickening on the RIO protocol, the RIO protocol is an XMPP extension that we use to communicate with Friswitch. It's interesting because it's called built-in load balancing, so it scales very well. Actually, addition scales better on Friswitch than Asterisk, thanks to RIO. As a side effect, every addition has its own XMPP address. You can use to control the instance or send out events or participate in chats. So that's not really restricted to Friswitch. It will just connect to Friswitch using RIO, but it will also have, okay, oh, yeah, eight, perfect. So addition on Asterisk has no RIO support. There's no RIO on Asterisk. It doesn't really matter. It connects via AMI. We're not using ARI yet. Again, sorry, we'll get there. There's much better native command support. So if you have a platform that's based on a very complicated Asterisk install with a giant dial plan with thousands of lines and context skips and whatnot, you'd better serve moving it to Adirshino right now because it will be at least one tenth of the lines of code in Ruby. It's slightly easy to get started because of configuration a couple reasons. So if you check in Adirshino, try Asterisk first. What can you do? As I said, calls, conferences, media, drive various types of speech recognition, commercial engines that use special grammars that are XML formats, build very complex IVRs from simply press one if you need to do this or press two to do that. We have IVRs recording anything from flight reservations to people placing complaints because they didn't receive their newspaper today to really anything else. And you can connect to a database. Deployment is, I'll speed up a little just to get to a few other interesting slides. It's deployed on any really, any Ruby flavor. It's usually one-on-one with Friswitch and Asterisk. So Friswitch now you can scale. It's sort of decoupled. You can have multiple nodes with multiple Friswitch nodes. It's easier to scale provided you have a load balancer. So it makes it everything easier to scale. Quickly, this is an XML dial plan for Friswitch. And this is, it's like a board with a nail through it. It's good, but it will only be worth for some things. And it's difficult to adapt. The controller code is not, well, we can read it of course, but you can't need the slides. It's, controller code is not necessarily shorter. So it doesn't look like it's any simpler, but keep in mind that this normal Ruby code, you can literally access anything a Ruby application can. So anything from a common line application to APIs to whatever you need. We used this to push events to the browser, as you saw, was just connecting to a web socket and pushing events to the browser right away without any intermediate process. Who's using edition? Just to give you a few examples, these guys have an interesting application based on asterisk if I'm not wrong, that does, so you're, you need a doctor. You don't know who to call. You're new in the area. They give you a number. You call that number. They put you through doctor to start their own call or available, depending on a complex routing table. For example, that's the kind of application that will be a pain to build in just in the dial plan or just using more HTTP curl, complicated stuff like that. Just because you will have to adapt your logic to how asterisk and Friswitch want you to build it. While here, what you do is just keep your logic in addition. All you do is a simple SQL query, manipulate whatever you need to do, generate specific messages for the different doctors, and just handle the call in an easier and more integrated way. Again, we're not doing anything with the media flows or signaling or whatnot. This is just call control. LiveConnect. This is the newest project we have. This is WebRTC based. This uses Friswitch to broadcast operating room surgery. Why is this SIP based? Because we got a very strange piece of hardware that's FDA approved in the U.S. and we could only use that. And that's a very weird thing because it connects through SIP to send out video. I don't know who came up with that. We had to use that just because changing that would have meant the millions of dollars invested and whatnot. And so what we do is that client calls Friswitch server and we just replicate the call over to WebRTC. People, they call him. In this case, what the challenge was is managing security. You have to be very sure that you as a student, you're only allowed to join a certain surgery during a certain time and not other times just to, you know, sort of all sensible data and stuff. So again, in this case, Adition allows you to very easily manage access to the conferences by going through the database. We actually check with, I think it's a, I don't remember which big CRM application is handling the actual access, but the university that's running this as their own platform. And as part of checking if the user is allowed to get into a conference, we check with that API. Power home remodeling. This is just a big customer of all things. They're, well, they're a customer of us that also big users of Adition. They have a very big, they sell half a billion dollars of windows every year, which is a huge amount of windows, not windows, computer windows, like literally glass windows. And they were built out of a, essentially out of call center. So the call center has always been very central to their strategy. And I'm talking about them more about the scaling properties that Adition has than the actual functionality. In this particular case, Adition is not doing much more than just a dollar. So what it does is they have a list of people, they call those people and connect them to agents to have them confirm appointments or take it. It's very simple from an operating standpoint, but they have 400 call center operators working 24 seven and they have only one Astros box and one Adition node. And they have a list of people with another cold spare standing by. So yeah, that's not ideal, but they're what they're doing. So the, we can run a lot of calls. It doesn't add any noticable overhead. You're going to hit your Astros fridge single box number of maximum number of calls before you hit the Ruby maximum number of calls in this case. Well, there's a few other, as I said, the publishing system. They do complaints and stuff. There's one MVNO. Oh, I have guess which one, but no one knows which one. It's ring plus in the US just because there's nobody here from the US side of things. I think and there's an interesting free project we have as a cultural mediator network. So people that speak a language and they've done a course by the detailing government say I'm available. I speak Hindi and I'm available to help out people who are in distress and only speak that language. And there's a writing system that the police can call a number and they will give it will give them a person speaking that language so they can put them in touch with say someone was with the police or in hospital and not speaking a language. We have cases though in which the nurse or policeman didn't know which which language the person was speaking. So there's in which case we wrap them to some people who speak a lot of languages and will try to understand what language that is. So thank you. I have to speed through a little. There's plenty of other things I could show you but main thing is addition removes a lot of the complexity of building applications like the one I sort of demoed earlier. Unfortunately, we can fix the web RTC certificate, but that's something I'll fix later. And if there's any questions, I'll be glad to answer. I should be about three minutes. Yes. Great. Hope you liked it.
 The Italian boy, Matthew, of the day.  That's a bunch of us.  Yeah, that's a bunch of you, four at least.  So tell us about speech recognition, the hearsay, where were you seeing all the things?  So, when I started preparing my talk for Fasdom, I was thinking about building something that was not usual.  We usually work using various APIs, most of them are commercial, stuff like IBM Watson and Google Voice and things like that.  And I decided, what if I start my talk by building an application that's based only on open source technology,  including the more advanced stuff such as machine learning and stuff like that.  So who am I first?  My name is Luca Pereira.  I'm the new lead at Mojolingo, which is a company that's been managing the addition project and the other stuff.  And I'm an addition contributor, and I will tell you all about addition today.  And I've been playing with phones since I was eight.  My father is also a telephone engineer at Watson and Family.  So, demo first.  Cool, someone call James Brody while I pull up the demo, please.  So, a warning, the demo doesn't actually work because we have no audio.  So what I've been trying to do here is get a call into a WebRTC client,  which was the part that didn't work essentially, which I touch on my presentation, I did a slide on this after,  and get the call through speech recognition using pocket things, so it's free and open source software,  running through free Flator on this call, NLU that does the actual bot interpretation,  kind of like wheat.AR, their stuff, and then spit out the result.  What I did here...  Nothing I know here.  I need a Chinese restaurant.  I'm just piping the event browser, and essentially what we're doing is that we get a Chinese restaurant out of that.  This looks simple, it is essentially simple, it's also very not polished, it's horribly complicated.  So, what just happened?  WebRTC going to Friswitch, check the disclaimer, so demo might not actually contain WebRTC.  Friswitch is sending a call to Adirshun for control.  Adirshun is using the result from CMU Sphinx to go through Reza and LU and ask Google for a restaurant.  F-Lite is what is speaking the voice, F-Lite is an open source TTS engine, that's not too bad.  Actually, that's one of the things I found out by building this application,  is that all of these products are surprisingly good, and we're skipping over and giving money and time to big corporations.  We should probably be working with some of these people in projects.  That's the important message you got from building this one.  Well, we're using HTTP just to go through.  And this camera says, no keyboards have been harmed during the preparation of this demo.  That's not actually true, I broke one just because I was so angry.  So, moving part, as I said, we have Friswitch with ModVerto, Adirshun, which is the main control layer.  Pockets, Sphinx and F-Lite are the voice and Reza and LU.  Everybody knows what Friswitch is by now, there's been like three presentations and Giovanni is better than me in explaining what Friswitch is.  So, it's a switching platform, you can use it for a variety of things.  It's got very good modularity, it's easy to turn off features you don't want.  And it's got very good WebRTC support through ModVerto.  I'm talking about Friswitch, just because I used Friswitch to build this particular demo.  Astros will be Asco, they don't have ModVerto, they just use SIP over WebRTC.  And it's pretty much, right now they're feature equal, depending exactly what you need.  There might be some differences, but I don't see anything that's really relevant in that.  The ear and voice, as I said, are pocket Sphinx.  Pocket Sphinx will be tuned for better results for starters.  This guy really only understands a few hundred words, just because I'm running with a stock grammar.  Just a larger grammar, I didn't have time to install that.  It sounds like it works pretty well.  And F-Lite gives you good TTS for the price, which is free.  The voice is a bit robotic, but then again, it does speak pretty well.  The interesting thing I discovered when working on this project was this library I didn't know about, which is Reza and LU.  This is a machine learning and network language processing library that aims at providing a service similar to Weith.ai,  Lewis.api.ai, so essentially a conversational interpretation of text.  You ask it a question, and on the back end you have built a tree of entities and phrases it understands,  and it will give you back an intent and subjects.  What's an intent?  What the person wanted to do or know, and what the subject is, is what the person wanted to do or know about.  So I need the Chinese restaurant as an intent of restaurant search, in my configured example,  and an object of Chinese, because I said I want the Chinese restaurant.  So that's then used to do a search.  This is a very, very interesting project also because it's almost batteries included.  It didn't take long to set up, and the time you're taking and setting up your model can be reused on any other platform,  just because they're compatible.  So what did I learn building the app?  Principally one thing, we need a better way to set up Friswitch or Asterisk for web or development locally.  Working with various security requirements, especially the SSL certificates, is a pain that needs to be fixed,  and I'm probably going to start and figure out how to fix it from Monday on.  Pocket things is not as bad as the reputation it has.  Pocket things is considered a toy.  That is wrong, essentially wrong.  It does most of what you need in a simple application, including interpreting simple grammars,  so you can ask it yes, no questions.  You're generally asking that kind of question in a voice application anyway.  You don't want people to tell you the story of their lives and then interpret that.  You don't really care if they need a counselor or they call a person, not a bot.  There's value in running your own brain.  Of course, there's an economical value.  You're not spending money at all, and essentially the setup time is so tiny.  It's probably worth just still looking to using something that you host personally,  and you're contributing back to open source, and you're not giving things back to Facebook or Google or whatever.  They have enough money. They don't need ours.  The other important part, a very important part, is adhesion itself.  All these services are disconnected in a sense from each other.  Friswitch records the audio. Pocket things interprets it.  NLU gives you the conversational representation of what the person said,  but you need something to run the call, pick up, answer the recordings,  shuttle around messages between the various parts, and that's where adhesion comes into place.  So first of all, has anybody here heard about adhesion before?  Woo! That's like 300% the usual amount, which is one.  That's bad. Here's adhesion. It's great.  It's a Ruby voice application framework. It provides third-party call control logic to telephone engines,  which means we don't mess with the actual media flow.  We just handle things like picking up the call, transferring, answering, recording, playing audio,  collecting input, using ASR, et cetera.  It connects to the two main open source platforms, which are Friswitch and asterisk, using two different mechanisms.  One is RIO for Friswitch and one is AMI for asterisk.  We're not on ARI, I know. We should.  Version 2 is stable, and it's been stable for a while, too.  We just released an update, and version 3 has been at RC1 for quite a while,  because there's one bug we need to fix, and we need a sponsor for that, essentially,  but we've come up soon enough.  And it's backed by a foundation, so the project has guaranteed long-term viability.  And so we have a few new things in Edition 3, which make it even easier to build,  to use as a glue for building your own complicated applications,  whether they're voice-based or under types.  Friswitch support is RIO only. It doesn't use the MN socket anymore,  in case anybody knows what an MN socket is.  Asterisk 1100 plus is required, which is, well, I mean, right now it's okay, I think.  We streamlines the internals a lot. It's 30% faster on benchmarks.  And it's got a built-in HTTP server, so if you need to build a very simple dashboard  to your, for example, PBX, internal telephony exchange for your office,  you can have it straight on the server without any other service.  And it's got native internalization support, which means you can easily play and send translated text.  It's important just because we didn't have it. I know it's forgiven these days,  but it wasn't there before.  What can you do? So there's plugins. Plugins give you voicemail.  CU.TTS. What's CU.TTS? It's a plugin I built that will play dates, numbers,  and generally ordinal numbers, so first, second, third,  just starting from a string with audio files it carries itself.  So you can even run without any kind of TTS, and it will still try and play audio for people.  There's platform-specific Fenceanitops against mostly for Asterisk.  Asterisk has very good, let's say, primitive support, native support,  while Friswitch is only supported through RIO.  We have clustering. Login has been unified, so it's easier to deploy to Heroku and whatnot.  There are all small changes by far and wide, but in general speaking,  means that you can just take an additional application, deploy it to Heroku,  point it at your Friswitch server, and you're done.  So every phone call is an actor.  Probably someone asserted about the actor model. Some people haven't.  Has anybody here used Erlang or Ledexer?  Same people from before.  Yeah, so anyways, the idea is that every call is isolated.  If a call crashes, the process doesn't go down.  So we're falling back into what Giovanni mentioned earlier.  So if a call goes down, is that a big deal,  or someone gets your system doesn't go down?  So the goal is to have something that always runs.  Messages are passed around as events,  and each call runs the handling logic and the extra thread,  which is a technical term to say if the call dies, only the call dies.  The rest stays up.  So as I said, controllers group up features.  So it's kind of like Rails or any MVC application, really.  There's a controller for everything you want people to do.  So the demo I run is one controller that asks the person for the restaurant,  checks for the, if there's valid input,  and then looks for the restaurant and sends the URL back to the browser.  Routing, kind of like a web app, controls which call goes to which controller,  using, I don't know, could be a phone number, could be time of the day.  So the usual time of the day thing when your office is open or closed.  But it's far more flexible than using the dial plan because you can,  this is a normal Ruby application.  You can access anything you want inside the application.  There's an event handler to handle as sync messages,  and it's generally based on celluloid,  which makes it behave similar to what an underlying application is.  And there are DSLs for all common operations,  so you can really replace your dial plan.  Quickening on the RIO protocol, the RIO protocol is an XMPP extension  that we use to communicate with Friswitch.  It's interesting because it's called built-in load balancing,  so it scales very well.  Actually, addition scales better on Friswitch than Asterisk, thanks to RIO.  As a side effect, every addition has its own XMPP address.  You can use to control the instance or send out events or participate in chats.  So that's not really restricted to Friswitch.  It will just connect to Friswitch using RIO, but it will also have, okay,  oh, yeah, eight, perfect.  So addition on Asterisk has no RIO support.  There's no RIO on Asterisk.  It doesn't really matter.  It connects via AMI.  We're not using ARI yet.  Again, sorry, we'll get there.  There's much better native command support.  So if you have a platform that's based on a very complicated Asterisk install  with a giant dial plan with thousands of lines and context skips and whatnot,  you'd better serve moving it to Adirshino right now  because it will be at least one tenth of the lines of code in Ruby.  It's slightly easy to get started because of configuration a couple reasons.  So if you check in Adirshino, try Asterisk first.  What can you do?  As I said, calls, conferences, media, drive various types of speech recognition,  commercial engines that use special grammars that are XML formats,  build very complex IVRs from simply press one if you need to do this or press two to do that.  We have IVRs recording anything from flight reservations to people placing complaints  because they didn't receive their newspaper today to really anything else.  And you can connect to a database.  Deployment is, I'll speed up a little just to get to a few other interesting slides.  It's deployed on any really, any Ruby flavor.  It's usually one-on-one with Friswitch and Asterisk.  So Friswitch now you can scale.  It's sort of decoupled.  You can have multiple nodes with multiple Friswitch nodes.  It's easier to scale provided you have a load balancer.  So it makes it everything easier to scale.  Quickly, this is an XML dial plan for Friswitch.  And this is, it's like a board with a nail through it.  It's good, but it will only be worth for some things.  And it's difficult to adapt.  The controller code is not, well, we can read it of course, but you can't need the slides.  It's, controller code is not necessarily shorter.  So it doesn't look like it's any simpler, but keep in mind that this normal Ruby code,  you can literally access anything a Ruby application can.  So anything from a common line application to APIs to whatever you need.  We used this to push events to the browser, as you saw,  was just connecting to a web socket and pushing events to the browser right away  without any intermediate process.  Who's using edition?  Just to give you a few examples, these guys have an interesting application based on  asterisk if I'm not wrong, that does, so you're, you need a doctor.  You don't know who to call.  You're new in the area.  They give you a number.  You call that number.  They put you through doctor to start their own call or available, depending on a complex routing table.  For example, that's the kind of application that will be a pain to build in just in the dial plan  or just using more HTTP curl, complicated stuff like that.  Just because you will have to adapt your logic to how asterisk and Friswitch want you to build it.  While here, what you do is just keep your logic in addition.  All you do is a simple SQL query, manipulate whatever you need to do,  generate specific messages for the different doctors, and just handle the call in an easier and more integrated way.  Again, we're not doing anything with the media flows or signaling or whatnot.  This is just call control.  LiveConnect.  This is the newest project we have.  This is WebRTC based.  This uses Friswitch to broadcast operating room surgery.  Why is this SIP based?  Because we got a very strange piece of hardware that's FDA approved in the U.S.  and we could only use that.  And that's a very weird thing because it connects through SIP to send out video.  I don't know who came up with that.  We had to use that just because changing that would have meant the millions of dollars invested and whatnot.  And so what we do is that client calls Friswitch server and we just replicate the call over to WebRTC.  People, they call him.  In this case, what the challenge was is managing security.  You have to be very sure that you as a student, you're only allowed to join a certain surgery during a certain time  and not other times just to, you know, sort of all sensible data and stuff.  So again, in this case, Adition allows you to very easily manage access to the conferences by going through the database.  We actually check with, I think it's a, I don't remember which big CRM application is handling the actual access,  but the university that's running this as their own platform.  And as part of checking if the user is allowed to get into a conference, we check with that API.  Power home remodeling.  This is just a big customer of all things.  They're, well, they're a customer of us that also big users of Adition.  They have a very big, they sell half a billion dollars of windows every year, which is a huge amount of windows,  not windows, computer windows, like literally glass windows.  And they were built out of a, essentially out of call center.  So the call center has always been very central to their strategy.  And I'm talking about them more about the scaling properties that Adition has than the actual functionality.  In this particular case, Adition is not doing much more than just a dollar.  So what it does is they have a list of people, they call those people and connect them to agents to have them confirm appointments or take it.  It's very simple from an operating standpoint, but they have 400 call center operators working 24 seven and they have only one Astros box and one Adition node.  And they have a list of people with another cold spare standing by.  So yeah, that's not ideal, but they're what they're doing.  So the, we can run a lot of calls.  It doesn't add any noticable overhead.  You're going to hit your Astros fridge single box number of maximum number of calls before you hit the Ruby maximum number of calls in this case.  Well, there's a few other, as I said, the publishing system.  They do complaints and stuff.  There's one MVNO. Oh, I have guess which one, but no one knows which one.  It's ring plus in the US just because there's nobody here from the US side of things.  I think and there's an interesting free project we have as a cultural mediator network.  So people that speak a language and they've done a course by the detailing government say I'm available.  I speak Hindi and I'm available to help out people who are in distress and only speak that language.  And there's a writing system that the police can call a number and they will give it will give them a person speaking that language so they can put them in touch with say someone was with the police or in hospital and not speaking a language.  We have cases though in which the nurse or policeman didn't know which which language the person was speaking.  So there's in which case we wrap them to some people who speak a lot of languages and will try to understand what language that is.  So thank you. I have to speed through a little. There's plenty of other things I could show you but main thing is addition removes a lot of the complexity of building applications like the one I sort of demoed earlier.  Unfortunately, we can fix the web RTC certificate, but that's something I'll fix later.  And if there's any questions, I'll be glad to answer.  I should be about three minutes.  Yes.  Great. Hope you liked it.
WebRTC_and_speech_recognition_service_0_131.png WebRTC_and_speech_recognition_service_132_923.png WebRTC_and_speech_recognition_service_924_1175.png WebRTC_and_speech_recognition_service_1176_1595.png WebRTC_and_speech_recognition_service_1596_1715.png WebRTC_and_speech_recognition_service_1716_3023.png WebRTC_and_speech_recognition_service_3024_4391.png WebRTC_and_speech_recognition_service_4392_6299.png WebRTC_and_speech_recognition_service_6300_10487.png WebRTC_and_speech_recognition_service_10488_12011.png WebRTC_and_speech_recognition_service_12012_20603.png WebRTC_and_speech_recognition_service_20604_21335.png WebRTC_and_speech_recognition_service_21336_21395.png WebRTC_and_speech_recognition_service_21396_28211.png WebRTC_and_speech_recognition_service_28212_29147.png WebRTC_and_speech_recognition_service_29148_29268.png