Hi and welcome to data futurology the podcast where you can get the skills required to take your career in the data space up to the next level in this podcast. We speak with top industry leaders out there in the world today to learn from their perspectives and their past mistakes. My name is Felipe Flores. I am your host. Thank you so much for being here. I hope that you are having a wonderful week today. We have a different type of episode. This is a presentation that I did. did at the chief data and analytics officer conference in Canberra, and it is on explainable AI so in this one I discussed what is explainable the I what is why it is important how you can start delving into explainable AI discussing some of the libraries and some of the approaches and then we talked about how it's going to become more important in the future as always if you enjoyed the episode, please share it with one friend and also, Please send me your favorite quote from the episode as with all episodes. Send me your favorite quote on LinkedIn Facebook Twitter Instagram on the website any of those. Thank you so much for tuning in. And here's a presentation on explainable a I hope you enjoy good and my design good morning. Hello. Obviously, I'm not from government. So thank you so much for having me here in the last day and a half almost. It's been really great hearing all the applications all the good work. We've done in government learning so much about the all your different agencies and realizing that the challenges that you guys are facing in government are so similar to the ones that we have an industry. So I've been taking a lot of notes around what you guys are doing. And with this talk. I hope to bring you some practical tools on what we're doing especially around the avoiding ethical pitfalls, which is something that we're all sort of starting to face against very quickly. I'm working at Liberty Financial for about six months now, Financial is a group of about 12 finance companies. All companies do lending or insurance. And these are some of the lending products Insurance life insurance tpd income protection and things like that. The reason why I came across is to centralize the data teams so that engineering that assigns Etc centralize them and uplift the capability across the 12 companies. So close the group. I also will interfere with the data science Melbourne Meetup Group, which is now the fourth largest Meetup Group in the world. And we run events in Melbourne for the community and it's a bunch of I sometimes technical presentations and sometimes more strategic ones. And the other thing that I do is I run this podcast called data futurology, which is a data science podcast with over 10,000 listeners worldwide. And in that one I speak to a leaders in the data space around the applications the good work that they're doing and also their Lessons Learned what they could teach the next generation of data scientists and Also speak to top data scientists around the world, like people winning international data science competitions and what were their approaches and some of them have been doing it for two years. So they tell us about how they got into that assigns. And what would the course is that they did we've had guests from government including Merrick obviously did a really popular post really popular episode. We also had dr. Anthony Ray from bond, which is also really good. So today we'll be talking about sexism. Racism and bias and how we can all minimize the likelihood that we get caught up in those type of problems. Obviously. It's a big issue where we have the large tech companies being caught in this. So for example here Amazon scrapes secret AI recruiting tool that had a biased against women. Obviously terrible. This is about a an algorithm that the u.s. Government had around people that were going to reoffend and who is likely to go back into jail. And what they found down here is that it targeted specific racial groups over here. We talk about there's an article about facial recognition. It works great. If you're a male and you're white and for everyone else too bad, we had a there was a study is researchers checked Google's algorithm on how it was working when he was presenting jobs to people almost like the LinkedIn function that shows you jobs that you would be good for or when your job hunting and what they found is that the high-paying jobs were shown where it was less likely that they would be shown to women drastically less likely with high paying jobs being shown to women 11 percent of the time and 89 percent of the time being shown to males and that is even worse than the representation of women in boards and Senior positions in the US which at the moment is about that about 25% and this algorithm was showing these jobs to women Time so super super terrible and then the last example that I have is Amazon ditching their recruiting tool that it was favoring men for certain types of jobs. Obviously a huge issue. The big tech companies have been in smeared with this by making mistakes in this space and as a result, there's been a lot of a lot of research and a lot of implementations of tools that we can start to use today. So walk you through some of those and with a view of how we can start to open the black box. And start to understand where we could potentially be making mistakes and obviously correcting them before we get to this point the dirty secret with all this examples that we just shown a talked about is that their racism and the sexism is in our data. So whenever there's a new algorithm and goes on to the news and people got AA that nobody algorithm poor algorithm, you know, it was so sexist. Let's take it down or that algorithm is so racist. Let's take it down and we're not really sort of address. The problem that the racism and the sexism is in the data and it's in the data because that's how we've operated accompanies our institutions in the past. And now we're taking that data teaching algorithms how to essentially replicate our decisions and then they are showing us in a quantifiable way as we'll see that later how racist or sexist that data is. So there's some approaches in terms of limiting the amount of data that goes into the algorithm and what I'll show you now is how we can measure those are negative impacts on the the algorithm. So at least the way that will talk about this today are the ethical AI is has a component of explainable AI and that's opening up the algorithms and then as a result, there's obviously a lot of tough discussions with ethical boards that a lot of companies have started to put in place. You probably heard that Google's ethical board lasted about seven days and then it was knocked back because of the pushback from the organization and externals on the type of people that were sitting in the board and the type of power that they were going to have so there's obviously the alone Russians on this tough talk side. And obviously there's a lot of progress happening in the government as well. So let's talk about the explainable AI side and how we go from this black box to ideally something more like that traditionally when we think of Black Box algorithms and the reason why we go with those is because they have a higher degree of accuracy as we use algorithms that are more interpretable. They accuracy starts to decrease and that sort of accepted reality of our industry and what we're trying to do. Now is that to create algorithms that are up here so much more open much more accurate. And what's happening is that there's a lot of developments on top of the algorithms that exist today that are taking these types of algorithms in terms of accuracy moving them across and making them more open. So I have a couple of examples that are sort of generic and something that you would have encountered as a user or financial services and I tried to make it both open and that's essentially Joe. Vacations insurance claims and loans and we'll talk about these three ways to tackle it. Y1 is an approach that you can take that easy that you could literally do this within a week. And then these two are packages that I'll describe a little bit more that are open source this one both in our and - and then this one in Python and I'll show a little bit worried a little bit about how they work and how you can use them to your advantage. So on the job applications what happens is you can take an CV and as you start to scale Gail up the number of applications that you have and you start to get two sizes of large companies or governments. There is definitely a need for an automated decision making and what starts to happen is that sometimes as this sort of bias starts to creep in if you start uncover some of the inputs that are leading that decision you can get to places where it's been decided on name on address and even on University and for example, somebody with a name like Felipe Might be scored lower than somebody with a name like Tyler and then obviously some of the tough discussions is that fair is that not as that of people should we look to bump up the university scores or something like that from say people from Latin America versus people who are from here. That's the type of tough discussions that we're heading into because of the inequality that we have in the world today and the algorithms being able to show it. So I really easy way to start to get over. This is to lean into it and predict the bias. So in machine learning at least in the supervised supervised machine learning when we give the algorithm the answer that we're looking for and we get it to learn from lots of examples. We're trying to say which customer is going to be most profitable or which applicant is going to be most likely to perform. Well at this job thinking about tackling the bias angle we can start to say algorithm take the CV data and try to predict the gender of the person. Try to predict the race of the person and obviously we would give them the examples. Once you have that from a lot of the algorithms. You can say, what are the inputs that you're taking into consideration to determine race and a simple way to do it is then to go back to your data and remove those key inputs that its flagging as predictors of race. Once you've removed those inputs. You can take your new data set without those inputs and then try to predict the job performance once again, so that's a sort of quick and easy way to set to remove the bias in terms of race and sexism from our data sets today with approaches that we have today. Now, we'll go into using or the eggs and the two examples that I have left are using unstructured data. So using text and images but these approaches work just as well with structured data. So essentially The Columns and the databases that were used to dealing with when thinking about images a lot of the algorithms that look at images are mostly black books and We heard for example deep learning the blackest of of boxes and then there's other approaches that are less black boxy and a good way to look at to understand how the algorithm is doing is by looking at the accuracy. How accurate is this model and here we're trying to predict do an image classification to see whether their image that's presented. Is it a wolf or a Husky and then here we have the what the model is predicting and what the actual answer is. So we say the models predicting wolf. It's actually a wolf in this image. And then here we have D husky so that's correct wolf wolf, it's correct. Here's a mistake. This is correct. This is correct. So when we look at it from an accuracy perspective we go. You know what this algorithms actually doing really? Well one of the six not too bad then with what I'll show you next is we can start to see exactly what are the components of the image. So the algorithm is focusing on to make that decision and what we see in this case is that when we're predicting Wolf the algorithm is focusing on snow and when Addicting husky. It's focusing on parts of the animal. So if you go through wolf a lot of focus on this now over here wolf a lot of focus on this now this part the second part of the image is what the algorithm is focusing on and over here because this husky was in the snow. It was predicted to be a wolf. So if you look at just the accuracy looks great and to try and differentiate these two races and then what you get is that it's actually just look at the snow. So obviously a terrible outcome and things that we want to know about One library that had can help us uncover this and that's how the the work that I just shown was done is called lime and lime stands for this and essentially what it's doing is running a bunch of very open models so interpret bro, and it's obviously a difficult word for me because I speak Spanish as their first language and this is when the tough words to say in English when you make the transition, but what this does is it runs this package runs multiple models that are quite open and quite transparent. And it automatically Compares it to very black box models in terms of accuracy and performance and then it chooses the ways of the open models to make the predictions because it can tell you exactly how that works this Library lime at the moment. It only does classifications. So not regressions in machine learning classification is when we are trying to pick from a category save you thinking about fraud you can say this transaction was fraudulent or not fraudulent or as we saw before the The categories were husky or wolf and the algorithm can pick it doesn't have to be too. It can be 10 or 20 and the more categories you have the more data you need but then classification is picking one out of a category. The other big type of machine learning models are regressions in which it gives you a number typically a probability between 0 and 1 or you can think about it as a percentage as well and this package line works for the classifications and it came out of this study done at MIT which are explaining the predictions of any classifier. They really good package. So if you're trying to do an insurance claim and trying to understand your bias there the algorithm will show you whether you are pro or against people that have pink bathtubs as you get sort of the pictures of the insurance claims and always I'm not gonna judge you have pink bathtubs all good to you and that might be positive or negative in terms of your insurance claim good for time loan application. And then this one is a tech space so similar to the previous example give you the non-financial. Anshel, the library and in the financial example, the non-financial here is using this package called li5. I'll explain it a little bit more in the next slide. It's processing text. And in this case to predict religion and they had lots of emails between people and they were trying to identify our trying to predict religion. Obviously one of the biases that we need to be careful with and what they found here is that they could predict it pretty well, but then when they started to look into what were the components that Them about the prediction accuracy. They saw that because these are emails that had the email addresses and they saw that the dot edu was a huge predictor of people who were not religious or atheist. So essentially saying people who work as researchers in universities are very likely to be non-religious. And is that dangerous territory for one of our applications? Is that a type of bias that we want to avoid in our algorithms? It depends on the application. But the idea here is that you can do this level of transparency in your algorithms by using this package. So essentially tells you in text and unstructured data exactly which are the words that are driving the prediction. So this the library is called li5 and it was named after so an internet meme that stands for explain it like I'm five and from the meme then there was a lot of Articles being written in this style. So if you say Elie, To the acronym li5 quantum physics, you will find articles that explain quantum physics. Like I'm a five-year-old and then the idea is that you go through this and you go great. I get it. I get this really tough Concept in a very simple way. So the package is leveraging the same understanding. So when you do look for this one, you do have to go Eli five python or la-5 package and then it'll take you to the specific part of a package as I mentioned before this one works with a lot of other libraries that That are for machine learning and library in Python including scikit-learn which is one that does many many types of machine learning models and XC boots and light GBM that are more specific ones XG boost has been extremely successful in the last few years on international data science competitions, many many of the winners have been using this package because it runs over the data hundreds or thousands of times getting more and more accurate every time so it's automating some of the work. Emmanuel that a data scientist would do you manually in terms of working on an algorithm over time for say a year this package looks to automate a lot of that and but it's very slow and then light GB m is a version of this one which is a lot faster and the CLI 5 uses one or many of those other packages and compares the Black Box version against multiple open algorithms. So then it can tell you how it's making the decisions and this is another example in where you're trying to classify. By the type of animal and then you have sort of the mathematical explanation over here and then you can have the visual representation of what the algorithm is focusing on to say that this one is classified as a frog with this level of certainty. I mentioned that this one or both of these packages like many other five they work with unstructured data and also with structured data. This is an example of structured data where the input data set was the list of people that boarded the Titanic and then they were labeled in terms of whether they Survived or not? Then this was run through li5 using extra boost. So typically a very Black Box model and then we started to see how the model is making the decisions with the la-5 and we say that if for the female passengers that were very likely to survive and obviously we know this is a rule of thumb of Women & Children First. So that's something that we is known and we understand and it's a good rule of thumb the one that's not known as sort of the bias may be under current is this one which is if you were in third class you were very likely to die. So that's sort of the unspoken ways that essentially the world Works some realities of the world that the algorithms are now helping us see and we can see here that anything stronger than this is bias and this number is a positive and negative. So obviously it's saying biased positively towards female and buyers negatively against lower classes and third was the lowest you had first class second class, and then the third was sort of the cattle class. So Eli, Five does classifications regressions across images text and in structured data, unstructured and structured very powerful package in Python. I think I'll skip through this one. This is just looking at a bank statement also text it can start to highlight the type of transactions that people are doing so just to wrap up I'll talk about the legislation that is coming sort of worldwide. There's a little movement in this space from any government and the citizens expectations. The one that's most famous so far is GDP. ER so from Europe that in Article 13 to 15, they mention a right to an explanation when an algorithm makes a decision that affects your life. So say a job application a loan application and insurance claim like they are those directly affect your life. And with the scale that we're hitting in the world all these are being automated. How do we go to explain it? Now? We have some of these packages that I mentioned and obviously Europe was just I think just the first mover and they're all different countries are going to put their own. license and standards and requirements and then we're going to be in this world that if we the more countries that we operate in then the more so legislations that we need to abide by and just at the end so xai so explainable AI is there it's more ethical in the sense that it's more transparent, but it can also help us to make Better Business decisions and by that, I mean that uncovering the ways that the algorithm is making that decision can help us avoid blunders as the examples that we saw, but it can Also inform Us in terms of how the algorithm is crunching through the data and how its perceiving it and making those decisions and we can learn from that and then come up with new hunches or new hypotheses that we would like to test and have sort of this virtual circle with the algorithm the expandable AI helps us with trust obviously being being a lot more open. We can verify that the predictions are being accurate and done in a way that we expect helps us manage expectations. So these Rising citizens expectations and as I just mentioned it helps us improve by being able To open up the way that these decisions are being made work with business with the experts with the stakeholders that know that the domain problem and show them the types of the inputs of the algorithms are using to make these decisions and we can all learn from that. That is all. Thank you very much. That brings this episode to conclusion. Thank you so much for listening. Please find us on data futurology.com or on Facebook Twitter LinkedIn or Instagram as that of utility. Also go to data futurology.com forward size podcast to find the show notes for this and any other episodes. If you liked this episode it would mean a lot to us. If you could leave us a review wherever you listen to our podcast. I hope you enjoyed this episode and that it was helpful and valuable for you. Thanks again and see you next time.
Today we have a different type of episode, this is a presentation that Felipe did at the Chief Data and Analytics Officer Conference in Canberra, and it is on explainable AI. First, Felipe explains how Amazon used a secret AI recruiting tool that had a bias against women. Also, the U.S. government used an algorithm predicting how likely people in the criminal justice system would reoffend. What they found is that it targeted specific racial groups. The algorithm isn’t racist or sexist, the data is. Regarding job applications, as your company scales up, the need to automate the process of looking at the applications becomes necessary. Sometimes, bias will creep into the automated decision-making algorithm. The bias can even be narrowed down to the person’s name. For example, somebody with name Felipe might get scored lower than somebody with the name Tyler. Lean into the inequality and predict the bias. You can plug in the CV information, and ask the algorithm to predict the person’s race and gender. Then, find out what key inputs they are flagging to determine this and remove them from the algorithm. Then, Felipe explains how algorithms can tackle unstructured data approaches. When discussing images, an algorithm was able to correctly identify a wolf from a husky 5 out of 6 times. However, when uncovering how the algorithm determined which was which, it was merely looking at if the animal was in the snow or not. If the picture had snow in it, then it must be a wolf. To determine how this algorithm was functioning, Felipe used LIME - Local Interpretable Model-Agnostic Explanations. It works for classifications and came out of a study from MIT. Later, Felipe discusses using EL15 and how transparency is essential for the public to understand how the algorithms could affect them. Enjoy the show! We speak about:   [03:40] Large companies and their biases   [05:40] Racism and sexism is in our data  [08:45] Uncovering inputs of the bias     [10:45] Unstructured data approaches   [14:30] Using ELI5   [19:20] The right to an explanation   Quotes:   “We teach our algorithms on how to replicate our decisions.”  “The algorithms show the inequality that we have in the world today.”   “Explainable AI is more ethical in the sense that it is more transparent.”   “Explainable AI helps us avoid blunders and informs us how the algorithm perceives the data.”   Thank you to our sponsors:  UNSW Master of Data Science Online: studyonline.unsw.edu.au  Datasource Services: datasourceservices.com.au or email Will Howard on will@datasourceservices.com.au  Fyrebox - Make Your Own Quiz! Thank you so much for listening. Enjoy the show! 