The following is a conversation with William McCaskill. He's a philosopher, ethicist, and one of the originators of the effective altruism movement. His research focuses on the fundamentals of effective altruism, or the use of evidence and reason to help others by as much as possible with our time and money, with a particular concentration on how to act given moral uncertainty. He's the author of Doing Good, Better, Effective Altruism, and a Radical New Way to Make a Difference. He is a cofounder and the president of the Center of Effective Altruism, CEA, that encourages people to commit to donate at least 10% of their income to the most effective charities. He cofounded 80,000 Hours, which is a nonprofit that provides research and advice on how you can best make a difference through your career. This conversation was recorded before the outbreak of the coronavirus pandemic. For everyone feeling the medical, psychological, and financial burden of this crisis, I'm sending love your way. Stay strong. We're in this together. We'll beat this thing. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcast, support it on Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F R I D M A N. As usual, I'll do one or two minutes of ads now, and never any ads in the middle that can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. When you get it, use code LEXPODCAST. Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. Since Cash App allows you to send and receive money digitally, peer to peer, and security in all digital transactions is very important, let me mention the PCI data security standard that Cash App is compliant with. I'm a big fan of standards for safety and security. PCI DSS is a good example of that, where a bunch of competitors got together and agreed that there needs to be a global standard around the security of transactions. Now, we just need to do the same for autonomous vehicles and AI systems in general. So again, if you get Cash App from the App Store or Google Play, and use the code LEXPODCAST, you get $10, and Cash App will also donate $10 to FIRST, an organization that is helping to advance robotics and STEM education for young people around the world. And now, here's my conversation with William McCaskill. What does utopia for humans and all life on Earth look like for you? That's a great question. What I want to say is that we don't know, and the utopia we want to get to is an indirect one that I call the long reflection. So, period of post scarcity, no longer have the kind of urgent problems we have today, but instead can spend, perhaps it's tens of thousands of years debating, engaging in ethical reflection in order, before we take any kind of drastic lock in, actions like spreading to the stars, and then we can figure out what is of kind of moral value. The long reflection, that's a really beautiful term. So, if we look at Twitter for just a second, do you think human beings are able to reflect in a productive way? I don't mean to make it sound bad, because there is a lot of fights and politics and division in our discourse. Maybe if you zoom out, it actually is civilized discourse. It might not feel like it, but when you zoom out. So, I don't want to say that Twitter is not civilized discourse. I actually believe it. It's more civilized than people give it credit for. But do you think the long reflection can actually be stable, where we as human beings with our descendant of eight brains would be able to sort of rationally discuss things together and arrive at ideas? I think, overall, we're pretty good at discussing things rationally, and at least in the earlier stages of our lives being open to many different ideas, and being able to be convinced and change our views. I think that Twitter is designed almost to bring out all the worst tendencies. So, if the long reflection were conducted on Twitter, maybe it would be better just not even to bother. But I think the challenge really is getting to a stage where we have a society that is as conducive as possible to rational reflection, to deliberation. I think we're actually very lucky to be in a liberal society where people are able to discuss a lot of ideas and so on. I think when we look to the future, that's not at all guaranteed that society would be like that, rather than a society where there's a fixed canon of values that are being imposed on all of society, and where you aren't able to question that. That would be very bad from my perspective, because it means we wouldn't be able to figure out what the truth is. I can already sense we're going to go down a million tangents, but what do you think is the... If Twitter is not optimal, what kind of mechanism in this modern age of technology can we design where the exchange of ideas could be both civilized and productive, and yet not be too constrained where there's rules of what you can say and can't say, which is, as you say, is not desirable, but yet not have some limits as to what can be said or not and so on? Do you have any ideas, thoughts on the possible future? Of course, nobody knows how to do it, but do you have thoughts of what a better Twitter might look like? I think that text based media are intrinsically going to be very hard to be conducive to rational discussion, because if you think about it from an informational perspective, if I just send you a text of less than, what is it now, 240 characters, 280 characters, I think, that's a tiny amount of information compared to, say, you and I talking now, where you have access to the words I say, which is the same as in text, but also my tone, also my body language, and we're very poorly designed to be able to assess... I have to read all of this context into anything you say, so maybe your partner sends you a text and has a full stop at the end. Are they mad at you? You don't know. You have to infer everything about this person's mental state from whether they put a full stop at the end of a text or not. Well, the flip side of that is it truly text that's the problem here, because there's a viral aspect to the text, where you could just post text nonstop. It's very immediate. The times before Twitter, before the internet, the way you would exchange texts is you would write books. And that, while it doesn't get body language, it doesn't get tone, it doesn't... so on, but it does actually boil down after some time of thinking, some editing, and so on, boil down ideas. So is the immediacy and the viral nature, which produces the outrage mobs and so on, the potential problem? I think that is a big issue. I think there's going to be this strong selection effect where something that provokes outrage, well, that's high arousal, you're more likely to retweet that, whereas kind of sober analysis is not as sexy, not as viral. I do agree that long form content is much better to productive discussion. In terms of the media that are very popular at the moment, I think that podcasting is great where your podcasts are two hours long, so they're much more in depth than Twitter are, and you are able to convey so much more nuance, so much more caveat, because it's an actual conversation. It's more like the sort of communication that we've evolved to do, rather than these very small little snippets of ideas that, when also combined with bad incentives, just clearly aren't designed for helping us get to the truth. It's kind of interesting that it's not just the length of the podcast medium, but it's the fact that it was started by people that don't give a damn about quote unquote demand, that there's a relaxed, sort of the style that Joe Rogan does, there's a freedom to express ideas in an unconstrained way that's very real. It's kind of funny that it feels so refreshingly real to us today, and I wonder what the future looks like. It's a little bit sad now that quite a lot of sort of more popular people are getting into podcasting, and they try to sort of create, they try to control it, they try to constrain it in different kinds of ways. People I love, like Conan O Brien and so on, different comedians, and I'd love to see where the real aspects of this podcasting medium persist, maybe in TV, maybe in YouTube, maybe Netflix is pushing those kind of ideas, and it's kind of, it's a really exciting word, that kind of sharing of knowledge. Yeah, I mean, I think it's a double edged sword as it becomes more popular and more profitable, where on the one hand you'll get a lot more creativity, people doing more interesting things with the medium, but also perhaps you get this place to the bottom where suddenly maybe it'll be hard to find good content on podcasts because it'll be so overwhelmed by the latest bit of viral outrage. So speaking of that, jumping on Effective Altruism for a second, so much of that internet content is funded by advertisements. Just in the context of Effective Altruism, we're talking about the richest companies in the world, they're funded by advertisements essentially, Google, that's their primary source of income. Do you see that as, do you have any criticism of that source of income? Do you see that source of money as a potentially powerful source of money that could be used, well, certainly could be used for good, but is there something bad about that source of money? I think there's significant worries with it, where it means that the incentives of the company might be quite misaligned with making people's lives better, where again, perhaps the incentives are towards increasing drama and debate on your social media feed in order that more people are going to be engaged, perhaps compulsively involved with the platform. Whereas there are other business models like having an opt in subscription service where perhaps they have other issues, but there's much more of an incentive to provide a product that its users are just really wanting, because now I'm paying for this product. I'm paying for this thing that I want to buy rather than I'm trying to use this thing and it's going to get a profit mechanism that is somewhat orthogonal to me actually just wanting to use the product. And so, I mean, in some cases it'll work better than others. I can imagine, I can in theory imagine Facebook having a subscription service, but I think it's unlikely to happen anytime soon. Well, it's interesting and it's weird now that you bring it up that it's unlikely. For example, I pay I think 10 bucks a month for YouTube Red and I don't think I get it much for that except just for no ads, but in general it's just a slightly better experience. And I would gladly, now I'm not wealthy, in fact I'm operating very close to zero dollars, but I would pay 10 bucks a month to Facebook and 10 bucks a month to Twitter for some kind of more control in terms of advertisements and so on. But the other aspect of that is data, personal data. People are really sensitive about this and I as one who hopes to one day create a company that may use people's data to do good for the world, wonder about this. One, the psychology of why people are so paranoid. Well, I understand why, but they seem to be more paranoid than is justified at times. And the other is how do you do it right? So it seems that Facebook is, it seems that Facebook is doing it wrong. That's certainly the popular narrative. It's unclear to me actually how wrong. Like I tend to give them more benefit of the doubt because it's a really hard thing to do right and people don't necessarily realize it, but how do we respect in your view people's privacy? Yeah, I mean in the case of how worried are people about using their data, I mean there's a lot of public debate and criticism about it. When we look at people's revealed preferences, people's continuing massive use of these sorts of services. It's not clear to me how much people really do care. Perhaps they care a bit, but they're happy to in effect kind of sell their data in order to be able to kind of use a certain service. That's a great term, revealed preferences. So these aren't preferences you self report in the survey. This is like your actions speak. Yeah, exactly. So you might say, oh yeah, I hate the idea of Facebook having my data. But then when it comes to it, you actually are willing to give that data in exchange for being able to use the service. And if that's the case, then I think unless we have some explanation about why there's some negative externality from that or why there's some coordination failure, or if there's something that consumers are just really misled about where they don't realize why giving away data like this is a really bad thing to do, then ultimately I kind of want to, you know, respect people's preferences. They can give away their data if they want. I think there's a big difference between companies use of data and governments having data where, you know, looking at the track record of history, governments knowing a lot about their people can be very bad if the government chooses to do bad things with it. And that's more worrying, I think. So let's jump into it a little bit. Most people know, but actually I, two years ago, had no idea what effective altruism was until I saw there was a cool looking event in an MIT group here. I think it's called the Effective Altruism Club or a group. I was like, what the heck is that? And one of my friends said, I mean, he said that they're just a bunch of eccentric characters. So I was like, hell yes, I'm in. So I went to one of their events and looked up what's it about. It's quite a fascinating philosophical and just a movement of ideas. So can you tell me what is effective altruism? Great, so the core of effective altruism is about trying to answer this question, which is how can I do as much good as possible with my scarce resources, my time and with my money? And then once we have our best guess answers to that, trying to take those ideas and put that into practice, and do those things that we believe will do the most good. And we're now a community of people, many thousands of us around the world, who really are trying to answer that question as best we can and then use our time and money to make the world better. So what's the difference between sort of classical general idea of altruism and effective altruism? So normally when people try to do good, they often just aren't so reflective about those attempts. So someone might approach you on the street asking you to give to charity. And if you're feeling altruistic, you'll give to the person on the street. Or if you think, oh, I wanna do some good in my life, you might volunteer at a local place. Or perhaps you'll decide, pursue a career where you're working in a field that's kind of more obviously beneficial like being a doctor or a nurse or a healthcare professional. But it's very rare that people apply the same level of rigor and analytical thinking to lots of other areas we think about. So take the case of someone approaching you on the street. Imagine if that person instead was saying, hey, I've got this amazing company. Do you want to invest in it? It would be insane. No one would ever think, oh, of course, I'm just a company like you'd think it was a scam. But somehow we don't have that same level of rigor when it comes to doing good, even though the stakes are more important when it comes to trying to help others than trying to make money for ourselves. Well, first of all, so there is a psychology at the individual level of doing good just feels good. And so in some sense, on that pure psychological part, it doesn't matter. In fact, you don't wanna know if it does good or not because most of the time it won't. So like in a certain sense, it's understandable why altruism without the effective part is so appealing to a certain population. By the way, let's zoom off for a second. Do you think most people, two questions. Do you think most people are good? And question number two is, do you think most people wanna do good? So are most people good? I think it's just super dependent on the circumstances that someone is in. I think that the actions people take and their moral worth is just much more dependent on circumstance than it is on someone's intrinsic character. So is there evil within all of us? It seems like with the better angels of our nature, there's a tendency of us as a society to tend towards good, less war. I mean, with all these metrics. Is that us becoming who we want to be or is that some kind of societal force? What's the nature versus nurture thing here? Yeah, so in that case, I just think, yeah, so violence has massively declined over time. I think that's a slow process of cultural evolution, institutional evolution such that now the incentives for you and I to be violent are very, very small indeed. In contrast, when we were hunter gatherers, the incentives were quite large. If there was someone who was potentially disturbing the social order and hunter gatherer setting, there was a very strong incentive to kill that person and people did and it was just the guarded 10% of deaths among hunter gatherers were murders. After hunter gatherers, when you have actual societies is when violence can probably go up because there's more incentive to do mass violence, right? To take over, conquer other people's lands and murder everybody in place and so on. Yeah, I mean, I think total death rate from human causes does go down, but you're right that if you're in a hunter gatherer situation you're kind of a group that you're part of is very small then you can't have massive wars that just massive communities don't exist. But anyway, the second question, do you think most people want to do good? Yeah, and then I think that is true for most people. I think you see that with the fact that most people donate, a large proportion of people volunteer. If you give people opportunities to easily help other people, they will take it. But at the same time, we're a product of our circumstances and if it were more socially awarded to be doing more good, if it were more socially awarded to do good effectively rather than not effectively, then we would see that behavior a lot more. So why should we do good? Yeah, my answer to this is there's no kind of deeper level of explanation. So my answer to kind of why should you do good is well, there is someone whose life is on the line, for example, whose life you can save via donating just actually a few thousand dollars to an effective nonprofit like the Against Malaria Foundation. That is a sufficient reason to do good. And then if you ask, well, why ought I to do that? I'm like, I just show you the same facts again. It's that fact that is the reason to do good. There's nothing more fundamental than that. I'd like to sort of make more concrete the thing we're trying to make better. So you just mentioned malaria. So there's a huge amount of suffering in the world. Are we trying to remove? So is ultimately the goal, not ultimately, but the first step is to remove the worst of the suffering. So there's some kind of threshold of suffering that we want to make sure does not exist in the world. Or do we really naturally want to take a much further step and look at things like income inequality? So not just getting everybody above a certain threshold, but making sure that there's some, that broadly speaking, there's less injustice in the world, unfairness, in some definition, of course, very difficult to define a fairness. Yeah, so the metric I use is how many people do we affect and by how much do we affect them? And so that can, often that means eliminating suffering, but it doesn't have to, could be helping promote a flourishing life instead. And so if I was comparing reducing income inequality or getting people from the very pits of suffering to a higher level, the question I would ask is just a quantitative one of just if I do this first thing or the second thing, how many people am I going to benefit and by how much am I going to benefit? Am I going to move that one person from kind of 10%, 0% well being to 10% well being? Perhaps that's just not as good as moving a hundred people from 10% well being to 50% well being. And the idea is the diminishing returns is the idea of when you're in terrible poverty, then the $1 that you give goes much further than if you were in the middle class in the United States, for example. Absolutely. And this fact is really striking. So if you take even just quite a conservative estimate of how we are able to turn money into well being, the economists put it as like a log curve. That's the or steeper. But that means that any proportional increase in your income has the same impact on your well being. And so someone moving from $1,000 a year to $2,000 a year has the same impact as someone moving from $100,000 a year to $200,000 a year. And then when you combine that with the fact that we in middle class members of rich countries are 100 times richer than financial terms in the global poor, that means we can do a hundred times to benefit the poorest people in the world as we can to benefit people of our income level. And that's this astonishing fact. Yeah, it's quite incredible. A lot of these facts and ideas are just difficult to think about because there's an overwhelming amount of suffering in the world. And even acknowledging it is difficult. Not exactly sure why that is. I mean, I mean, it's difficult because you have to bring to mind, you know, it's an unpleasant experience thinking about other people's suffering. It's unpleasant to be empathizing with it, firstly. And then secondly, thinking about it means that maybe we'd have to change our lifestyles. And if you're very attached to the income that you've got, perhaps you don't want to be confronting ideas or arguments that might cause you to use some of that money to help others. So it's quite understandable in the psychological terms, even if it's not the right thing that we ought to be doing. So how can we do better? How can we be more effective? How does data help? Yeah, in general, how can we do better? It's definitely hard. And we have spent the last 10 years engaged in kind of some deep research projects, to try and answer kind of two questions. One is, of all the many problems the world is facing, what are the problems we ought to be focused on? And then within those problems that we judge to be kind of the most pressing, where we use this idea of focusing on problems that are the biggest in scale, that are the most tractable, where we can make the most progress on that problem, and that are the most neglected. Within them, what are the things that have the kind of best evidence, or we have the best guess, will do the most good. And so we have a bunch of organizations. So GiveWell, for example, is focused on global health and development, and has a list of seven top recommended charities. So the idea in general, and sorry to interrupt, is, so we'll talk about sort of poverty and animal welfare and existential risk. Those are all fascinating topics, but in general, the idea is there should be a group, sorry, there's a lot of groups that seek to convert money into good. And then you also on top of that want to have a accounting of how good they actually perform that conversion, how well they did in converting money to good. So ranking of these different groups, ranking these charities. So does that apply across basically all aspects of effective altruism? So there should be a group of people, and they should report on certain metrics of how well they've done, and you should only give your money to groups that do a good job. That's the core idea. I'd make two comments. One is just, it's not just about money. So we're also trying to encourage people to work in areas where they'll have the biggest impact. Absolutely. And in some areas, you know, they're really people heavy, but money poor. Other areas are kind of money rich and people poor. And so whether it's better to focus time or money depends on the cause area. And then the second is that you mentioned metrics, and while that's the ideal, and in some areas we do, we are able to get somewhat quantitative information about how much impact an area is having. That's not always true. For some of the issues, like you mentioned existential risks, well, we're not able to measure in any sort of precise way like how much progress we're making. And so you have to instead fall back on just rigorous argument and evaluation, even in the absence of data. So let's first sort of linger on your own story for a second. How do you yourself practice effective altruism in your own life? Because I think that's a really interesting place to start. So I've tried to build effective altruism into at least many components of my life. So on the donation side, my plan is to give away most of my income over the course of my life. I've set a bar I feel happy with and I just donate above that bar. So at the moment, I donate about 20% of my income. Then on the career side, I've also shifted kind of what I do, where I was initially planning to work on very esoteric topics in the philosophy of logic, philosophy of language, things that are intellectually extremely interesting, but the path by which they really make a difference to the world is, let's just say it's very unclear at best. And so I switched instead to researching ethics to actually just working on this question of how we can do as much good as possible. And then I've also spent a very large chunk of my life over the last 10 years creating a number of nonprofits who again in different ways are tackling this question of how we can do the most good and helping them to grow over time too. Yeah, we mentioned a few of them with the career selection, 80,000. 80,000 hours. 80,000 hours is a really interesting group. So maybe also just a quick pause on the origins of effective altruism because you paint a picture who the key figures are, including yourself in the effective altruism movement today. Yeah, there are two main strands that kind of came together to form the effective altruism movement. So one was two philosophers, myself and Toby Ord at Oxford, and we had been very influenced by the work of Peter Singer, an Australian model philosopher who had argued for many decades that because one can do so much good at such little cost to oneself, we have an obligation to give away most of our income to benefit those in extreme poverty, just in the same way that we have an obligation to run in and save a child from drowning in a shallow pond if it would just ruin your suit that cost a few thousand dollars. And we set up Giving What We Can in 2009, which is encouraging people to give at least 10% of their income to the most effective charities. And the second main strand was the formation of GiveWell, which was originally based in New York and started in about 2007. And that was set up by Holden Carnovsky and Elie Hassenfeld, who were two hedge fund dudes who were making good money and thinking, well, where should I donate? And in the same way as if they wanted to buy a product for themselves, they would look at Amazon reviews. They were like, well, what are the best charities? Found they just weren't really good answers to that question, certainly not that they were satisfied with. And so they formed GiveWell in order to try and work out what are those charities where they can have the biggest impact. And then from there and some other influences, kind of community grew and spread. Can we explore the philosophical and political space that effective altruism occupies a little bit? So from the little and distant in my own lifetime that I've read of Ayn Rand's work, Ayn Rand's philosophy of objectivism, espouses, and it's interesting to put her philosophy in contrast with effective altruism. So it espouses selfishness as the best thing you can do. But it's not actually against altruism. It's just you have that choice, but you should be selfish in it, right? Or not, maybe you can disagree here. But so it can be viewed as the complete opposite of effective altruism or it can be viewed as similar because the word effective is really interesting. Because if you want to do good, then you should be damn good at doing good, right? I think that would fit within the morality that's defined by objectivism. So do you see a connection between these two philosophies and other perhaps in this complicated space of beliefs that effective altruism is positioned as opposing or aligned with? I would definitely say that objectivism, Ayn Rand's philosophy, is a philosophy that's quite fundamentally opposed to effective altruism. In which way? Insofar as Ayn Rand's philosophy is about championing egoism and saying that I'm never quite sure whether the philosophy is meant to say that just you ought to do whatever will best benefit yourself, that's ethical egoism, no matter what the consequences are. Or second, if there's this alternative view, which is, well, you ought to try and benefit yourself because that's actually the best way of benefiting society. Certainly, in Atlas Shalaguchi is presenting her philosophy as a way that's actually going to bring about a flourishing society. And if it's the former, then well, effective altruism is all about promoting the idea of altruism and saying, in fact, we ought to really be trying to help others as much as possible. So it's opposed there. And then on the second side, I would just dispute the empirical premise. It would seem, given the major problems in the world today, it would seem like this remarkable coincidence, quite suspicious, one might say, if benefiting myself was actually the best way to bring about a better world. So on that point, and I think that connects also with career selection that we'll talk about, but let's consider not objectives, but capitalism. And the idea that you focusing on the thing that you are damn good at, whatever that is, may be the best thing for the world. Part of it is also mindset, right? The thing I love is robots. So maybe I should focus on building robots and never even think about the idea of effective altruism, which is kind of the capitalist notion. Is there any value in that idea in just finding the thing you're good at and maximizing your productivity in this world and thereby sort of lifting all boats and benefiting society as a result? Yeah, I think there's two things I'd want to say on that. So one is what your comparative advantages, what your strengths are when it comes to career. That's obviously super important because there's lots of career paths I would be terrible at if I thought being an artist was the best thing one could do. Well, I'd be doomed, just really quite astonishingly bad. And so I do think, at least within the realm of things that could plausibly be very high impact, choose the thing that you think you're going to be able to really be passionate at and excel at over the long term. Then on this question of should one just do that in an unrestricted way and not even think about what the most important problems are. I do think that in a kind of perfectly designed society, that might well be the case. That would be a society where we've corrected all market failures, we've internalized all externalities, and then we've managed to set up incentives such that people just pursuing their own strengths is the best way of doing good. But we're very far from that society. So if one did that, then it would be very unlikely that you would focus on improving the lives of nonhuman animals that aren't participating in markets or ensuring the long run future goes well, where future people certainly aren't participating in markets or benefiting the global poor who do participate, but have so much less kind of power from a starting perspective that their views aren't accurately kind of represented by market forces too. Got it. So yeah, instead of pure definition capitalism, it just may very well ignore the people that are suffering the most, the white swath of them. So if you could allow me this line of thinking here. So I've listened to a lot of your conversations online. I find, if I can compliment you, they're very interesting conversations. Your conversation on Rogan, on Joe Rogan was really interesting, with Sam Harris and so on, whatever. There's a lot of stuff that's really good out there. And yet, when I look at the internet and I look at YouTube, which has certain mobs, certain swaths of right leaning folks, whom I dearly love. I love all people, especially people with ideas. They seem to not like you very much. So I don't understand why exactly. So my own sort of hypothesis is there is a right left divide that absurdly so caricatured in politics, at least in the United States. And maybe you're somehow pigeonholed into one of those sides. And maybe that's what it is. Maybe your message is somehow politicized. Yeah, I mean. How do you make sense of that? Because you're extremely interesting. Like you got the comments I see on Joe Rogan. There's a bunch of negative stuff. And yet, if you listen to it, the conversation is fascinating. I'm not speaking, I'm not some kind of lefty extremist, but just it's a fascinating conversation. So why are you getting some small amount of hate? So I'm actually pretty glad that Effective Altruism has managed to stay relatively unpoliticized because I think the core message to just use some of your time and money to do as much good as possible to fight some of the problems in the world can be appealing across the political spectrum. And we do have a diversity of political viewpoints among people who have engaged in Effective Altruism. We do, however, do get some criticism from the left and the right. Oh, interesting. What's the criticism? Both would be interesting to hear. Yeah, so criticism from the left is that we're not focused enough on dismantling the capitalist system that they see as the root of most of the problems that we're talking about. And there I kind of disagree on partly the premise where I don't think relevant alternative systems would say to the animals or to the global poor or to the future generations kind of much better. And then also the tactics where I think there are particular ways we can change society that would massively benefit, you know, be massively beneficial on those things that don't go via dismantling like the entire system, which is perhaps a million times harder to do. Then criticism on the right, there's definitely like in response to the Joe Rogan podcast. There definitely were a number of Ayn Rand fans who weren't keen on the idea of promoting altruism. There was a remarkable set of ideas. Just the idea that Effective Altruism was unmanly, I think, was driving a lot of criticism. Okay, so I love fighting. I've been in street fights my whole life. I'm as alpha in everything I do as it gets. And the fact that Joe Rogan said that I thought Scent of a Woman is a better movie than John Wick put me into this beta category amongst people who are like basically saying this, yeah, unmanly or it's not tough. It's not some principled view of strength that is represented by a spasmodic. So actually, so how do you think about this? Because to me, altruism, especially Effective Altruism, I don't know what the female version of that is, but on the male side, manly as fuck, if I may say so. So how do you think about that kind of criticism? I think people who would make that criticism are just occupying a like state of mind that I think is just so different from my state of mind that I kind of struggle to maybe even understand it where if something's manly or unmanly or feminine or unfeminine, I'm like, I don't care. Like, is it the right thing to do or the wrong thing to do? So let me put it not in terms of man or woman. I don't think that's useful, but I think there's a notion of acting out of fear as opposed to out of principle and strength. Yeah. So, okay. Yeah. Here's something that I do feel as an intuition and that I think drives some people who do find Canvaean Land attractive and so on as a philosophy, which is a kind of taking control of your own life and having power over how you're steering your life and not kind of kowtowing to others, you know, really thinking things through. I find like that set of ideas just very compelling and inspirational. I actually think of effect of altruism has really, you know, that side of my personality. It's like scratch that itch where you are just not taking the kind of priorities that society is giving you as granted. Instead, you're choosing to act in accordance with the priorities that you think are most important in the world. And often that involves then doing quite unusual things from a societal perspective, like donating a large chunk of your earnings or working on these weird issues about AI and so on that other people might not understand. Yeah, I think that's a really gutsy thing to do. That is taking control. That's at least at this stage. I mean, that's you taking ownership, not of just yourself, but your presence in this world that's full of suffering and saying as opposed to being paralyzed by that notion is taking control and saying I could do something. Yeah, I mean, that's really powerful. But I mean, sort of the one thing I personally hate too about the left currently that I think those folks to detect is the social signaling. When you look at yourself, sort of late at night, would you do everything you're doing in terms of effective altruism if your name, because you're quite popular, but if your name was totally unattached to it, so if it was in secret. Yeah, I mean, I think I would. To be honest, I think the kind of popularity is like, you know, it's mixed bag, but there are serious costs. And I don't particularly, I don't like love it. Like, it means you get all these people calling you a cuck on Joe Rogan. It's like not the most fun thing. But you also get a lot of sort of brownie points for doing good for the world. Yeah, you do. But I think my ideal life, I would be like in some library solving logic puzzles all day and I'd like really be like learning maths and so on. So you have a like good body of friends and so on. So your instinct for effective altruism is something deep. It's not one that is communicating socially. It's more in your heart. You want to do good for the world. Yeah, I mean, so we can look back to early giving what we can. So, you know, we're setting this up, me and Toby. And I really thought that doing this would be a big hit to my academic career because I was now spending, you know, at that time more than half my time setting up this nonprofit at the crucial time when you should be like producing your best academic work and so on. And it was also the case at the time. It was kind of like the Toby order club. You know, he was he was the most popular. There's this personal interest story about him and his plans donate and sorry to interrupt but Toby was donating a large amount. Can you tell just briefly what he was doing? Yeah, so he made this public commitment to give everything he earned above 20,000 pounds per year to the most effective causes. And even as a graduate student, he was still donating about 15, 20% of his income, which is so quite significant given that graduate students are not known for being super wealthy. That's right. And when we launched Giving What We Can, the media just loved this as like a personal interest story. So the story about him and his pledge was the most, yeah, it was actually the most popular news story of the day. And we kind of ran the same story a year later and it was the most popular news story of the day a year later too. And so it really was kind of several years before then I was also kind of giving more talks and starting to do more writing and then especially with, you know, I wrote this book Doing Good Better that then there started to be kind of attention and so on. But deep inside your own relationship with effective altruism was, I mean, it had nothing to do with the publicity. Did you see yourself? How did the publicity connect with it? Yeah, I mean, that's kind of what I'm saying is I think the publicity came like several years afterwards. I mean, at the early stage when we set up Giving What We Can, it was really just every person we get to pledge 10% is, you know, something like $100,000 over their lifetime. That's huge. And so it was just we had started with 23 members, every single person was just this like kind of huge accomplishment. And at the time, I just really thought, you know, maybe over time we'll have a hundred members and that'll be like amazing. Whereas now we have, you know, over four thousand and one and a half billion dollars pledged. That's just unimaginable to me at the time when I was first kind of getting this, you know, getting the stuff off the ground. So can we talk about poverty and the biggest problems that you think in the near term effective altruism can attack in each one. So poverty obviously is a huge one. Yeah. How can we help? Great. Yeah. So poverty, absolutely this huge problem. 700 million people in extreme poverty living in less than two dollars per day where that's what that means is what two dollars would buy in the US. So think about that. It's like some rice, maybe some beans. It's very, you know, really not much. And at the same time, we can do an enormous amount to improve the lives of people in extreme poverty. So the things that we tend to focus on interventions in global health and that's for a couple of few reasons. One is like global health just has this amazing track record life expectancy globally is up 50% relative to 60 or 70 years ago. We've eradicated smallpox that's which killed 2 million lives every year almost eradicated polio. Second is that we just have great data on what works when it comes to global health. So we just know that bed nets protect children from prevent them from dying from malaria. And then the third is just that's extremely cost effective. So it costs $5 to buy one bed net, protects two children for two years against malaria. If you spend about $3,000 on bed nets, then statistically speaking, you're going to save a child's life. And there are other interventions too. And so given the people in such suffering and we have this opportunity to, you know, do such huge good for such low cost. Well, yeah, why not? So the individual. So for me today, if I wanted to look at poverty, how would I help? And I wanted to say, I think donating 10% of your income is a very interesting idea or some percentage or some setting a bar and sort of sticking to it. How do we then take the step towards the effective part? So you've conveyed some notions, but who do you give the money to? Yeah. So GiveWell, this organization I mentioned, well, it makes charity recommendations and some of its top recommendations. So Against Malaria Foundation is this organization that buys and distributes these insecticide seeded bed nets. And then it has a total of seven charities that it recommends very highly. So that recommendation, is it almost like a star of approval or is there some metrics? So what are the ways that GiveWell conveys that this is a great charity organization? Yeah. So GiveWell is looking at metrics and it's trying to compare charities ultimately in the number of lives that you can save or an equivalent benefit. So one of the charities it recommends is GiveDirectly, which simply just transfers cash to the poorest families where poor family will get a cash transfer of $1,000 and they kind of regard that as the baseline intervention because it's so simple and people, you know, they know what to do with how to benefit themselves. That's quite powerful, by the way. So before GiveWell, before the Effective Altruism Movement, was there, I imagine there's a huge amount of corruption, funny enough, in charity organizations or misuse of money. Yeah. So there was nothing like GiveWell before that? No. I mean, there were some. So, I mean, the charity corruption, I mean, obviously there's some, I don't think it's a huge issue. They're also just focusing on the long things. Prior to GiveWell, there were some organizations like Charity Navigator, which were more aimed at worrying about corruption and so on. So they weren't saying, these are the charities where you're going to do the most good. Instead, it was like, how good are the charities financials? How good is its health? Are they transparent? And yeah, so that would be more useful for weeding out some of those worst charities. So GiveWell has just taken a step further, sort of in this 21st century of data. It's actually looking at the effective part. Yeah. So it's like, you know, if you know the wire cutter for if you want to buy a pair of headphones, they will just look at all the headphones and be like, these are the best headphones you can buy. That's the idea with GiveWell. Okay. So do you think there's a bar of what suffering is? And do you think one day we can eradicate suffering in our world? Yeah. Amongst humans? Let's talk humans for now. Talk humans. But in general, yeah, actually. So there's a colleague of mine calling the term abolitionism for the idea that we should just be trying to abolish suffering. And in the long run, I mean, I don't expect to anytime soon, but I think we can. I think that would require, you know, quite change, quite drastic changes to the way society is structured and perhaps even the, you know, the human, in fact, even changes to human nature. But I do think that suffering whenever it occurs is bad and we should want it to not occur. So there's a line. There's a gray area between suffering. Now I'm Russian. So I romanticize some aspects of suffering. There's a gray line between struggle, gray area between struggle and suffering. So one, do we want to eradicate all struggle in the world? So there's an idea, you know, that the human condition inherently has suffering in it and it's a creative force. It's a struggle of our lives and we somehow grow from that. How do you think about, how do you think about that? I agree that's true. So, you know, often, you know, great artists can be also suffering from, you know, major health conditions or depression and so on. They come from abusive parents. Most great artists, I think, come from abusive parents. Yeah, that seems to be at least commonly the case, but I want to distinguish between suffering as being instrumentally good, you know, it causes people to produce good things and whether it's intrinsically good and I think intrinsically it's always bad. And so if we can produce these, you know, great achievements via some other means where, you know, if we look at the scientific enterprise, we've produced incredible things often from people who aren't suffering, have, you know, pretty good lives. They're just, they're driven instead of, you know, being pushed by a certain sort of anguish. They're being driven by intellectual curiosity. If we can instead produce a society where it's all cavet and no stick, that's better from my perspective. Yeah, but I'm going to disagree with the notion that that's possible, but I would say most of the suffering in the world is not productive. So I would dream of effective altruism curing that suffering. Yeah, but then I would say that there is some suffering that is productive that we want to keep the because but that's not even the focus of because most of the suffering is just absurd and needs to be eliminated. So let's not even romanticize this usual notion I have, but nevertheless struggle has some kind of inherent value that to me at least, you're right. There's some elements of human nature that also have to be modified in order to cure all suffering. Yeah, I mean, there's an interesting question of whether it's possible. So at the moment, you know, most of the time we're kind of neutral and then we burn ourselves and that's negative and that's really good that we get that negative signal because it means we won't burn ourselves again. There's a question like could you design agents humans such that you're not hovering around the zero level you're hovering it like bliss. Yeah, and then you touch the flame and you're like, oh no, you're just slightly worse bliss. Yeah, but that's really bad compared to the bliss you were normally in so that you can have like a gradient of bliss instead of like pain and pleasure on that point. I think it's a really important point on the experience of suffering the relative nature of it. Maybe having grown up in the Soviet Union were quite poor by any measure and when I when I was in my childhood, but it didn't feel like you're poor because everybody around you were poor there's a and then in America, I feel I for the first time begin to feel poor. Yeah. Yeah, because of the road there's different. There's some cultural aspects to it that really emphasize that it's good to be rich. And then there's just the notion that there is a lot of income inequality and therefore you experience that inequality. That's where suffering go. Do you so what do you think about the inequality of suffering that that we have to think about do you think we have to think about that as part of effective altruism? Yeah, I think we're just things vary in terms of whether you get benefits or costs from them just in relative terms or in absolute terms. So a lot of the time yeah, there's this hedonic treadmill where if you get you know, there's money is useful because it helps you buy things or good for you because it helps you buy things, but there's also a status component too and that status component is kind of zero sum if you were saying like in Russia, you know, no one else felt poor because everyone around you is poor. Whereas now you've got this these other people who are you know super rich and maybe that makes you feel. You know less good about yourself. There are some other things however, which are just intrinsically good or bad. So commuting for example, it's just people hate it. It doesn't really change knowing the other people are commuting to doesn't make it any any kind of less bad, but it's sort of to push back on that for a second. I mean, yes, but also if some people were, you know on horseback your commute on the train might feel a lot better. Yeah, you know the there is a relative Nick. I mean everybody's complaining about society today forgetting it's forgetting how much better is the better angels of our nature how the technologies improve fundamentally improving most of the world's lives. Yeah, and actually there's some psychological research on the well being benefits of volunteering where people who volunteer tend to just feel happier about their lives and one of the suggested explanations is it because it extends your reference class. So no longer you comparing yourself to the Joneses who have their slightly better car because you realize that you know people in much worse conditions than you and so now, you know your life doesn't seem so bad. That's actually on the psychological level. One of the fundamental benefits of effective altruism. Yeah is is I mean, I guess it's the altruism part of effective altruism is exposing yourself to the suffering in the world allows you to be more. Yeah happier and actually allows you in the sort of meditative introspective way realize that you don't need most of the wealth you have to to be happy. Absolutely. I mean, I think effective options have been this huge benefit for me and I really don't think that if I had more money that I was living on that that would change my level of well being at all. Whereas engaging in something that I think is meaningful that I think is stealing humanity in a positive direction. That's extremely rewarding. And so yeah, I mean despite my best attempts at sacrifice. Um, I don't you know, I think I've actually ended up happier as a result of engaging in effective altruism than I would have done. That's such an interesting idea. Yeah, so let's let's talk about animal welfare. Sure, easy question. What is consciousness? Yeah, especially as it has to do with the capacity to suffer. I think there seems to be a connection between how conscious something is the amount of consciousness and stability to suffer and that all comes into play about us thinking how much suffering there's in the world with regard to animals. So how do you think about animal welfare and consciousness? Okay. Well consciousness easy question. Okay. Um, yeah, I mean, I think we don't have a good understanding of consciousness. My best guess is it's got and by consciousness. I'm meaning what it is feels like to be you the subjective experience that's seems to be different from everything else we know about in the world. Yeah, I think it's clear. It's very poorly understood at the moment. I think it has something to do with information processing. So the fact that the brain is a computer or something like a computer. So that would mean that very advanced AI could be conscious of information processors in general could be conscious with some suitable complexity, but that also some suitable complexity. It's a question whether greater complexity creates some kind of greater consciousness which relates to animals. Yeah, right. Is there if it's an information processing system and it's smaller and smaller is an ant less conscious than a cow less conscious than a monkey. Yeah, and again this super hard question, but I think my best guess is yes, like if you if I think well consciousness, it's not some magical thing that appears out of nowhere. It's not you know, Descartes thought it was just comes in from this other realm and then enters through the pineal gland in your brain and that's kind of soul and it's conscious. So it's got something to do with what's going on in your brain. A chicken has one three hundredth of the size of the brain that you have ants. I don't know how small it is. Maybe it's a millionth the size my best guess which I may well be wrong about because this is so hard is that in some relevant sense the chicken is experiencing consciousness to a less degree than the human and the ants significantly less again. I don't think it's as little as three hundredth as much. I think there's everyone who's ever seen a chicken that's there's evolutionary reasons for thinking that like the ability to feel pain comes on the scene relatively early on and we have lots of our brain that's dedicated stuff that doesn't seem to have to do in anything to do with consciousness language processing and so on. So it seems like the easy so there's a lot of complicated questions there that we can't ask the animals about but it seems that there is easy questions in terms of suffering which is things like factory farming that could be addressed. Yeah, is that is that the lowest hanging fruit? If I may use crude terms here of animal welfare. Absolutely. I think that's the lowest hanging fruit. So at the moment we kill we raise and kill about 50 billion animals every year. So how many 50 billion in? Yeah, so for every human on the planet several times that number of being killed and the vast majority of them are raised in factory farms where basically whatever your view on animals, I think you should agree even if you think well, maybe it's not bad to kill an animal. Maybe if the animal was raised in good conditions, that's just not the empirical reality. The empirical reality is that they are kept in incredible cage confinement. They are de beaked or detailed without an aesthetic, you know chickens often peck each other to death other like otherwise because of them such stress. It's really, you know, I think when a chicken gets killed that's the best thing that happened to the chicken in the course of its life and it's also completely unnecessary. This is in order to save, you know a few pence for the price of meat or price of eggs and we have indeed found it's also just inconsistent with consumer preference as well people who buy the products if they could they all they when you do surveys are extremely against suffering in factory farms. It's just they don't appreciate how bad it is and you know, just tend to go with easy options. And so then the best the most effective programs I know of at the moment are nonprofits that go to companies and work with companies to get them to take a pledge to cut certain sorts of animal products like eggs from cage confinement out of their supply chain. And it's now the case that the top 50 food retailers and fast food companies have all made these kind of cage free pledges and when you do the numbers you get the conclusion that every dollar you're giving to these nonprofits result in hundreds of chickens being spared from cage confinement. And then they're working to other other types of animals other products too. So is that the most effective way to do in have a ripple effect essentially it's supposed to directly having regulation from on top that says you can't do this. So I would be more open to the regulation approach, but at least in the US there's quite intense regulatory capture from the agricultural industry. And so attempts that we've seen to try and change regulation have it's been a real uphill struggle. There are some examples of ballot initiatives where the people have been able to vote in a ballot to say we want to ban eggs from cage conditions and that's been huge. That's been really good, but beyond that it's much more limited. So I've been really interested in the idea of hunting in general and wild animals and seeing nature as a form of cruelty that I am ethically more okay with. Okay, just from my perspective and then I read about wild animal suffering that I'm just I'm just giving you the kind of yeah notion of how I felt because animal because animal factory farming is so bad. Yeah that living in the woods seem good. Yeah, and yet when you actually start to think about it all I mean all of the animals in the animal world the living in like terrible poverty, right? Yeah. Yeah, so you have all the medical conditions all of that. I mean they're living horrible lives. It could be improved. That's a really interesting notion that I think may not even be useful to talk about because factory farming is such a big thing to focus on. Yeah, but it's nevertheless an interesting notion to think of all the animals in the wild as suffering in the same way that humans in poverty are suffering. Yeah, I mean and often even worse so many animals we produce by our selection. So you have a very large number of children in the expectation that only a small number survive. And so for those animals almost all of them just live short lives where they starve to death. So yeah, there's huge amounts of suffering in nature that I don't think we should you know pretend that it's this kind of wonderful paradise for most animals. Yeah, their life is filled with hunger and fear and disease. Yeah, I did agree with you entirely that when it comes to focusing on animal welfare, we should focus in factory farming, but we also yeah should be aware to the reality of what life for most animals is like. So let's talk about a topic I've talked a lot about and you've actually quite eloquently talked about which is the third priority that effective altruism considers is really important is existential risks. Yeah, when you think about the existential risks that are facing our civilization, what's before us? What concerns you? What should we be thinking about from in the especially from an effective altruism perspective? Great. So the reason I started getting concerned about this was thinking about future generations where the key idea is just well future people matter morally. There are vast numbers of future people. If we don't cause our own extinction, there's no reason why civilization might not last a million years. I mean we last as long as a typical mammalian species or a billion years is when the Earth is no longer habitable or if we can take to the stars then perhaps it's trillions of years beyond that. So the future could be very big indeed and it seems like we're potentially very early on in civilization. Then the second idea is just well, maybe there are things that are going to really derail that things that actually could prevent us from having this long wonderful civilization and instead could cause our own cause our own extinction or otherwise perhaps like lock ourselves into a very bad state. And what ways could that happen? Well causing our own extinction development of nuclear weapons in the 20th century at least put on the table that we now had weapons that were powerful enough that you could very significantly destroy society perhaps and all that nuclear war would cause a nuclear winter. Perhaps that would be enough for the human race to go extinct. Why do you think we haven't done it? Sorry to interrupt. Why do you think we haven't done it yet? Is it surprising to you that having, you know, always for the past few decades several thousand of active ready to launch nuclear weapons warheads and yet we have not launched them ever since the initial launch on Hiroshima and Nagasaki. I think it's a mix of luck. So I think it's definitely not inevitable that we haven't used them. So John F. Kennedy, general Cuban Missile Crisis put the estimate of nuclear exchange between the US and USSR that somewhere between one and three and even so, you know, we really did come close. At the same time, I do think mutually assured destruction is a reason why people don't go to war. It would be, you know, why nuclear powers don't go to war. Do you think that holds if you can linger on that for a second, like my dad is a physicist amongst other things and he believes that nuclear weapons are actually just really hard to build which is one of the really big benefits of them currently so that you don't have it's very hard if you're crazy to build to acquire a nuclear weapon. So the mutually shared destruction really works when you talk seems to work better when it's nation states, when it's serious people, even if they're a little bit, you know, dictatorial and so on. Do you think this mutually sure destruction idea will carry how far will it carry us in terms of different kinds of weapons? Oh, yeah, I think it's your point that nuclear weapons are very hard to build and relatively easy to control because you can control fissile material is a really important one and future technology that's equally destructive might not have those properties. So for example, if in the future people are able to design viruses, perhaps using a DNA printing kit that's on that, you know, one can just buy. In fact, there are companies in the process of creating home DNA printing kits. Well, then perhaps that's just totally democratized. Perhaps the power to reap huge destructive potential is in the hands of most people in the world or certainly most people with effort and then yeah, I no longer trust mutually assured destruction because some for some people the idea that they would die is just not a disincentive. There was a Japanese cult, for example. Ohm Shinrikyo in the 90s that had they what they believed was that Armageddon was coming if you died before Armageddon, you would get good karma. You wouldn't go to hell if you died during Armageddon. Maybe you would go to hell and they had a biological weapons program chemical weapons program when they were finally apprehended. They hadn't stocks of southern gas that were sufficient to kill 4 million people engaged in multiple terrorist acts. If they had had the ability to print a virus at home, that would have been very scary. So it's not impossible to imagine groups of people that hold that kind of belief of death as suicide as a good thing for passage into the next world and so on and then connect them with some weapons then ideology and weaponry may create serious problems for us. Let me ask you a quick question on what do you think is the line between killing most humans and killing all humans? How hard is it to kill everybody? Yeah, have you thought about this? I've thought about it a bit. I think it is very hard to kill everybody. So in the case of let's say an all out nuclear exchange and let's say that leads to nuclear winter. We don't really know but we you know might well happen that would I think result in billions of deaths would it kill everybody? It's quite it's quite hard to see how that how it would kill everybody for a few reasons. One is just those are so many people. Yes, you know seven and a half billion people. So this bad event has to kill all you know, all almost all of them. Secondly live in such a diversity of locations. So a nuclear exchange or the virus that has to kill people who live in the coast of New Zealand which is going to be climatically much more stable than other areas in the world or people who are on submarines or who have access to bunkers. So there's a very like there's just like I'm sure there's like two guys in Siberia just badass. There's the just human nature somehow just perseveres. Yeah, and then the second thing is just if there's some catastrophic event people really don't want to die. So there's going to be like, you know, huge amounts of effort to ensure that it doesn't affect everyone. Have you thought about what it takes to rebuild a society with smaller smaller numbers like how big of a setback these kinds of things are? Yeah, so then that's something where there's real uncertainty I think where at some point you just lose genetic sufficient genetic diversity such that you can't come back. There's it's unclear how small that population is. But if you've only got say a thousand people or fewer than a thousand, then maybe that's small enough. What about human knowledge and then there's human knowledge. I mean, it's striking how short on geological timescales or evolutionary timescales the progress in or how quickly the progress in human knowledge has been like agriculture. We only invented in 10,000 BC cities were only, you know, 3000 BC whereas typical mammal species is half a million years to a million years. Do you think it's inevitable in some sense agriculture everything that came the Industrial Revolution cars planes the internet that level of innovation you think is inevitable. I think given how quickly it arose. So in the case of agriculture, I think that was dependent on climate. So it was the kind of glacial period was over the earth warmed up a bit that made it much more likely that humans would develop agriculture when it comes to the Industrial Revolution. It's just you know, again only took a few thousand years from cities to Industrial Revolution if we think okay, we've gone back to this even let's say agricultural era, but there's no reason why we wouldn't go extinct in the coming tens of thousands of years or hundreds of thousands of years. It seems just vet. It would be very surprising if we didn't rebound unless there's some special reason that makes things different. Yes. So perhaps we just have a much greater like disease burden now so HIV exists. It didn't exist before and perhaps that's kind of latent and you know and being suppressed by modern medicine and sanitation and so on but would be a much bigger problem for some, you know, utterly destroyed the society that was trying to rebound or there's just maybe there's something we don't know about. So another existential risk comes from the mysterious the beautiful artificial intelligence. Yeah. So what what's the shape of your concerns about AI? I think there are quite a lot of concerns about AI and sometimes the different risks don't get distinguished enough. So the kind of classic worry most is closely associated with Nick Bostrom and Elias Joukowski is that we at some point move from having narrow AI systems to artificial general intelligence. You get this very fast feedback effect where AGI is able to build, you know, artificial intelligence helps you to build greater artificial intelligence. We have this one system that suddenly very powerful far more powerful than others than perhaps far more powerful than, you know, the rest of the world combined and then secondly, it has goals that are misaligned with human goals. And so it pursues its own goals. It realize, hey, there's this competition namely from humans. It would be better if we eliminated them in just the same way as homo sapiens eradicated the Neanderthals. In fact, it in fact killed off most large animals on the planet that walk the planet. So that's kind of one set of worries. I think that's not my I think these shouldn't be dismissed as science fiction. I think it's something we should be taking very seriously, but it's not the thing you visualize when you're concerned about the biggest near term. Yeah, I think it's I think it's like one possible scenario that would be astronomically bad. I think that other scenarios that would also be extremely bad comparably bad that are more likely to occur. So one is just we are able to control AI. So we're able to get it to do what we want it to do. And perhaps there's not like this fast takeoff of AI capabilities within a single system. It's distributed across many systems that do somewhat different things, but you do get very rapid economic and technological progress as a result that concentrates power into the hands of a very small number of individuals, perhaps a single dictator. And secondly, that single individual is or small group of individuals or single country is then able to like lock in their values indefinitely via transmitting those values to artificial systems that have no reason to die like, you know, their code is copyable. Perhaps, you know, Donald Trump or Xi Jinping creates their kind of AI progeny in their own image. And once you have a system that's once you have a society that's controlled by AI, you no longer have one of the main drivers of change historically, which is the fact that human lifespans are you know, only a hundred years give or take. So that's really interesting. So as opposed to sort of killing off all humans is locking in creating a hell on earth, basically a set of principles under which the society operates that's extremely undesirable. So everybody is suffering indefinitely. Or it doesn't, I mean, it also doesn't need to be hell on earth. It could just be the long values. So we talked at the very beginning about how I want to see this kind of diversity of different values and exploration so that we can just work out what is kind of morally like what is good, what is bad and then pursue the thing that's bad. So actually, so the idea of wrong values is actually probably the beautiful thing is there's no such thing as right and wrong values because we don't know the right answer. We just kind of have a sense of which value is more right, which is more wrong. So any kind of lock in makes a value wrong because it prevents exploration of this kind. Yeah, and just, you know, imagine if fascist value, you know, imagine if there was Hitler's utopia or Stalin's utopia or Donald Trump's or Xi Jinping's forever. Yeah, you know, how good or bad would that be compared to the best possible future we could create? And my suggestion is it would really suck compared to the best possible future we could create. And you're just one individual. There's some individuals for whom Donald Trump is perhaps the best possible future. And so that's the whole point of us individuals exploring the space together. Exactly. Yeah, and what's trying to figure out which is the path that will make America great again. Yeah, exactly. So how can effective altruism help? I mean, this is a really interesting notion they actually describing of artificial intelligence being used as extremely powerful technology in the hands of very few potentially one person to create some very undesirable effect. So as opposed to AI and again, the source of the undesirableness there is the human. Yeah, AI is just a really powerful tool. So whether it's that or whether AI's AGI just runs away from us completely. How as individuals, as people in the effective altruism movement, how can we think about something like this? I understand poverty and animal welfare, but this is a far out incredibly mysterious and difficult problem. Great. Well, I think there's three paths as an individual. So if you're thinking about, you know, career paths you can pursue. So one is going down the line of technical AI safety. So this is most relevant to the kind of AI winning AI taking over scenarios where this is just technical work on current machine learning systems often sometimes going more theoretical to on how we can ensure that an AI is able to learn human values and able to act in the way that you want it to act. And that's a pretty mainstream issue and approach in machine learning today. So, you know, we definitely need more people doing that. Second is on the policy side of things, which I think is even more important at the moment, which is how should developments in AI be managed on a political level? How can you ensure that the benefits of AI are very distributed? It's not being, power isn't being concentrated in the hands of a small set of individuals. How do you ensure that there aren't arms races between different AI companies that might result in them, you know, cutting corners with respect to safety. And so there the input as individuals who can have is this. We're not talking about money. We're talking about effort. We're talking about career choices. We're talking about career choice. Yeah, but then it is the case that supposing, you know, you're like, I've already decided my career. I'm doing something quite different. You can contribute with money too, where at the Center for Effective Altruism, we set up the Long Term Future Fund. So if you go on to effectivealtruism.org, you can donate where a group of individuals will then work out what's the highest value place they can donate to work on existential risk issues with a particular focus on AI. What's path number three? This was path number three. This is donations with the third option I was thinking of. Okay. And then, yeah, there are, you can also donate directly to organizations working on this, like Center for Human Compatible AI at Berkeley, Future of Humanity Institute at Oxford, or other organizations too. Does AI keep you up at night? This kind of concern? Yeah, it's kind of a mix where I think it's very likely things are going to go well. I think we're going to be able to solve these problems. I think that's by far the most likely outcome, at least over the next. By far the most likely. So if you look at all the trajectories running away from our current moment in the next hundred years, you see AI creating destructive consequences as a small subset of those possible trajectories. Or at least, yeah, kind of eternal, destructive consequences. I think that being a small subset. At the same time, it still freaks me out. I mean, when we're talking about the entire future of civilization, then small probabilities, you know, 1% probability, that's terrifying. What do you think about Elon Musk's strong worry that we should be really concerned about existential risks of AI? Yeah, I mean, I think, you know, broadly speaking, I think he's right. I think if we talked, we would probably have very different probabilities on how likely it is that we're doomed. But again, when it comes to talking about the entire future of civilization, it doesn't really matter if it's 1% or if it's 50%, we ought to be taking every possible safeguard we can to ensure that things go well rather than poorly. Last question, if you yourself could eradicate one problem from the world, what would that problem be? That's a great question. I don't know if I'm cheating in saying this, but I think the thing I would most want to change is just the fact that people don't actually care about ensuring the long run future goes well. People don't really care about future generations. They don't think about it. It's not part of their aims. In some sense, you're not cheating at all because in speaking the way you do, in writing the things you're writing, you're doing, you're addressing exactly this aspect. Exactly. That is your input into the effective altruism movement. So for that, Will, thank you so much. It's an honor to talk to you. I really enjoyed it. Thanks so much for having me on. If that were the case, we'd probably be pretty generous. Next round's on me, but that's effectively the situation we're in all the time. It's like a 99% off sale or buy one get 99 free. Might be the most amazing deal you'll see in your life. Thank you for listening and hope to see you next time.

William MacAskill: Effective Altruism | Lex Fridman Podcast #84