Lex Fridman Podcast #84 – William MacAskill: Effective Altruism [Reupload]
william macaskill lex fridman podcast william macaskill lex fridman lex fridman effective altruism Lex Fridman #84 Lex Fridman Podcast #84 William MacAskill
deleted episode with William MacAskill on Effective altruism
0:00 – Introduction
2:39 – Utopia – the Long Reflection
10:25 – Advertisement model
15:56 – Effective altruism
38:28 – Criticism
49:02 – Biggest problems in the world
53:40 – Suffering
1:01:40 – Animal welfare
1:09:23 – Existential risks
1:19:08 – Existential risk from AGI
 The following is a conversation with William McCaskill. He's a philosopher, ethicist, and one of the originators of the effective altruism movement. His research focuses on the fundamentals of effective altruism, or the use of evidence and reason to help others by as much as possible with our time and money. With a particular concentration on how to act given moral uncertainty. He's the author of Doing Good, Better, Effective Altruism and a Radical New Way to Make a Difference. He is a co-founder and the president of the Center of Effective Altruism, CEA, that encourages people to commit to donate at least 10% of their income to the most effective charities. He co-founded 80,000 Hours, which is a nonprofit that provides research and advice on how you can best make a difference through your career. This conversation was recorded before the outbreak of the coronavirus pandemic. For everyone feeling the medical, psychological, and financial burden of this crisis, I'm sending love your way. Stay strong, we're in this together. We'll beat this thing. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, review it with five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter, Alex Friedman, spelled F-R-I-D-M-A-N. As usual, I'll do one or two minutes of ads now and never any ads in the middle. They can break the flow of the conversation. I hope that works for you and doesn't hurt the listening experience. This show is presented by Cash App, the number one finance app in the App Store. When you get it, use code LEXPODCAST. Cash App lets you send money to friends, buy Bitcoin, and invest in the stock market with as little as $1. Since Cash App allows you to send and receive money digitally, peer-to-peer, and security in all digital transactions is very important, let me mention the PCI data security standard that Cash App is compliant with. I'm a big fan of standards for safety and security. PCI DSS is a good example of that, where a bunch of competitors got together and agreed that there needs to be a global standard around the security of transactions. Now, we just need to do the same for autonomous vehicles and AI systems in general. So again, if you get Cash App from the App Store, Google Play, and use the code LEXPODCAST, you get $10, and Cash App will also donate $10 to FIRST, an organization that is helping to advance robotics and STEM education for young people around the world. And now, here's my conversation with William McCaskill. What does utopia for humans and all life on Earth look like for you? That's a great question. What I want to say is that we don't know, and the utopia we want to get to is an indirect one that I call the long reflection. So a period of post-scarcity, no longer have the kind of urgent problems we have today, but instead can spend perhaps it's tens of thousands of years debating, engaging in ethical reflection in order before we take any kind of drastic lock-in actions, like spreading to the stars, and then we can figure out what is right, what is of kind of moral value. The long reflection, that's a really beautiful term. So if we look at Twitter for just a second, do you think human beings are able to reflect in a productive way? I don't mean to make it sound bad because there is a lot of fights and politics and division in our discourse. Maybe if you zoom out, it actually is civilized discourse. It might not feel like it, but when you zoom out. So I don't want to say that Twitter is not civilized discourse. I actually believe it's more civilized than people give it credit for. But do you think the long reflection can actually be stable where we as human beings with our descendants of a brains would be able to sort of rationally discuss things together and arrive at ideas? I think overall, we're pretty good at discussing things, rationally, and at least in the earlier stages of our mind, of our lives being open to many different ideas and being able to be convinced and change our views. I think that Twitter is designed almost to bring out all of the worst tendencies. So if the long reflection were conducted on Twitter, maybe it would be better just not even to bother. But I think the challenge really is getting to a stage where we have a society that is as conducive as possible to rational reflection, to deliberation. I think we're actually very lucky to be in a liberal society where people are able to discuss a lot of ideas and so on. I think when we look to the future, that's not at all guaranteed that society would be like that rather than a society where there's a fixed canon of values that are being imposed on all of society and where you aren't able to question that. That would be very bad from my perspective because it means we wouldn't be able to figure out what the truth is. I can already sense we're gonna go down a million tangents, but what do you think is the, if Twitter's not optimal, what kind of mechanism in this modern age of technology can we design where the exchange of ideas could be both civilized and productive and yet not be too constrained where there's rules of what you can say and can't say, which is, as you say, is not desirable, but yet not have some limits as to what can be said or not and so on. Do you have any ideas, thoughts on the possible future? Of course, nobody knows how to do it, but do you have thoughts of what a better Twitter might look like? I think that text-based media are intrinsically gonna be very hard to be conducive to rational discussion because if you think about it from an informational perspective, if I just send you a text of less than, what is it now, 240 characters, 280 characters, I think, that's a tiny amount of information compared to, say, you and I talking now where you have access to the words I say, which is the same as in text, but also my tone, also my body language, and we're very poorly designed to be able to assess. I have to read all of this context into anything you say, so I say maybe your partner sends you a text and as a full stop at the end, are they mad at you? You don't know, you have to infer everything about this person's mental state from whether they put a full stop at the end of a text or not. Well, the flip side of that, is it truly text that's the problem here because there's a viral aspect to the text where you could just post text nonstop, it's very immediate. The times before Twitter, before the internet, the way you would exchange text is you would write books, and that, while it doesn't get body language, it doesn't get tone, it doesn't, so on, but it does actually boil down after some time thinking some editing and so on, boil down ideas. So is the immediacy and the viral nature, which produces the outrage mobs and so on, the potential problem? I think that is a big issue. I think there's gonna be this strong selection effect where something that provokes outrage, well, that's high arousal, you're more likely to retweet that, where there's kind of sober analysis is not as sexy, not as viral. I do agree that long form content is much better to productive discussion. In terms of the media that are very popular at the moment, I think the podcasting is great where, like your podcasts are two hours long, so they're much more in depth than Twitter are. And you are able to convey so much more nuance, so much more caveat, because it's an actual conversation. It's more like the sort of communication that we've evolved to do, rather than kind of these very small little snippets of ideas that, when also combined with bad incentives, just clearly aren't designed for helping us get to the truth. It's kind of interesting. It's not just the length of the podcast medium, but it's the fact that it was started by people that don't give a damn about quote unquote demand. That there's a relaxed, sort of the style that Joe Rogan does, there's a freedom to express ideas in an unconstrained way that's very real. It's kind of funny that it feels so refreshingly real to us today, and I wonder what the future looks like. It's a little bit sad now that quite a lot of sort of more popular people are getting into podcasting. And they try to sort of create, they try to control it, they try to constrain it in different kinds of ways. People I love, like Conan O'Brien and so on, different comedians. And I'd love to see where the real aspects of this podcasting medium persists. Maybe in TV, maybe in YouTube, maybe Netflix is pushing those kind of ideas. And it's a really exciting word, that kind of sharing of knowledge. Yeah, I mean I think it's a double-edged sword as it becomes more popular and more profitable where on the one hand you'll get a lot more creativity, people doing more interesting things with the medium, but also perhaps you get this lace to the bottom where suddenly maybe it'll be hard to find good content on podcasts because it'll be so overwhelmed by the latest bit of viral outrage. So speaking of that, jumping on effective altruism for a second, so much of that internet content is funded by advertisements. Just in the context of effective altruism, we're talking about the richest companies in the world, they're funded by advertisements essentially. Google, that's their primary source of income. Do you see that as, do you have any criticism of that source of income? Do you see that source of money as a potentially powerful source of money that could be used, well certainly could be used for good, but is there something bad about that source of money? I think there's significant worries with it where it means that the incentives of the company might be quite misaligned with making people's lives better, where again, perhaps the incentives are towards increasing drama and debate on your social media feed in order that more people are going to be engaged, perhaps kind of compulsively involved with the platform. Whereas there are other business models like having an opt-in subscription service where perhaps they have other issues, but there's much more of a incentive to provide a product that its users are just really wanting, because now I'm paying for this product, I'm paying for this thing that I wanna buy, rather than I'm trying to use this thing and then trying to get a profit mechanism that is somewhat orthogonal to me actually just wanting to use the product. And so, I mean in some cases it'll work better than others. I can imagine, I can in theory imagine Facebook having a subscription service, but I think it's unlikely to happen anytime soon. Well it's interesting, it's weird now that you bring it up that it's unlikely. This example, I pay I think 10 bucks a month for YouTube Red, and that's, and I don't think I get it much for that, except just, so no ads, but in general it's just a slightly better experience. And I would gladly, now I'm not wealthy, in fact I'm operating very close to zero dollars, but I would pay 10 bucks a month to Facebook and 10 bucks a month to Twitter for some kind of more control in terms of advertisements and so on, but the other aspect of that is data, personal data, people are really sensitive about this. And I as one who hopes to one day create a company that may use people's data to do good for the world, wonder about this, one the psychology of why people are so paranoid, well I understand why, but they seem to be more paranoid than is justified at times. And the other is how do you do it right? So it seems that Facebook is, it seems that Facebook is doing it wrong. That's certainly the popular narrative. It's unclear to me actually how wrong, like I tend to give them more benefit of the doubt because they're, you know it's a really hard thing to do right, and people don't necessarily realize it, but how do we respect in your view people's privacy? Yeah, I mean in the case of how worried are people about using their data, I mean there's a lot of public debate and criticism about it. When we look at people's revealed preferences, you know people's continuing massive use of these sorts of services, it's not clear to me how much people really do care. Perhaps they care a bit, but they're happy to in effect kind of sell their data in order to be able to kind of use a certain service. That's a great term, revealed preferences. So these aren't preferences you self report in a survey, this is like your actions speak. Yeah, exactly, so you might say, oh yeah I hate the idea of Facebook having my data, but then when it comes to it, you actually are willing to give that data in exchange for being able to use the service. And if that's the case, then I think unless we have some explanation about why there's some negative externality from that or why there's some coordination failure, or if there's something that consumers are just really misled about where they don't realize why giving away data like this is a really bad thing to do, then ultimately I kind of want to respect people's preferences, they can give away their data if they want. I think there's a big difference between companies' use of data and governments having data where looking at the record of history, governments knowing a lot about their people can be very bad if the government chooses to do bad things with it. And that's more worrying I think. So let's jump into it a little bit. Most people know, but actually I two years ago had no idea what Effective Altruism was until I saw there was a cool looking event in an MIT group here, I think it's called the Effective Altruism Club or a group. I was like, what the heck is that? Yeah. And one of my friends said, he said that they're just a bunch of eccentric characters. So I was like, hell yes, I'm in. So I went to one of their events and looked up what's it about, it's quite a fascinating philosophical and just a movement of ideas. So can you tell me what is Effective Altruism? Great, so the core of Effective Altruism is about trying to answer this question, which is how can I do as much good as possible with my scarce resources, my time and with my money? And then once we have our best guess answers to that, trying to take those ideas and put that into practice and to do those things that we believe will do the most good and we're now a community of people, many thousands of us around the world who really are trying to answer that question as best we can and then use our time and money to make the world better. So what's the difference between sort of classical, general idea of altruism and Effective Altruism? So normally when people try to do good, they often just aren't so reflective about those attempts. So someone might approach you on the street asking you to give to charity and if you're feeling altruistic, you'll give to the person on the street. Or if you think, oh, I wanna do some good in my life, you might volunteer at a local place or perhaps you'll decide, you know, pursue a career where you're working in a field that's kind of more obviously beneficial like being a doctor or a nurse or a healthcare professional. But it's very rare that people apply the same level of rigor and analytical thinking to lots of other areas we think about. So take the case of someone approaching you on the street. Imagine if that person instead was saying, hey, I've got this amazing company, do you want to invest in it? It would be insane. No one would ever think, oh, of course, I'm just a company like you'd think it was a scam. But somehow we don't have that same level of rigor when it comes to doing good, even though the stakes are more important when it comes to trying to help others than trying to make money for ourselves. First of all, so there is a psychology at the individual level of doing good just feels good. And so in some sense, on that pure psychological part, it doesn't matter. In fact, you don't want to know if it does good or not because most of the time it won't. So like in a certain sense, it's understandable why altruism without the effective part is so appealing to a certain population. By the way, let's zoom out for a second. Do you think most people, two questions. Do you think most people are good? And question number two is, do you think most people want to do good? So are most people good? I think it's just super dependent on the circumstances that someone is in. I think that the actions people take and their moral worth is just much more dependent on circumstance than it is on someone's intrinsic character. So is it evil within all of us? It seems like the better angels of our nature, there's a tendency of us as a society to tend towards good, less war, I mean with all these metrics. What is that us becoming who we want to be? Or is that some kind of societal force? What's the nature versus nurture thing here? Yeah, so in that case, I just think, yeah, so violence has massively declined over time. I think that's a slow process of cultural evolution, institutional evolution, such that now the incentives for you and I to be violent are very, very small indeed. In contrast, when we were hunter-gatherers, the incentives were quite large. If there was someone who was potentially disturbing the social order and hunter-gatherer setting, there was a very strong incentive to kill that person. And people did. And it was just the guarded 10% of deaths among hunter-gatherers were murders. After hunter-gatherers, when you have actual societies, is when violence can probably go up, because there's more incentive to do mass violence, to take over or conquer other people's lands and murder everybody in place and so on. Yeah, I mean, I think total death rate from human causes does go down. But you're right that if you're in a hunter-gatherer situation, you're kind of a group that you're part of is very small, then you can't have massive wars, that just massive communities don't exist. But anyway, the second question, do you think most people want to do good? Yeah, and then I think that is true for most people. I think you see that with the fact that most people donate, a large proportion of people volunteer. If you give people opportunities to easily help other people, they will take it. But at the same time, we're a product of our circumstances. And if it were more socially rewarded to be doing more good, if it were more socially rewarded to do good effectively rather than not effectively, then we would see that behavior a lot more. So why should we do good? Yeah, my answer to this is there's no kind of deeper level of explanation. So my answer to kind of why should you do good is, well, there is someone whose life is on the line, for example, whose life you can save via donating just actually a few thousand dollars to an effective nonprofit, like the Against Malaria Foundation. That is a sufficient reason to do good. And then if you ask, well, why ought I to do that? I'm like, I just show you the same facts again. It's that fact that is the reason to do good. There's nothing more fundamental than that. I'd like to sort of make more concrete the thing we're trying to make better. So you just mentioned malaria. So there's a huge amount of suffering in the world. Are we trying to remove? So is ultimately the goal, not ultimately, but the first step is to remove the worst of the suffering? So there's some kind of threshold of suffering that we want to make sure does not exist in the world. Or do we really naturally want to take a much further step and look at things like income inequality? So not just getting everybody above a certain threshold, but making sure that there is some, that broadly speaking, there's less injustice in the world, unfairness, in some definition, of course, very difficult to define, and fairness. So the metric I use is how many people do we affect and by how much do we affect them? And so that can, often that means eliminating suffering. But it doesn't have to. Could be helping promote a flourishing life instead. And so if I was comparing reducing income inequality or getting people from the very pits of suffering to a higher level, the question I would ask is just a quantitative one of just, if I do this first thing or the second thing, how many people am I going to benefit? And by how much am I going to benefit? Am I going to move that one person from kind of 10% or 0% well-being to 10% well-being? Perhaps that's just not as good as moving 100 people from 10% well-being to 50% well-being. And the idea is the diminishing returns is the idea of when you're in terrible poverty, then the $1 that you give goes much further than if you were in the middle class in the United States, for example. Absolutely. And this fact is really striking. So if you take even just quite a conservative estimate of how we are able to turn money into well-being, the economists put it as like a log curve, or steeper. But that means that any proportional increase in your income has the same impact on your well-being. And so someone moving from $1,000 a year to $2,000 a year has the same impact as someone moving from $100,000 a year to $200,000 a year. And then when you combine that with the fact that we, in middle class members of rich countries, are 100 times richer in financial terms than the global poor, that means we can do 100 times to benefit the poorest people in the world as we can to benefit people of our income level. And that's this astonishing fact. Yeah, it's quite incredible. A lot of these facts and ideas are just difficult to think about. Because there's an overwhelming amount of suffering in the world. And even acknowledging it is difficult. I'm not exactly sure why that is. I mean, it's difficult because you have to bring to mind. It's an unpleasant experience thinking about other people's suffering. It's unpleasant to be empathizing with it, firstly. And then secondly, thinking about it means that maybe we'd have to change our lifestyles. And if you're very attached to the income that you've got, perhaps you don't want to be confronting ideas or arguments that might cause you to use some of that money to help others. So it's quite understandable in the psychological terms, even if it's not the right thing that we ought to be doing. So how can we do better? How can we be more effective? How does data help? In general, how can we do better? It's definitely hard. And we have spent the last 10 years engaged in some deep research projects to try and answer two questions. One is, of all the many problems the world is facing, what are the problems we ought to be focused on? And then within those problems that we judge to be the most pressing, where we use this idea of focusing on problems that are the biggest in scale, that are the most tractable, where we can make the most progress on that problem, and that are the most neglected. Within them, what are the things that have the best evidence or we have the best guess that will do the most good? And so we have a bunch of organizations. So GiveWell, for example, is focused on global health and development and has a list of seven top recommended charities. So the idea in general, and sorry to interrupt, is we'll talk about poverty and animal welfare and existential risk. Those are all fascinating topics. But in general, the idea is there should be a group. Sorry. There's a lot of groups that seek to convert money into good. And then you also, on top of that, want to have an accounting of how good they actually perform that conversion, how well they did in converting money to good. So ranking of these different groups, ranking these charities. So does that apply across basically all aspects of effective altruism? So there should be a group of people, and they should report on certain metrics of how well they've done. And you should only give your money to groups that do a good job. That's the core idea. I'd make two comments. One is just it's not just about money. So we're also trying to encourage people to work in areas where they'll have the biggest impact. Absolutely. And in some areas, they're really people heavy, but money poor. Other areas are kind of money rich and people poor. And so whether it's better to focus time or money depends on the cause area. And then the second is that you mentioned metrics. And while that's the ideal, and in some areas, we are able to get somewhat quantitative information about how much impact an area is having. That's not always true. For some of the issues, like you mentioned, existential risks, well, we're not able to measure in any sort of precise way how much progress we're making. And so you have to instead fall back on just rigorous argument and evaluation, even in the absence of data. So let's first sort of linger on your own story for a second. How do you yourself practice effective altruism in your own life? Because I think that's a really interesting place to start. So I've tried to build effective altruism into at least many components of my life. So on the donation side, my plan is to give away most of my income over the course of my life. I've set a bar I feel happy with, and I just donate above that bar. So at the moment, I donate about 20% of my income. Then on the career side, I've also shifted kind of what I do, where I was initially planning to work on very esoteric topics in the philosophy of logic, philosophy of language, things that are intellectually extremely interesting. But the path by which they really make a difference to the world is, let's just say it's very unclear at best. And so I switched instead to researching ethics to actually just working on this question of how we can do as much good as possible. And then I've also spent a very large chunk of my life over the last 10 years creating a number of nonprofits who, again, in different ways are tackling this question of how we can do the most good and helping them to grow over time too. Yeah, we'll mention a few of them with the career selection, 80,000. 80,000 hours. 80,000 hours is a really interesting group. So maybe also just a quick pause on the origins of effective altruism, because you paint a picture who the key figures are, including yourself in the effective altruism movement today. Yeah, there are two main strands that came together to form the effective altruism movement. So one was two philosophers, myself and Toby Ord at Oxford. And we had been very influenced by the work of Peter Singer, an Australian moral philosopher, who had argued for many decades that because one can do so much good at such little cost to oneself, we have an obligation to give away most of our income to benefit those in extreme poverty, just in the same way that we have an obligation to run in and save a child from drowning in a shallow pond if it were just to ruin your suit that cost a few thousand dollars. And we set up Giving What We Can in 2009, which is encouraging people to give at least 10% of their income to the most effective charities. And the second main strand was the formation of Give Well, which was originally based in New York and started in about 2007. And that was set up by Holden Karnavisi and Eli Hassenfeld, who were two hedge fund dudes who were making good money and thinking, well, where should I donate? And in the same way as if they wanted to buy a product for themselves, they would look at Amazon reviews. They were like, well, what are the best charities? Found they just weren't really good answers to that question, certainly not that they were satisfied with. And so they formed Give Well in order to try and work out what are those charities where they can have the biggest impact. And then from there and some other influences, community grew and spread. Can we explore the philosophical and political space that effective altruism occupies a little bit? So from the little and distant in my own lifetime that I've read of Ayn Rand's work, Ayn Rand's philosophy of objectivism, it espouses, and it's interesting to put her philosophy in contrast with effective altruism. So it espouses selfishness as the best thing you can do. But it's not actually against altruism. You have that choice, but you should be selfish in it, right? Or not. Maybe you can disagree here. So it can be viewed as the complete opposite of effective altruism, or it can be viewed as similar because the word effective is really interesting. Because if you want to do good, then you should be damn good at doing good, right? I think that would fit within the morality that's defined by objectivism. So do you see a connection between these two philosophies in other, perhaps, other, in this complicated space of beliefs that effective altruism is positioned as opposing or aligned with? I would definitely say that objectivism, Ayn Rand's philosophy, is a philosophy that's quite fundamentally opposed to effective altruism. In which way? Insofar as Ayn Rand's philosophy is about championing egoism and saying that I'm never quite sure whether the philosophy is meant to say that just you ought to do whatever will best benefit yourself, that's ethical egoism, no matter what the consequences are. Or second, if there's this alternative view, which is, well, you ought to try and benefit yourself because that's actually the best way of benefiting society. Certainly in Atlas Shluguchi is presenting her philosophy as a way that's actually going to bring about a flourishing society. And if it's the former, then well, effective altruism is all about promoting the idea of altruism and saying, in fact, we ought to really be trying to help others as much as possible. So it's opposed there. And then on the second side, I would just dispute the empirical premise. Given the major problems in the world today, it would seem like this remarkable coincidence. Quite suspicious, one might say, if benefiting myself was actually the best way to bring about a better world. So on that point, and I think that connects also with career selection that we'll talk about, but let's consider not object to this, but capitalism. And the idea that you focusing on the thing that you are damn good at, whatever that is, may be the best thing for the world. Part of it is also mindset. The thing I love is robots. So maybe I should focus on building robots and never even think about the idea of effective altruism, which is kind of the capitalist notion. Is there any value in that idea in just finding the thing you're good at and maximizing your productivity in this world, and thereby sort of lifting all boats and benefiting society as a result? Yeah, I think there's two things I'd want to say on that. So one is what your comparative advantages, what your strengths are when it comes to career, is obviously super important because there's lots of career paths I would be terrible at. If I thought being an artist was the best thing one could do, well, I'd be doomed. Just really quite astonishingly bad. And so I do think, at least within the realm of things that could plausibly be very high impact, choose the thing that you think you're going to be able to really be passionate at and excel at over the long term. Then on this question of should one just do that in an unrestricted way and not even think about what the most important problems are, I do think that in a perfectly designed society, that might well be the case. That would be a society where we've corrected all market failures, we've internalized all externalities, and then we've managed to set up incentives such that people just pursuing their own strengths is the best way of doing good. But we're very far from that society. So if one did that, then it would be very unlikely that you would focus on improving the lives of non-human animals that aren't participating in markets or ensuring the long run future goes well where future people certainly aren't participating in markets or benefiting the global poor who do participate but have so much less power from a starting perspective that their views aren't accurately represented by market forces too. Got it. So yeah, pure definition capitalism just may very well ignore the people that are suffering the most, the white swath of them. So if you could allow me this line of thinking here. So I've listened to a lot of your conversations online. I find, if I can compliment you, they're very interesting conversations. Your conversation on Rogan, on Joe Rogan was really interesting with Sam Harris and so on. Whatever. There's a lot of stuff that's really good out there. And yet, when I look at the internet, and I look at YouTube, which has certain mobs, certain swaths of right leaning folks whom I dearly love. I love all people, especially people with ideas. They seem to not like you very much. So I don't understand why exactly. So my own hypothesis is there is a right-left divide that, absurdly so, caricatured in politics, at least in the United States. And maybe you're somehow pigeonholed into one of those sides. And maybe that's what it is. Maybe your message is somehow politicized. Yeah. How do you make sense of that? Because you're extremely interesting. Like, you got the comments I see on Joe Rogan. There's a bunch of negative stuff. And yet, if you listen to it, the conversation is fascinating. I'm not speaking, I'm not some kind of lefty extremist. But just this fascinating conversation. So why are you getting some small amount of hate? So I'm actually pretty glad that effective altruism has managed to stay relatively unpoliticized. Because I think the core message, just use some of your time and money to do as much good as possible to fight some of the problems in the world, can be appealing across the political spectrum. And we do have a diversity of political viewpoints among people who have engaged in effective altruism. We do, however, do get some criticism from the left and the right. Oh, interesting. Both would be interesting to hear. Yeah. So criticism from the left is that we're not focused enough on dismantling the capitalist system that they see as the root of most of the problems that we're talking about. And there, I kind of disagree on partly the premise where I don't think relevant alternative systems would say to the animals or to the global poor or to the future generations much better. And then also the tactics where I think there are particular ways we can change society that would be massively beneficial on those things that don't go via dismantling the entire system, which is perhaps a million times harder to do. Then criticism on the right, there's definitely in response to the Joe Rogan podcast, there definitely were a number of A and Land fans who weren't keen on the idea of promoting altruism. There was a remarkable set of ideas, just the idea that effective altruism was unmanly, I think, was deriving a lot of criticism. Let me, OK, so I love fighting. I've been in street fights my whole life. As alpha in everything I do as it gets. And the fact that Joe Rogan said that I thought Scent of a Woman is a better movie than John Wick put me into this beta category amongst people who are like basically saying that it's yeah, unmanly or it's not tough. It's not there's not some principled view of strength that is represented by a spaz blizzard. So actually, so how do you think about this? Because to me, altruism, especially effective altruism, is I don't know what the female version of that is, but on the male side, manly as fuck, if I may say so. So how do you think about that kind of criticism? I think people who would make that criticism are just occupying a state of mind that I think is just so different from my state of mind that I kind of struggle to maybe even understand it, where if something's manly or unmanly or feminine or unfeminine, I'm like, I don't care. Like, is it the right thing to do or the wrong thing to do? So let me put it not in terms of man or woman. Yeah. I don't think that's useful. But I think there's a notion of acting out of fear. Yeah. That or as opposed to out of principle and strength. Yeah. So OK, yeah, here's something that I do feel as an intuition and that I think drives some people who do find kind of A and land attractive and so on as a philosophy, which is a kind of taking control of your own life and having power over how you're steering your life and not kind of kowtowing to others, you know, really thinking things through. I find like that set of ideas just very compelling and inspirational. But I actually think of the effect of altruism has really that side of my personality. It's like scratch that itch where you are just not taking the kind of priorities that society is giving you as granted. Instead, you're choosing to act in accordance with the priorities that you think are most important in the world. And often that involves then doing quite unusual things from a societal perspective, like donating a large chunk of your earnings or working on these weird issues about AI and so on that other people might not understand. Yeah, I think that's a really gutsy thing to do. That is taking control. That's at least at this stage, I mean, that's you taking ownership not of just yourself, but your presence in this world that's full of suffering and saying as opposed to being paralyzed by that notion. Yeah. Taking control and saying I could do something. Yeah. I mean, that's really powerful. But the one thing I personally hate too about the left currently that I think those folks to detect is the social signaling. When you look at yourself late at night, would you do everything you're doing in terms of effective altruism if your name, because you're quite popular, but if your name was totally unattached to it? Yeah. If it was in secret? Yeah, I mean, I think I would. To be honest, I think the kind of popularity is like, it's a mixed bag, but there are serious costs. And I don't particularly, I don't love it. It means you get all these people calling you a cuck on Joe Rogan. It's not the most fun thing. But you also get a lot of brownie points for doing good for the world. Yeah, you do. But I think my ideal life would be in some library solving logic puzzles all day. And I'd really be learning maths and so on. So you're in a good body of friends and so on. So your instinct for effective altruism is something deep. It's not one that is communicating socially. It's more in your heart you want to do good for the world. Yeah, I mean, so we can look back to early giving what we can. So we're setting this up, me and Toby. And I really thought that doing this would be a big hit to my academic career, because I was now spending, at that time, more than half my time setting up this nonprofit at the crucial time when you should be producing your best academic work and so on. And it was also the case at the time, it was kind of like the Toby Ord Club. He was the most popular. There was this personal interest story around him. His plan is to donate. Sorry to interrupt, but Toby was donating a large amount. Can you tell just briefly what he was doing? Yeah, so he made this public commitment to give everything he earned above 20,000 pounds per year to the most effective causes. And even as a graduate student, he was still donating about 15%, 20% of his income, which is so quite significant, given that graduate students are not known for being super wealthy. And when we launched Giving What We Can, the media just loved this as a personal interest story. So the story about him and his pledge was the most, yeah, it was actually the most popular news story of the day. And we ran the same story a year later, and it was the most popular news story of the day a year later, too. And so it really was several years before then I was also giving more talks and starting to do more writing. And then especially with this book, Doing Good Better, then there started to be attention and so on. But deep inside, your own relationship with effective altruism was, I mean, it had nothing to do with the publicity. Did you see yourself, how did the publicity connect with it? Yeah, I mean, that's kind of what I'm saying, is I think the publicity came several years afterwards. I mean, at the early stage when we set up Giving What We Can, it was really just every person we get to pledge 10% is something like $100,000 over their lifetime. That's huge. And so it was just we had started with 23 members. Every single person was just this kind of huge accomplishment. And at the time, I just really thought, maybe over time we'll have 100 members, and that'll be amazing. Whereas now we have over $4,000 and $1.5 billion pledged. That's just unimaginable to me at the time when I was first kind of getting this stuff off the ground. So can we talk about poverty and the biggest problems that you think in the near term effective altruism can attack in each one? So poverty obviously is a huge one. Yeah. How can we help? Great, yeah. So poverty, absolutely this huge problem. 700 million people in extreme poverty living in less than $2 per day. What that means is what $2 would buy in the US. So think about that. It's like some rice, maybe some beans. It's really not much. And at the same time, we can do an enormous amount to improve the lives of people in extreme poverty. So the things that we tend to focus on are interventions in global health. And that's for a few reasons. One is that global health just has this amazing track record. Life expectancy globally is up 50% relative to 60 or 70 years ago. We've eradicated smallpox, which killed 2 million lives every year, almost eradicated polio. Second is that we just have great data on what works when it comes to global health. So we just know that bed nets protect children and prevent them from dying from malaria. And then the third is just that it's extremely cost effective. So it costs $5 to buy one bed net, protects two children for two years against malaria. If you spend about $3,000 on bed nets, then statistically speaking, you're going to save a child's life. And there are other interventions too. And so given the people in such suffering, and we have this opportunity to do such huge good for such low cost, well, yeah, why not? So the individuals, so for me today, if I wanted to look at poverty, how would I help? And I wanted to say, I think donating 10% of your income is a very interesting idea, or some percentage or some. Setting a bar and sort of sticking to it, how do we then take the step towards the effective part? So you've conveyed some notions, but who do you give the money to? Yeah, so GiveWell, this organization I mentioned, well, it makes charity recommendations. And some of its top recommendations, so Against Malaria Foundation is this organization that buys and distributes these insecticide-deleted bed nets. And then it has a total of seven charities that it recommends very highly. So that recommendation, is it almost like a star of approval? Or is there some metrics? So what are the ways that GiveWell conveys that this is a great charity organization? Yeah, so GiveWell is looking at metrics. And it's trying to compare charities ultimately in the number of lives that you can save or an equivalent benefit. So one of the charities it recommends is GiveDirectly, which simply just transfers cash to the poorest families, where a poor family will get a cash transfer of $1,000. And they kind of regard that as the baseline intervention, because it's so simple and people, they know what to do with, how to benefit themselves. That's quite powerful, by the way. So before GiveWell, before the Effective Altruism Movement, was there, I imagine, there's a huge amount of corruption, funny enough, in charity organizations, or misuse of money. So there was nothing like GiveWell before that? No, I mean, there were some. So I mean, the charity corruption, I mean, obviously there's some. I don't think it's a huge issue, relative to just focusing on the wrong things. Prior to GiveWell, there were some organizations like Charity Navigator, which were more aimed at worrying about corruption and so on. So they weren't saying, these are the charities where you're going to do the most good. Instead, it was like, how good are the charities' financials? How good is this? How far are they transparent? And yeah, so that would be more useful for weeding out some of those worst charities. So GiveWell, just taking it a step further, sort of in this 21st century of data, it's actually looking at the effective part. Yeah, so it's like, if you know the wire cutter, if you want to buy a pair of headphones, they will just look at all the headphones and be like, these are the best headphones you can buy. And that's the idea with GiveWell. Okay, so do you think there's a bar of what suffering is and do you think one day we can eradicate suffering in our world? Yeah, exactly. Amongst humans? Let's talk humans for now. Talk humans, but in general, yeah, actually. So there's a colleague of mine calling the term abolitionism for the idea that we should just be trying to abolish suffering. And in the long run, I mean, I don't expect it anytime soon, but I think we can. I think that would require quite fantastic changes to the way society is structured and perhaps even the, in fact, even changes to human nature. But I do think that suffering, whenever it occurs is bad and we should want it to not occur. So there's a line, there's a gray area between suffering. Now I'm Russian, so I romanticize some aspects of suffering. There's a gray line between struggle, gray area between struggle and suffering. So one, do we want to eradicate all struggle in the world? So there's an idea that the human condition inherently has suffering in it. And it's a creative force. It's a struggle of our lives and we somehow grow from that. How do you think about that? I agree that's true. So, often, great artists can be also suffering from major health conditions or depression and so on. Or they come from abusive parents. Most great artists, they think come from abusive parents. Yeah, that seems to be at least commonly the case. But I want to distinguish between suffering as being instrumentally good, it causes people to produce good things and whether it's intrinsically good. And I think intrinsically, it's always bad. And so if we can produce these great achievements via some other means where, if we look at the scientific enterprise, we've produced incredible things, often from people who aren't suffering, who have pretty good lives. They're driven instead of being pushed by a certain sort of anguish, they're being driven by intellectual curiosity. If we can instead produce a society where it's all carrot and no stick, that's better from my perspective. Yeah, but I'm gonna have to disagree with the notion that that's possible. But I would say most of the suffering in the world is not productive. So I would dream of effective altruism curing that suffering. But then I would say that there is some suffering that is productive that we wanna keep. But that's not even the focus of, because most of the suffering is just absurd that needs to be eliminated. So let's not even romanticize this usual notion I have, but nevertheless, struggle has some kind of inherent value that to me at least, you're right, there's some elements of human nature that also have to be modified in order to cure all suffering. Yeah, I mean, there's an interesting question of whether it's possible. So at the moment, most of the time we're kind of neutral, and then we burn ourselves and that's negative. And that's really good that we get that negative signal because it means we won't burn ourselves again. There's a question like, could you design agents, humans, such that you're not hovering around the zero level, you're hovering at like bliss, and then you touch the flame and you're like, oh no, you're just slightly worse bliss. But that's really bad compared to the bliss you were normally in. So you can have like a gradient of bliss instead of like pain and pleasure. On that point, I think it's a really important point on the experience of suffering, the relative nature of it. Having grown up in the Soviet Union, we're quite poor by any measure and when I was in my childhood, but it didn't feel like you were poor because everybody around you were poor. And then in America, I feel, I for the first time begin to feel poor. Yeah, yeah. Because of the, there's different, there's some cultural aspects to it that really emphasize that it's good to be rich. And then there's just the notion that there is a lot of income inequality and therefore you experience that inequality. That's where suffering comes. So what do you think about the inequality of suffering that we have to think about? Do you think we have to think about that as part of effective altruism? Yeah, I think we're just, things vary in terms of whether you get benefits or costs from them just in relative terms or in absolute terms. So a lot of the time, yeah, there's this hedonic treadmill where if you get, you know, there's, money is useful because it helps you buy things or good for you because it helps you buy things. But there's also a status component too. And that status component is kind of zero sum. If you were saying like in Russia, you know, no one else felt poor because everyone around you was poor. Whereas now you've got this, these other people who are, you know, super rich and maybe that makes you feel, you know, less good about yourself. There are some other things however, which are just intrinsically good or bad. So commuting, for example, it's just people hate it. It doesn't really change. Knowing that other people are commuting too, doesn't make it any kind of less bad. But sort of to push back on that for a second, I mean, yes, but also if some people were, you know, on horseback, your commute on the train might feel a lot better. Yeah. You know, there is a relative, I mean, everybody's complaining about society today forgetting it's forgetting how much better it is, the better it is in the ages of our nature, how the technology is improved, fundamentally improving most of the world's lives. Yeah, and actually there's some psychological research on the wellbeing benefits of volunteering, where people who volunteer tend to just feel happier about their lives. And one of the suggested explanations is it because it extends your reference class. So no longer are you comparing yourself to the Joneses who have their slightly better car, but you realize that, you know, people in much worse conditions than you. And so now, you know, your life doesn't seem so bad. That's actually on a psychological level, one of the fundamental benefits of effective altruism. Yeah. Is, I mean, I guess it's the altruism part of effective altruism is exposing yourself to the suffering in the world allows you to be more, yeah, happier, and actually allows you in a sort of meditative introspective way realize that you don't need most of the wealth you have to be happy. Absolutely, I mean, I think effective altruism has been this huge benefit for me. And I really don't think that if I had more money that I was living on, that that would change my level of wellbeing at all. Whereas engaging in something that I think is meaningful, that I think is steering humanity in a positive direction, that's extremely rewarding. And so, yeah, I mean, despite my best attempts at sacrifice, I don't, you know, I think I've actually ended up happier as a result of engaging in effective altruism than I would have done. That's such an interesting idea. So let's talk about animal welfare. Sure. Easy question, what is consciousness? Especially as it has to do with the capacity to suffer. I think there seems to be a connection between how conscious something is, the amount of consciousness and its ability to suffer. And that all comes into play about us thinking how much suffering there is in the world with regard to animals. So how do you think about animal welfare and consciousness? Okay, well, consciousness, easy question. Yeah, I mean, I think we don't have a good understanding of consciousness. My best guess is it's got, and by consciousness, I'm meaning what it feels like to be you, the subjective experience that seems to be different from everything else we know about in the world. Yeah, I think it's clear it's very poorly understood at the moment. I think it has something to do with information processing. So the fact that the brain is a computer or something like a computer. So that would mean that very advanced AI could be conscious, information processors in general could be conscious with some suitable complexity. But that also, some suitable complexity, it's a question whether greater complexity creates some kind of greater consciousness, which relates to animals. Yeah. Right, is there, if it's an information processing system and it's smaller and smaller, is an ant less conscious than a cow, less conscious than a monkey? Yeah, and again, this super hard question, but I think my best guess is yes. Like if I think, well, consciousness, it's not some magical thing that appears out of nowhere. It's not, Descartes thought it was just, comes in from this other realm and then enters through the pineal gland in your brain and that's kind of soul in its conscious. So it's got something to do with what's going on in your brain. A chicken has one 300th of the size of the brain that you have. Ants, I don't know how small it is. Maybe it's a millionth the size. My best guess, which I may well be wrong about because this is so hard, is that in some relevant sense, the chicken is experiencing consciousness to a less degree than the human and the ant significantly less again. I don't think it's as little as 300th as much, I think. There's evolutionary reasons for thinking that the ability to feel pain comes on the scene relatively early on and we have lots of our brain that's dedicated to stuff that doesn't seem to have to do anything to do with consciousness, language processing and so on. So it seems like the easy, so there's a lot of complicated questions there that we can't ask the animals about, but it seems that there's easy questions in terms of suffering, which is things like factory farming that could be addressed. Is that the lowest hanging fruit, if I may use crude terms here, of animal welfare? Absolutely, I think that's the lowest hanging fruit. So at the moment we kill, we raise and kill about 50 billion animals every year. So how many? 50 billion. Yeah, so for every human on the planet, several times that number are being killed. And the vast majority of them are raised in factory farms where basically whatever your view on animals, I think you should agree, even if you think, well, maybe it's not bad to kill an animal, maybe if the animal was raised in good conditions, that's just not the empirical reality. The empirical reality is that they are kept in incredible cage confinement, they are de-beaked or detailed without anesthetic, chickens often peck each other to death, like otherwise, because of such stress. It's really, I think when a chicken gets killed, that's the best thing that happened to the chicken in the course of its life. And it's also completely unnecessary. This is in order to save a few pence for the price of meat or price of eggs. And we have indeed found, it's also just inconsistent with consumer preference as well. People who buy the products, if they could, they, when you do surveys, are extremely against suffering in factory farms. It's just that they don't appreciate how bad it is and just tend to go with the easy options. And so then the best, the most effective programs I know of at the moment are nonprofits that go to companies and work with companies to get them to take a pledge to cut certain sorts of animal products like eggs from cage confinement out of their supply chain. And it's now the case that the top 50 food retailers and fast food companies have all made these kind of cage-free pledges. And when you do the numbers, you get the conclusion that every dollar you're giving to these nonprofits result in hundreds of chickens being spared from cage confinement. And then they're working to other types of animals, other products too. So is that the most effective way to do, have a ripple effect essentially? As opposed to directly having regulation from on top that says you can't do this? So I would be more open to the regulation approach, but at least in the US, there's quite intense regulatory capture from the agricultural industry. And so attempts that we've seen to try and change regulation have, it's been a real uphill struggle. There are some examples of ballot initiatives where the people have been able to vote in a ballot to say we wanna ban eggs from cage conditions. And that's been huge, that's been really good. But beyond that, it's much more limited. So I've been really interested in the idea of hunting in general and wild animals and seeing nature as a form of cruelty that I am ethically more okay with. Just from my perspective. And then I read about wild animal suffering. I'm just giving you the kind of notion of how I felt because animal factory farming is so bad that living in the woods seemed good. And yet when you actually start to think about it, all of the animals in the animal world are living in terrible poverty, right? So you have all the medical conditions, all of that. I mean, they're living horrible lives, it could be improved. That's a really interesting notion that I think may not even be useful to talk about because factory farming is such a big thing to focus on. But it's nevertheless an interesting notion to think of all the animals in the wild as suffering in the same way that humans and poverty are suffering. Yeah, I mean, and often even worse. So many animals we produce by our selection. So you have a very large number of children in the expectation that only a small number survive. And so for those animals, almost all of them just live short lives where they starve to death. So yeah, there's huge amounts of suffering in nature that I don't think we should pretend that it's this kind of wonderful paradise for most animals. Yeah, their life is filled with hunger and fear and disease. I agree with you entirely that when it comes to focusing on animal welfare, we should focus on factory farming. But we also, yeah, should be aware to the reality of what life for most animals is like. So let's talk about a topic I've talked a lot about and you've actually quite eloquently talked about, which is the third priority that effective altruism considers as really important is existential risks. When you think about the existential risks that are facing our civilization, what's before us? What concerns you? What should we be thinking about from the, especially from an effective altruism perspective? Great, so the reason I started getting concerned about this was thinking about future generations where the key idea is just, well, future people matter morally. There are vast numbers of future people. If we don't cause our own extinction, there's no reason why civilization might not last a million years. I mean, we last as long as a typical mammalian species. Or a billion years is when the earth is no longer habitable. Or if we can take to the stars, then perhaps it's trillions of years beyond that. So the future could be very big indeed. And it seems like we're potentially very early on in civilization. Then the second idea is just, well, maybe there are things that are gonna really derail that. Things that actually could prevent us from having this long, wonderful civilization. And instead, could cause our own extinction or otherwise perhaps like lock ourselves into a very bad state. And what ways could that happen? Well, causing our own extinction, the development of nuclear weapons in the 20th century, at least put on the table that we now had weapons that were powerful enough that you could very significantly destroy society. Perhaps an all-black nuclear war would cause a nuclear winter. Perhaps that would be enough for the human race to go extinct. When you think we haven't done it, sorry to interrupt, why do you think we haven't done it yet? Is it surprising to you that having always for the past few decades, several thousand of active, ready to launch nuclear weapons, warheads, and yet we have not launched them ever since the initial launch on Hiroshima and Nagasaki? I think it's a mix of luck. So I think it's definitely not inevitable that we haven't used them. So John F. Kennedy, general Cuban missile crisis, put the estimate of nuclear exchange between the US and USSR at somewhere between one and three and even. So we really did come close. At the same time, I do think mutually assured destruction is a reason why people don't go to war. It would be why nuclear powers don't go to war. Do you think that holds, if you can linger on that for a second, like my dad is a physicist amongst other things, and he believes that nuclear weapons are actually just really hard to build, which is one of the really big benefits of them currently. So that you don't have, it's very hard if you're crazy to acquire a nuclear weapon. So the mutually assured destruction really works when you talk, seems to work better when it's nation states, when it's serious people, even if they're a little bit, you know, dictatorial and so on. Do you think this mutually assured destruction idea will carry, how far will it carry us in terms of different kinds of weapons? Oh yeah, I think it's your point that nuclear weapons are very hard to build and relatively easy to control because you can control fissile material, is a really important one. And future technology that's equally destructive might not have those properties. So for example, if in the future, people are able to design viruses, perhaps using a DNA printing kit that's on, that, you know, one can just buy. In fact, there are companies in the process of creating home DNA printing kits. Well then perhaps that's just totally democratized, perhaps the power to reap huge destructive potential is in the hands of most people in the world, or certainly most people with effort. And then, yeah, I no longer trust mutually assured destruction because for some people, the idea that they would die is just not a disincentive. There was a Japanese cult, for example, Omshin Rikkyo in the 90s that had, but they what they believed was that Armageddon was coming. If you died before Armageddon, you would get good karma, you wouldn't go to hell. If you died during Armageddon, maybe you would go to hell. And they had a biological weapons program, a chemical weapons program. When they were finally apprehended, they hadn't stocks of saline gas that were sufficient to kill four million people, engaged in multiple terrorist acts. If they had had the ability to print a virus at home, that would have been very scary. So it's not impossible to imagine groups of people that hold that kind of belief of death and suicide as a good thing for passage into the next world and so on. And then connect them with some weapons. Then ideology and weaponry may create serious problems for us. Let me ask you a quick question on what do you think is the line between killing most humans and killing all humans? How hard is it to kill everybody? Have you thought about this? I've thought about it a bit. I think it is very hard to kill everybody. So in the case of, let's say, an all-out nuclear exchange, and let's say that leads to nuclear winter, we don't really know, but it might well happen. That would, I think, result in billions of deaths. Would it kill everybody? It's quite hard to see how it would kill everybody for a few reasons. One is just there's just so many people. 7.5 billion people. So this bad event has to kill almost all of them. Secondly, live in such a diversity of locations. So a nuclear exchange or the virus, it has to kill people who live on the coast of New Zealand, which is gonna be climatically much more stable than other areas in the world. Or people who are on submarines. Or who have access to bunkers. So there's a very... I'm sure there's two guys in Siberia just badass. There's just human nature somehow just perseveres. Yeah, and then the second thing is just if there's some catastrophic event, people really don't want to die. So there's gonna be huge amounts of effort to ensure that it doesn't affect everyone. If you thought about what it takes to rebuild a society with smaller and smaller numbers, like how big of a setback these kinds of things are. Yeah, so then that's something where there's real uncertainty, I think. Where at some point you just lose sufficient genetic diversity such that you can't come back. It's unclear how small that population is, but if you've only got say a thousand people or fewer than a thousand, then maybe that's small enough. What about human knowledge? And then there's human knowledge. I mean, it's striking how short on geological timescales or evolutionary timescales the progress in or how quickly the progress in human knowledge has been. Agriculture we only invented in 10,000 BC. Cities were only 3000 BC. Whereas typical mammal species is half a million years to a million years. Do you think it's inevitable in some sense? Agriculture, everything that came, the Industrial Revolution, cars, planes, the internet, that level of innovation you think is inevitable? I think given how quickly it arose, so in the case of agriculture, I think that was dependent on climate. It was the kind of glacial period was over, the earth warmed up a bit. That made it much more likely that humans would develop agriculture. When it comes to the Industrial Revolution, it's just, again, it only took a few thousand years from cities to Industrial Revolution. If we think, okay, we've gone back to this, even let's say agricultural era, but there's no reason why we wouldn't go extinct in the coming tens of thousands of years or hundreds of thousands of years, it would be very surprising if we didn't rebound, unless there's some special reason that makes things different. So perhaps we just have a much greater disease burden now. So HIV exists, it didn't exist before. And perhaps that's latent in being suppressed by modern medicine and sanitation and so on, but would be a much bigger problem for some utterly destroyed society that was trying to rebound. Or there's just maybe there's something we don't know about. So another existential risk comes from the mysterious, the beautiful artificial intelligence. Yeah. So what's the shape of your concerns about AI? I think there are quite a lot of concerns about AI, and sometimes the different risks don't get distinguished enough. So the kind of classic worry, most is closely associated with Nick Bostrom and Elias Joukowsky is that we at some point move from having narrow AI systems to artificial general intelligence. You get this very fast feedback effect where AI is able to build, artificial intelligence helps you to build greater artificial intelligence. We have this one system that's suddenly very powerful, far more powerful than others than perhaps far more powerful than the rest of the world combined. And then secondly, it has goals that are misaligned with human goals. And so it pursues its own goals. It realized, hey, there's this competition, namely from humans, it would be better if we eliminated them. In just the same way as Homo sapiens eradicated the Neanderthals, in fact, it in fact killed off most large animals on the planet that walked the planet. So that's kind of one set of worries. I think that's not my, I think these shouldn't be dismissed as science fiction. I think it's something we should be taking very seriously. But it's not the thing you visualize when you're concerned about the biggest near term. Yeah, I think it's like one possible scenario that would be astronomically bad. I think there are other scenarios that would also be extremely bad, comparably bad, that are more likely to occur. So one is just we are able to control AI. So we're able to get it to do what we want it to do. And perhaps there's not like this fast takeoff of AI capabilities within a single system. It's distributed across many systems that do somewhat different things. But you do get very rapid economic and technological progress as a result. That concentrates power into the hands of a very small number of individuals, perhaps a single dictator. And secondly, that single individual is, or small group of individuals or single country, is then able to like lock in their values indefinitely via transmitting those values to artificial systems that have no reason to die. Like, their code is copyable. Perhaps Donald Trump or Xi Jinping creates their kind of AI progeny in their own image. And once you have a system that's, once you have a society that's controlled by AI, you no longer have one of the main drivers of change historically, which is the fact that human lifespans are only a hundred years, give or take. So that's really interesting. So as opposed to sort of killing off all humans, is locking in, like creating a hell on earth, basically a set of principles under which the society operates that's extremely undesirable. So everybody is suffering indefinitely. Or it doesn't, I mean, it also doesn't need to be hell on earth. It could just be the wrong values. So we talked at the very beginning about how I wanna see this kind of diversity of different values and exploration so that we can just work out what is kind of morally right, what is good, what is bad, and then pursue the thing that's best. So actually, so the idea of wrong values is actually probably, the beautiful thing is there's no such thing as right and wrong values because we don't know the right answer. We just kind of have a sense of which value is more right and which is more wrong. So any kind of lock-in makes a value wrong because it prevents exploration of this kind. Yeah, and just imagine if fascist, imagine if there was Hitler's utopia or Stalin's utopia or Donald Trump's or Xi Jinping's forever. Yeah. You know, how good or bad would that be compared to the best possible future we could create? And my suggestion is it would really suck compared to the best possible future we could create. And you're just one individual. There's some individuals for whom Donald Trump is perhaps the best possible future. And so that's the whole point of us individuals exploring this space together. Exactly, yeah. And what's trying to figure out, which is the path that will make America great again? Yeah, exactly. So how can effective altruism help? I mean, this is a really interesting notion they actually describe being of artificial intelligence being used as extremely powerful technology in the hands of very few, potentially one person, to create some very undesirable effect. So as opposed to AI, and again, the source of the undesirableness there is the human. AI is just a really powerful tool. So whether it's that or whether AI's AGI just runs away from us completely, how as individuals, as people in the effective altruism movement, how can we think about something like this? I understand poverty and welfare, but this is a far out incredibly mysterious and difficult problem. Great, well, I think there's three paths as an individual. So if you're thinking about career paths you can pursue, so one is going down the line of technical AI safety. So this is most relevant to the kind of AI winning, AI taking over scenarios where this is just technical work on current machine learning systems, often sometimes going more theoretical too, on how we can ensure that an AI is able to learn human values and able to act in the way that you want it to act. And that's a pretty mainstream issue and approach in machine learning today. So we definitely need more people doing that. Second is on the policy side of things, which I think is even more important at the moment, which is how should developments in AI be managed on a political level? How can you ensure that the benefits of AI are very distributed, power isn't being concentrated in the hands of a small set of individuals? How do you ensure that there aren't arms races between different AI companies that might result in them cutting corners with respect to safety? And so there the input as individuals who can have is this, we're not talking about money, we're talking about effort. We're talking about career choices. Yeah, we're talking about career choice, yeah. But then it is the case that supposing, you're like, I've already decided my career and I'm doing something quite different, you can contribute with money too, where at the Center for Effective Altruism, we set up the Long-Term Future Fund. So if you go onto effectivealtruism.org, you can donate where a group of individuals will then work out what's the highest value place they can donate to work on existential risk issues with a particular focus on AI. What's path number three? This was path number three. This was donations were the third option I was thinking of. And then yeah, you can also donate directly to organizations working on this like Center for Human Compatible AI at Berkeley, Future of Humanity Institute at Oxford, or other organizations too. Does AI keep you up at night, this kind of concern? Yeah, it's kind of a mix where I think it's very likely things are gonna go well. I think we're gonna be able to solve these problems. I think that's by far the most likely outcome, at least over the next. By far the most likely. So if you look at all the trajectories running away from our current moment in the next 100 years, you see AI creating destructive consequences as a small subset of those possible trajectories. Or at least yeah, kind of eternal destructive consequences. I think that being a small subset. At the same time, it still freaks me out. I mean, when we're talking about the entire future of civilization, then small probabilities, 1% probability, that's terrifying. What do you think about Elon Musk's strong worry that we should be really concerned about existential risks of AI? Yeah, I mean, I think, broadly speaking, I think he's right. I think if we talked, we would probably have very different probabilities on how likely it is that we're doomed. But again, when it comes to talking about the entire future of civilization, it doesn't really matter if it's 1% or if it's 50%. We ought to be taking every possible safeguard we can to ensure that things go well rather than poorly. Last question, if you yourself could eradicate one problem from the world, what would that problem be? That's a great question. I don't know if I'm cheating in saying this, but I think the thing I would most want to change is just the fact that people don't actually care about ensuring the long run future goes well. People don't really care about future generations. They don't think about it. It's not part of their aims. Well, in some sense, you're not cheating at all, because in speaking the way you do, in writing the things you're writing, you're addressing exactly this aspect. Exactly. That is your input into the effect of altruism movement. So for that, Will, thank you so much. It's an honor to talk to you. I really enjoyed it. Thanks so much for having me on. Thanks for listening to this conversation with William McCaskill. Thank you to our presenting sponsor, Cash App. Please consider supporting the podcast by downloading Cash App and using code LexPodcast. If you enjoy this podcast, subscribe on YouTube, review it with five stars on Apple Podcasts, support it on Patreon, or simply connect with me on Twitter at Lex Friedman. And now let me leave you with some words from William McCaskill. One additional unit of income can do 100 times as much to benefit the extreme poor as it can to benefit you or I, earning the typical US wage of $28,000 a year. It's not often that you have two options, one of which is 100 times better than the other. Imagine a happy hour where you can either buy yourself a beer for $5 or buy someone else a beer for 5 cents. If that were the case, we'd probably be pretty generous. Next round's on me. But that's effectively the situation we're in all the time. It's like a 99% off sale or buy one, get 99 free. Might be the most amazing deal you'll see in your life. Thank you for listening and hope to see you next time.VoiceoverIf this video could gateetztthat it, thank you so much.
 The following is a conversation with William McCaskill.
 He's a philosopher, ethicist, and one of the originators
 of the effective altruism movement.
 His research focuses on the fundamentals
 of effective altruism, or the use of evidence and reason
 to help others by as much as possible
 with our time and money.
 With a particular concentration
 on how to act given moral uncertainty.
 He's the author of Doing Good, Better, Effective Altruism
 and a Radical New Way to Make a Difference.
 He is a co-founder and the president
 of the Center of Effective Altruism, CEA,
 that encourages people to commit to donate
 at least 10% of their income
 to the most effective charities.
 He co-founded 80,000 Hours, which is a nonprofit
 that provides research and advice
 on how you can best make a difference through your career.
 This conversation was recorded before the outbreak
 of the coronavirus pandemic.
 For everyone feeling the medical, psychological,
 and financial burden of this crisis,
 I'm sending love your way.
 Stay strong, we're in this together.
 We'll beat this thing.
 This is the Artificial Intelligence Podcast.
 If you enjoy it, subscribe on YouTube,
 review it with five stars on Apple Podcasts,
 support it on Patreon, or simply connect with me
 on Twitter, Alex Friedman, spelled F-R-I-D-M-A-N.
 As usual, I'll do one or two minutes of ads now
 and never any ads in the middle.
 They can break the flow of the conversation.
 I hope that works for you
 and doesn't hurt the listening experience.
 This show is presented by Cash App,
 the number one finance app in the App Store.
 When you get it, use code LEXPODCAST.
 Cash App lets you send money to friends,
 buy Bitcoin, and invest in the stock market
 with as little as $1.
 Since Cash App allows you to send
 and receive money digitally, peer-to-peer,
 and security in all digital transactions is very important,
 let me mention the PCI data security standard
 that Cash App is compliant with.
 I'm a big fan of standards for safety and security.
 PCI DSS is a good example of that,
 where a bunch of competitors got together and agreed
 that there needs to be a global standard
 around the security of transactions.
 Now, we just need to do the same for autonomous vehicles
 and AI systems in general.
 So again, if you get Cash App from the App Store,
 Google Play, and use the code LEXPODCAST,
 you get $10, and Cash App will also donate $10 to FIRST,
 an organization that is helping to advance robotics
 and STEM education for young people around the world.
 And now, here's my conversation with William McCaskill.
 What does utopia for humans and all life on Earth
 look like for you?
 That's a great question.
 What I want to say is that we don't know,
 and the utopia we want to get to is an indirect one
 that I call the long reflection.
 So a period of post-scarcity,
 no longer have the kind of urgent problems we have today,
 but instead can spend perhaps it's tens of thousands
 of years debating, engaging in ethical reflection
 in order before we take any kind of drastic lock-in actions,
 like spreading to the stars,
 and then we can figure out what is right,
 what is of kind of moral value.
 The long reflection, that's a really beautiful term.
 So if we look at Twitter for just a second,
 do you think human beings are able to reflect
 in a productive way?
 I don't mean to make it sound bad
 because there is a lot of fights and politics
 and division in our discourse.
 Maybe if you zoom out, it actually is civilized discourse.
 It might not feel like it, but when you zoom out.
 So I don't want to say that Twitter
 is not civilized discourse.
 I actually believe it's more civilized
 than people give it credit for.
 But do you think the long reflection can actually be stable
 where we as human beings with our descendants of a brains
 would be able to sort of rationally discuss things together
 and arrive at ideas?
 I think overall, we're pretty good at discussing things,
 rationally, and at least in the earlier stages of our mind,
 of our lives being open to many different ideas
 and being able to be convinced and change our views.
 I think that Twitter is designed almost
 to bring out all of the worst tendencies.
 So if the long reflection were conducted on Twitter,
 maybe it would be better just not even to bother.
 But I think the challenge really is getting to a stage
 where we have a society that is as conducive as possible
 to rational reflection, to deliberation.
 I think we're actually very lucky to be in a liberal society
 where people are able to discuss a lot of ideas and so on.
 I think when we look to the future, that's not at all
 guaranteed that society would be like that
 rather than a society where there's a fixed canon of values
 that are being imposed on all of society
 and where you aren't able to question that.
 That would be very bad from my perspective
 because it means we wouldn't be able to figure out
 what the truth is.
 I can already sense we're gonna go down
 a million tangents, but what do you think is the,
 if Twitter's not optimal, what kind of mechanism
 in this modern age of technology can we design
 where the exchange of ideas could be both civilized
 and productive and yet not be too constrained
 where there's rules of what you can say and can't say,
 which is, as you say, is not desirable,
 but yet not have some limits as to what can be said or not
 and so on.
 Do you have any ideas, thoughts on the possible future?
 Of course, nobody knows how to do it,
 but do you have thoughts of what a better
 Twitter might look like?
 I think that text-based media are intrinsically
 gonna be very hard to be conducive to rational discussion
 because if you think about it from an informational
 perspective, if I just send you a text of less than,
 what is it now, 240 characters, 280 characters, I think,
 that's a tiny amount of information compared to, say,
 you and I talking now where you have access to the words
 I say, which is the same as in text, but also my tone,
 also my body language, and we're very poorly designed
 to be able to assess.
 I have to read all of this context into anything you say,
 so I say maybe your partner sends you a text
 and as a full stop at the end, are they mad at you?
 You don't know, you have to infer everything
 about this person's mental state from whether they put
 a full stop at the end of a text or not.
 Well, the flip side of that, is it truly text
 that's the problem here because there's a viral aspect
 to the text where you could just post text nonstop,
 it's very immediate.
 The times before Twitter, before the internet,
 the way you would exchange text is you would write books,
 and that, while it doesn't get body language,
 it doesn't get tone, it doesn't, so on,
 but it does actually boil down after some time
 thinking some editing and so on, boil down ideas.
 So is the immediacy and the viral nature,
 which produces the outrage mobs and so on,
 the potential problem?
 I think that is a big issue.
 I think there's gonna be this strong selection effect
 where something that provokes outrage,
 well, that's high arousal, you're more likely
 to retweet that, where there's kind of sober analysis
 is not as sexy, not as viral.
 I do agree that long form content is much better
 to productive discussion.
 In terms of the media that are very popular at the moment,
 I think the podcasting is great where,
 like your podcasts are two hours long,
 so they're much more in depth than Twitter are.
 And you are able to convey so much more nuance,
 so much more caveat, because it's an actual conversation.
 It's more like the sort of communication
 that we've evolved to do, rather than kind of
 these very small little snippets of ideas
 that, when also combined with bad incentives,
 just clearly aren't designed
 for helping us get to the truth.
 It's kind of interesting.
 It's not just the length of the podcast medium,
 but it's the fact that it was started by people
 that don't give a damn about quote unquote demand.
 That there's a relaxed, sort of the style
 that Joe Rogan does, there's a freedom to express ideas
 in an unconstrained way that's very real.
 It's kind of funny that it feels so refreshingly real
 to us today, and I wonder what the future looks like.
 It's a little bit sad now that quite a lot of
 sort of more popular people are getting into podcasting.
 And they try to sort of create, they try to control it,
 they try to constrain it in different kinds of ways.
 People I love, like Conan O'Brien and so on,
 different comedians.
 And I'd love to see where the real aspects
 of this podcasting medium persists.
 Maybe in TV, maybe in YouTube, maybe Netflix
 is pushing those kind of ideas.
 And it's a really exciting word,
 that kind of sharing of knowledge.
 Yeah, I mean I think it's a double-edged sword
 as it becomes more popular and more profitable
 where on the one hand you'll get a lot more creativity,
 people doing more interesting things with the medium,
 but also perhaps you get this lace to the bottom
 where suddenly maybe it'll be hard to find good content
 on podcasts because it'll be so overwhelmed
 by the latest bit of viral outrage.
 So speaking of that, jumping on effective altruism
 for a second, so much of that internet content
 is funded by advertisements.
 Just in the context of effective altruism,
 we're talking about the richest companies in the world,
 they're funded by advertisements essentially.
 Google, that's their primary source of income.
 Do you see that as, do you have any criticism
 of that source of income?
 Do you see that source of money
 as a potentially powerful source of money
 that could be used, well certainly could be used for good,
 but is there something bad about that source of money?
 I think there's significant worries with it
 where it means that the incentives of the company
 might be quite misaligned with making people's lives better,
 where again, perhaps the incentives
 are towards increasing drama and debate
 on your social media feed in order that more people
 are going to be engaged, perhaps kind of compulsively
 involved with the platform.
 Whereas there are other business models
 like having an opt-in subscription service
 where perhaps they have other issues,
 but there's much more of a incentive to provide a product
 that its users are just really wanting,
 because now I'm paying for this product,
 I'm paying for this thing that I wanna buy,
 rather than I'm trying to use this thing
 and then trying to get a profit mechanism
 that is somewhat orthogonal to me actually just wanting
 to use the product.
 And so, I mean in some cases it'll work better than others.
 I can imagine, I can in theory imagine Facebook
 having a subscription service,
 but I think it's unlikely to happen anytime soon.
 Well it's interesting, it's weird now that you bring it up
 that it's unlikely.
 This example, I pay I think 10 bucks a month
 for YouTube Red, and that's,
 and I don't think I get it much for that,
 except just, so no ads, but in general
 it's just a slightly better experience.
 And I would gladly, now I'm not wealthy,
 in fact I'm operating very close to zero dollars,
 but I would pay 10 bucks a month to Facebook
 and 10 bucks a month to Twitter for some kind
 of more control in terms of advertisements
 and so on, but the other aspect of that is data,
 personal data, people are really sensitive about this.
 And I as one who hopes to one day create a company
 that may use people's data to do good for the world,
 wonder about this, one the psychology
 of why people are so paranoid, well I understand why,
 but they seem to be more paranoid than is justified at times.
 And the other is how do you do it right?
 So it seems that Facebook is,
 it seems that Facebook is doing it wrong.
 That's certainly the popular narrative.
 It's unclear to me actually how wrong,
 like I tend to give them more benefit of the doubt
 because they're, you know it's a really hard thing
 to do right, and people don't necessarily realize it,
 but how do we respect in your view people's privacy?
 Yeah, I mean in the case of how worried are people
 about using their data, I mean there's a lot
 of public debate and criticism about it.
 When we look at people's revealed preferences,
 you know people's continuing massive use
 of these sorts of services, it's not clear to me
 how much people really do care.
 Perhaps they care a bit, but they're happy to in effect
 kind of sell their data in order to be able
 to kind of use a certain service.
 That's a great term, revealed preferences.
 So these aren't preferences you self report in a survey,
 this is like your actions speak.
 Yeah, exactly, so you might say, oh yeah I hate the idea
 of Facebook having my data, but then when it comes to it,
 you actually are willing to give that data in exchange
 for being able to use the service.
 And if that's the case, then I think unless we have
 some explanation about why there's some negative externality
 from that or why there's some coordination failure,
 or if there's something that consumers are just really
 misled about where they don't realize why giving away data
 like this is a really bad thing to do,
 then ultimately I kind of want to respect people's
 preferences, they can give away their data if they want.
 I think there's a big difference between companies' use
 of data and governments having data where looking at the
 record of history, governments knowing a lot about
 their people can be very bad if the government chooses
 to do bad things with it.
 And that's more worrying I think.
 So let's jump into it a little bit.
 Most people know, but actually I two years ago had no idea
 what Effective Altruism was until I saw there was a cool
 looking event in an MIT group here, I think it's called
 the Effective Altruism Club or a group.
 I was like, what the heck is that?
 Yeah.
 And one of my friends said, he said that they're just
 a bunch of eccentric characters.
 So I was like, hell yes, I'm in.
 So I went to one of their events and looked up what's it
 about, it's quite a fascinating philosophical and just
 a movement of ideas.
 So can you tell me what is Effective Altruism?
 Great, so the core of Effective Altruism is about trying
 to answer this question, which is how can I do as much
 good as possible with my scarce resources, my time
 and with my money?
 And then once we have our best guess answers to that,
 trying to take those ideas and put that into practice
 and to do those things that we believe will do the most
 good and we're now a community of people, many thousands
 of us around the world who really are trying to answer
 that question as best we can and then use our time
 and money to make the world better.
 So what's the difference between sort of classical,
 general idea of altruism and Effective Altruism?
 So normally when people try to do good, they often just
 aren't so reflective about those attempts.
 So someone might approach you on the street asking you
 to give to charity and if you're feeling altruistic,
 you'll give to the person on the street.
 Or if you think, oh, I wanna do some good in my life,
 you might volunteer at a local place or perhaps you'll
 decide, you know, pursue a career where you're working
 in a field that's kind of more obviously beneficial
 like being a doctor or a nurse or a healthcare professional.
 But it's very rare that people apply the same level
 of rigor and analytical thinking to lots of other areas
 we think about.
 So take the case of someone approaching you on the street.
 Imagine if that person instead was saying,
 hey, I've got this amazing company, do you want
 to invest in it?
 It would be insane.
 No one would ever think, oh, of course, I'm just
 a company like you'd think it was a scam.
 But somehow we don't have that same level of rigor
 when it comes to doing good, even though the stakes
 are more important when it comes to trying to help others
 than trying to make money for ourselves.
 First of all, so there is a psychology at the individual
 level of doing good just feels good.
 And so in some sense, on that pure psychological part,
 it doesn't matter.
 In fact, you don't want to know if it does good or not
 because most of the time it won't.
 So like in a certain sense, it's understandable why
 altruism without the effective part is so appealing
 to a certain population.
 By the way, let's zoom out for a second.
 Do you think most people, two questions.
 Do you think most people are good?
 And question number two is, do you think most people
 want to do good?
 So are most people good?
 I think it's just super dependent on the circumstances
 that someone is in.
 I think that the actions people take and their moral
 worth is just much more dependent on circumstance
 than it is on someone's intrinsic character.
 So is it evil within all of us?
 It seems like the better angels of our nature,
 there's a tendency of us as a society to tend towards good,
 less war, I mean with all these metrics.
 What is that us becoming who we want to be?
 Or is that some kind of societal force?
 What's the nature versus nurture thing here?
 Yeah, so in that case, I just think, yeah, so violence
 has massively declined over time.
 I think that's a slow process of cultural evolution,
 institutional evolution, such that now the incentives for you
 and I to be violent are very, very small indeed.
 In contrast, when we were hunter-gatherers,
 the incentives were quite large.
 If there was someone who was potentially
 disturbing the social order and hunter-gatherer setting,
 there was a very strong incentive to kill that person.
 And people did.
 And it was just the guarded 10% of deaths among hunter-gatherers
 were murders.
 After hunter-gatherers, when you have actual societies,
 is when violence can probably go up,
 because there's more incentive to do mass violence,
 to take over or conquer other people's lands
 and murder everybody in place and so on.
 Yeah, I mean, I think total death rate from human causes
 does go down.
 But you're right that if you're in a hunter-gatherer
 situation, you're kind of a group that you're part of
 is very small, then you can't have massive wars,
 that just massive communities don't exist.
 But anyway, the second question, do you think
 most people want to do good?
 Yeah, and then I think that is true for most people.
 I think you see that with the fact that most people donate,
 a large proportion of people volunteer.
 If you give people opportunities to easily help other people,
 they will take it.
 But at the same time, we're a product of our circumstances.
 And if it were more socially rewarded
 to be doing more good, if it were more socially rewarded
 to do good effectively rather than not effectively,
 then we would see that behavior a lot more.
 So why should we do good?
 Yeah, my answer to this is there's
 no kind of deeper level of explanation.
 So my answer to kind of why should you do good is,
 well, there is someone whose life is on the line,
 for example, whose life you can save via donating just actually
 a few thousand dollars to an effective nonprofit,
 like the Against Malaria Foundation.
 That is a sufficient reason to do good.
 And then if you ask, well, why ought I to do that?
 I'm like, I just show you the same facts again.
 It's that fact that is the reason to do good.
 There's nothing more fundamental than that.
 I'd like to sort of make more concrete the thing we're
 trying to make better.
 So you just mentioned malaria.
 So there's a huge amount of suffering in the world.
 Are we trying to remove?
 So is ultimately the goal, not ultimately,
 but the first step is to remove the worst of the suffering?
 So there's some kind of threshold of suffering
 that we want to make sure does not exist in the world.
 Or do we really naturally want to take a much further step
 and look at things like income inequality?
 So not just getting everybody above a certain threshold,
 but making sure that there is some, that broadly speaking,
 there's less injustice in the world, unfairness,
 in some definition, of course, very difficult
 to define, and fairness.
 So the metric I use is how many people do we affect
 and by how much do we affect them?
 And so that can, often that means eliminating suffering.
 But it doesn't have to.
 Could be helping promote a flourishing life instead.
 And so if I was comparing reducing income inequality
 or getting people from the very pits of suffering
 to a higher level, the question I would ask
 is just a quantitative one of just,
 if I do this first thing or the second thing,
 how many people am I going to benefit?
 And by how much am I going to benefit?
 Am I going to move that one person from kind of 10%
 or 0% well-being to 10% well-being?
 Perhaps that's just not as good as moving 100 people
 from 10% well-being to 50% well-being.
 And the idea is the diminishing returns
 is the idea of when you're in terrible poverty,
 then the $1 that you give goes much further
 than if you were in the middle class in the United States,
 for example.
 Absolutely.
 And this fact is really striking.
 So if you take even just quite a conservative estimate
 of how we are able to turn money into well-being,
 the economists put it as like a log curve, or steeper.
 But that means that any proportional increase
 in your income has the same impact on your well-being.
 And so someone moving from $1,000 a year to $2,000 a year
 has the same impact as someone moving from $100,000 a year
 to $200,000 a year.
 And then when you combine that with the fact that we,
 in middle class members of rich countries,
 are 100 times richer in financial terms
 than the global poor, that means we
 can do 100 times to benefit the poorest people in the world
 as we can to benefit people of our income level.
 And that's this astonishing fact.
 Yeah, it's quite incredible.
 A lot of these facts and ideas are just
 difficult to think about.
 Because there's an overwhelming amount of suffering
 in the world.
 And even acknowledging it is difficult.
 I'm not exactly sure why that is.
 I mean, it's difficult because you have to bring to mind.
 It's an unpleasant experience thinking
 about other people's suffering.
 It's unpleasant to be empathizing with it, firstly.
 And then secondly, thinking about it
 means that maybe we'd have to change our lifestyles.
 And if you're very attached to the income that you've got,
 perhaps you don't want to be confronting ideas or arguments
 that might cause you to use some of that money to help others.
 So it's quite understandable in the psychological terms,
 even if it's not the right thing that we ought to be doing.
 So how can we do better?
 How can we be more effective?
 How does data help?
 In general, how can we do better?
 It's definitely hard.
 And we have spent the last 10 years
 engaged in some deep research projects to try and answer
 two questions.
 One is, of all the many problems the world is facing,
 what are the problems we ought to be focused on?
 And then within those problems that we
 judge to be the most pressing, where
 we use this idea of focusing on problems that
 are the biggest in scale, that are the most tractable,
 where we can make the most progress on that problem,
 and that are the most neglected.
 Within them, what are the things that have the best evidence
 or we have the best guess that will do the most good?
 And so we have a bunch of organizations.
 So GiveWell, for example, is focused
 on global health and development and has
 a list of seven top recommended charities.
 So the idea in general, and sorry to interrupt,
 is we'll talk about poverty and animal welfare
 and existential risk.
 Those are all fascinating topics.
 But in general, the idea is there should be a group.
 Sorry.
 There's a lot of groups that seek to convert money into good.
 And then you also, on top of that,
 want to have an accounting of how good they actually
 perform that conversion, how well they did in converting
 money to good.
 So ranking of these different groups,
 ranking these charities.
 So does that apply across basically all aspects
 of effective altruism?
 So there should be a group of people,
 and they should report on certain metrics
 of how well they've done.
 And you should only give your money
 to groups that do a good job.
 That's the core idea.
 I'd make two comments.
 One is just it's not just about money.
 So we're also trying to encourage people
 to work in areas where they'll have the biggest impact.
 Absolutely.
 And in some areas, they're really people heavy,
 but money poor.
 Other areas are kind of money rich and people poor.
 And so whether it's better to focus time or money
 depends on the cause area.
 And then the second is that you mentioned metrics.
 And while that's the ideal, and in some areas,
 we are able to get somewhat quantitative information
 about how much impact an area is having.
 That's not always true.
 For some of the issues, like you mentioned, existential risks,
 well, we're not able to measure in any sort of precise way
 how much progress we're making.
 And so you have to instead fall back on just rigorous argument
 and evaluation, even in the absence of data.
 So let's first sort of linger on your own story for a second.
 How do you yourself practice effective altruism
 in your own life?
 Because I think that's a really interesting place to start.
 So I've tried to build effective altruism into at least
 many components of my life.
 So on the donation side, my plan is
 to give away most of my income over the course of my life.
 I've set a bar I feel happy with,
 and I just donate above that bar.
 So at the moment, I donate about 20% of my income.
 Then on the career side, I've also
 shifted kind of what I do, where I was initially
 planning to work on very esoteric topics
 in the philosophy of logic, philosophy of language,
 things that are intellectually extremely interesting.
 But the path by which they really
 make a difference to the world is,
 let's just say it's very unclear at best.
 And so I switched instead to researching ethics
 to actually just working on this question of how we
 can do as much good as possible.
 And then I've also spent a very large chunk of my life
 over the last 10 years creating a number of nonprofits who,
 again, in different ways are tackling this question of how
 we can do the most good and helping
 them to grow over time too.
 Yeah, we'll mention a few of them with the career
 selection, 80,000.
 80,000 hours.
 80,000 hours is a really interesting group.
 So maybe also just a quick pause on the origins
 of effective altruism, because you paint a picture who
 the key figures are, including yourself
 in the effective altruism movement today.
 Yeah, there are two main strands that
 came together to form the effective altruism movement.
 So one was two philosophers, myself and Toby Ord at Oxford.
 And we had been very influenced by the work of Peter Singer,
 an Australian moral philosopher, who
 had argued for many decades that because one can do so much
 good at such little cost to oneself,
 we have an obligation to give away most of our income
 to benefit those in extreme poverty,
 just in the same way that we have an obligation to run in
 and save a child from drowning in a shallow pond
 if it were just to ruin your suit that cost a few thousand
 dollars.
 And we set up Giving What We Can in 2009,
 which is encouraging people to give at least 10% of their income
 to the most effective charities.
 And the second main strand was the formation
 of Give Well, which was originally based in New York
 and started in about 2007.
 And that was set up by Holden Karnavisi and Eli Hassenfeld,
 who were two hedge fund dudes who were making good money
 and thinking, well, where should I donate?
 And in the same way as if they wanted
 to buy a product for themselves, they
 would look at Amazon reviews.
 They were like, well, what are the best charities?
 Found they just weren't really good answers to that question,
 certainly not that they were satisfied with.
 And so they formed Give Well in order
 to try and work out what are those charities where they
 can have the biggest impact.
 And then from there and some other influences,
 community grew and spread.
 Can we explore the philosophical and political space
 that effective altruism occupies a little bit?
 So from the little and distant in my own lifetime
 that I've read of Ayn Rand's work, Ayn Rand's philosophy
 of objectivism, it espouses, and it's
 interesting to put her philosophy in contrast
 with effective altruism.
 So it espouses selfishness as the best thing you can do.
 But it's not actually against altruism.
 You have that choice, but you should be selfish in it, right?
 Or not.
 Maybe you can disagree here.
 So it can be viewed as the complete opposite
 of effective altruism, or it can be viewed as similar
 because the word effective is really interesting.
 Because if you want to do good, then you
 should be damn good at doing good, right?
 I think that would fit within the morality that's
 defined by objectivism.
 So do you see a connection between these two philosophies
 in other, perhaps, other, in this complicated space
 of beliefs that effective altruism is positioned
 as opposing or aligned with?
 I would definitely say that objectivism,
 Ayn Rand's philosophy, is a philosophy that's
 quite fundamentally opposed to effective altruism.
 In which way?
 Insofar as Ayn Rand's philosophy is about championing egoism
 and saying that I'm never quite sure
 whether the philosophy is meant to say that just you ought
 to do whatever will best benefit yourself,
 that's ethical egoism, no matter what the consequences are.
 Or second, if there's this alternative view, which is,
 well, you ought to try and benefit yourself
 because that's actually the best way of benefiting society.
 Certainly in Atlas Shluguchi is presenting her philosophy
 as a way that's actually going to bring
 about a flourishing society.
 And if it's the former, then well, effective altruism
 is all about promoting the idea of altruism
 and saying, in fact, we ought to really be trying to help others
 as much as possible.
 So it's opposed there.
 And then on the second side, I would just
 dispute the empirical premise.
 Given the major problems in the world today,
 it would seem like this remarkable coincidence.
 Quite suspicious, one might say, if benefiting myself
 was actually the best way to bring about a better world.
 So on that point, and I think that connects also
 with career selection that we'll talk about,
 but let's consider not object to this, but capitalism.
 And the idea that you focusing on the thing that you are damn
 good at, whatever that is, may be the best thing for the world.
 Part of it is also mindset.
 The thing I love is robots.
 So maybe I should focus on building robots
 and never even think about the idea of effective altruism, which
 is kind of the capitalist notion.
 Is there any value in that idea in just finding
 the thing you're good at and maximizing
 your productivity in this world, and thereby sort of lifting
 all boats and benefiting society as a result?
 Yeah, I think there's two things I'd want to say on that.
 So one is what your comparative advantages, what
 your strengths are when it comes to career,
 is obviously super important because there's
 lots of career paths I would be terrible at.
 If I thought being an artist was the best thing one could do,
 well, I'd be doomed.
 Just really quite astonishingly bad.
 And so I do think, at least within the realm of things
 that could plausibly be very high impact,
 choose the thing that you think you're
 going to be able to really be passionate at and excel at
 over the long term.
 Then on this question of should one just
 do that in an unrestricted way and not even think about what
 the most important problems are, I
 do think that in a perfectly designed society,
 that might well be the case.
 That would be a society where we've corrected all market
 failures, we've internalized all externalities,
 and then we've managed to set up incentives such that people
 just pursuing their own strengths
 is the best way of doing good.
 But we're very far from that society.
 So if one did that, then it would be very unlikely
 that you would focus on improving
 the lives of non-human animals that aren't participating
 in markets or ensuring the long run future goes well
 where future people certainly aren't participating
 in markets or benefiting the global poor who do participate
 but have so much less power from a starting perspective
 that their views aren't accurately represented
 by market forces too.
 Got it.
 So yeah, pure definition capitalism just
 may very well ignore the people that are suffering the most,
 the white swath of them.
 So if you could allow me this line of thinking here.
 So I've listened to a lot of your conversations online.
 I find, if I can compliment you, they're
 very interesting conversations.
 Your conversation on Rogan, on Joe Rogan
 was really interesting with Sam Harris and so on.
 Whatever.
 There's a lot of stuff that's really good out there.
 And yet, when I look at the internet,
 and I look at YouTube, which has certain mobs,
 certain swaths of right leaning folks whom I dearly love.
 I love all people, especially people with ideas.
 They seem to not like you very much.
 So I don't understand why exactly.
 So my own hypothesis is there is a right-left divide
 that, absurdly so, caricatured in politics,
 at least in the United States.
 And maybe you're somehow pigeonholed
 into one of those sides.
 And maybe that's what it is.
 Maybe your message is somehow politicized.
 Yeah.
 How do you make sense of that?
 Because you're extremely interesting.
 Like, you got the comments I see on Joe Rogan.
 There's a bunch of negative stuff.
 And yet, if you listen to it, the conversation is fascinating.
 I'm not speaking, I'm not some kind of lefty extremist.
 But just this fascinating conversation.
 So why are you getting some small amount of hate?
 So I'm actually pretty glad that effective altruism has
 managed to stay relatively unpoliticized.
 Because I think the core message, just use some of your time
 and money to do as much good as possible to fight
 some of the problems in the world,
 can be appealing across the political spectrum.
 And we do have a diversity of political viewpoints
 among people who have engaged in effective altruism.
 We do, however, do get some criticism
 from the left and the right.
 Oh, interesting.
 Both would be interesting to hear.
 Yeah.
 So criticism from the left is that we're not
 focused enough on dismantling the capitalist system
 that they see as the root of most of the problems
 that we're talking about.
 And there, I kind of disagree on partly the premise
 where I don't think relevant alternative systems would say
 to the animals or to the global poor
 or to the future generations much better.
 And then also the tactics where I
 think there are particular ways we can change society that
 would be massively beneficial on those things that
 don't go via dismantling the entire system, which
 is perhaps a million times harder to do.
 Then criticism on the right, there's definitely
 in response to the Joe Rogan podcast,
 there definitely were a number of A and Land fans
 who weren't keen on the idea of promoting altruism.
 There was a remarkable set of ideas, just the idea
 that effective altruism was unmanly, I think,
 was deriving a lot of criticism.
 Let me, OK, so I love fighting.
 I've been in street fights my whole life.
 As alpha in everything I do as it gets.
 And the fact that Joe Rogan said that I thought
 Scent of a Woman is a better movie than John Wick
 put me into this beta category amongst people who are like
 basically saying that it's yeah, unmanly or it's not tough.
 It's not there's not some principled view of strength
 that is represented by a spaz blizzard.
 So actually, so how do you think about this?
 Because to me, altruism, especially effective altruism,
 is I don't know what the female version of that is,
 but on the male side, manly as fuck, if I may say so.
 So how do you think about that kind of criticism?
 I think people who would make that criticism
 are just occupying a state of mind
 that I think is just so different from my state of mind
 that I kind of struggle to maybe even understand it,
 where if something's manly or unmanly or feminine
 or unfeminine, I'm like, I don't care.
 Like, is it the right thing to do or the wrong thing to do?
 So let me put it not in terms of man or woman.
 Yeah.
 I don't think that's useful.
 But I think there's a notion of acting out of fear.
 Yeah.
 That or as opposed to out of principle and strength.
 Yeah.
 So OK, yeah, here's something that I do feel as an intuition
 and that I think drives some people who
 do find kind of A and land attractive and so on
 as a philosophy, which is a kind of taking control
 of your own life and having power over how
 you're steering your life and not kind of kowtowing to others,
 you know, really thinking things through.
 I find like that set of ideas just very
 compelling and inspirational.
 But I actually think of the effect of altruism
 has really that side of my personality.
 It's like scratch that itch where you are just not
 taking the kind of priorities that society
 is giving you as granted.
 Instead, you're choosing to act in accordance
 with the priorities that you think
 are most important in the world.
 And often that involves then doing quite unusual things
 from a societal perspective, like donating
 a large chunk of your earnings or working
 on these weird issues about AI and so on
 that other people might not understand.
 Yeah, I think that's a really gutsy thing to do.
 That is taking control.
 That's at least at this stage, I mean,
 that's you taking ownership not of just yourself,
 but your presence in this world that's full of suffering
 and saying as opposed to being paralyzed by that notion.
 Yeah.
 Taking control and saying I could do something.
 Yeah.
 I mean, that's really powerful.
 But the one thing I personally hate too about the left
 currently that I think those folks to detect
 is the social signaling.
 When you look at yourself late at night,
 would you do everything you're doing
 in terms of effective altruism if your name,
 because you're quite popular, but if your name was totally
 unattached to it?
 Yeah.
 If it was in secret?
 Yeah, I mean, I think I would.
 To be honest, I think the kind of popularity is like,
 it's a mixed bag, but there are serious costs.
 And I don't particularly, I don't love it.
 It means you get all these people calling you
 a cuck on Joe Rogan.
 It's not the most fun thing.
 But you also get a lot of brownie points
 for doing good for the world.
 Yeah, you do.
 But I think my ideal life would be in some library solving
 logic puzzles all day.
 And I'd really be learning maths and so on.
 So you're in a good body of friends and so on.
 So your instinct for effective altruism is something deep.
 It's not one that is communicating socially.
 It's more in your heart you want to do good for the world.
 Yeah, I mean, so we can look back to early giving what we can.
 So we're setting this up, me and Toby.
 And I really thought that doing this
 would be a big hit to my academic career,
 because I was now spending, at that time, more than half
 my time setting up this nonprofit at the crucial time
 when you should be producing your best academic work
 and so on.
 And it was also the case at the time,
 it was kind of like the Toby Ord Club.
 He was the most popular.
 There was this personal interest story around him.
 His plan is to donate.
 Sorry to interrupt, but Toby was donating a large amount.
 Can you tell just briefly what he was doing?
 Yeah, so he made this public commitment
 to give everything he earned above 20,000 pounds per year
 to the most effective causes.
 And even as a graduate student, he
 was still donating about 15%, 20% of his income,
 which is so quite significant, given
 that graduate students are not known for being super wealthy.
 And when we launched Giving What We Can,
 the media just loved this as a personal interest story.
 So the story about him and his pledge was the most,
 yeah, it was actually the most popular news story of the day.
 And we ran the same story a year later,
 and it was the most popular news story of the day a year later,
 too.
 And so it really was several years before then
 I was also giving more talks and starting
 to do more writing.
 And then especially with this book, Doing Good Better,
 then there started to be attention and so on.
 But deep inside, your own relationship
 with effective altruism was, I mean,
 it had nothing to do with the publicity.
 Did you see yourself, how did the publicity connect with it?
 Yeah, I mean, that's kind of what I'm saying,
 is I think the publicity came several years afterwards.
 I mean, at the early stage when we set up Giving What We Can,
 it was really just every person we get to pledge 10%
 is something like $100,000 over their lifetime.
 That's huge.
 And so it was just we had started with 23 members.
 Every single person was just this kind of huge accomplishment.
 And at the time, I just really thought, maybe over time
 we'll have 100 members, and that'll be amazing.
 Whereas now we have over $4,000 and $1.5 billion pledged.
 That's just unimaginable to me at the time
 when I was first kind of getting this stuff off the ground.
 So can we talk about poverty and the biggest problems
 that you think in the near term effective altruism can attack
 in each one?
 So poverty obviously is a huge one.
 Yeah.
 How can we help?
 Great, yeah.
 So poverty, absolutely this huge problem.
 700 million people in extreme poverty
 living in less than $2 per day.
 What that means is what $2 would buy in the US.
 So think about that.
 It's like some rice, maybe some beans.
 It's really not much.
 And at the same time, we can do an enormous amount
 to improve the lives of people in extreme poverty.
 So the things that we tend to focus on
 are interventions in global health.
 And that's for a few reasons.
 One is that global health just has this amazing track record.
 Life expectancy globally is up 50% relative to 60 or 70 years
 ago.
 We've eradicated smallpox, which killed 2 million lives
 every year, almost eradicated polio.
 Second is that we just have great data on what works
 when it comes to global health.
 So we just know that bed nets protect children and prevent
 them from dying from malaria.
 And then the third is just that it's extremely cost effective.
 So it costs $5 to buy one bed net,
 protects two children for two years against malaria.
 If you spend about $3,000 on bed nets,
 then statistically speaking, you're
 going to save a child's life.
 And there are other interventions too.
 And so given the people in such suffering,
 and we have this opportunity to do such huge good
 for such low cost, well, yeah, why not?
 So the individuals, so for me today,
 if I wanted to look at poverty, how would I help?
 And I wanted to say, I think donating 10% of your income
 is a very interesting idea, or some percentage or some.
 Setting a bar and sort of sticking to it,
 how do we then take the step towards the effective part?
 So you've conveyed some notions, but who
 do you give the money to?
 Yeah, so GiveWell, this organization I mentioned,
 well, it makes charity recommendations.
 And some of its top recommendations,
 so Against Malaria Foundation is this organization
 that buys and distributes these insecticide-deleted bed nets.
 And then it has a total of seven charities
 that it recommends very highly.
 So that recommendation, is it almost like a star of approval?
 Or is there some metrics?
 So what are the ways that GiveWell
 conveys that this is a great charity organization?
 Yeah, so GiveWell is looking at metrics.
 And it's trying to compare charities ultimately
 in the number of lives that you can save
 or an equivalent benefit.
 So one of the charities it recommends
 is GiveDirectly, which simply just transfers cash
 to the poorest families, where a poor family will get a cash
 transfer of $1,000.
 And they kind of regard that as the baseline intervention,
 because it's so simple and people,
 they know what to do with, how to benefit themselves.
 That's quite powerful, by the way.
 So before GiveWell, before the Effective Altruism Movement,
 was there, I imagine, there's a huge amount of corruption,
 funny enough, in charity organizations,
 or misuse of money.
 So there was nothing like GiveWell before that?
 No, I mean, there were some.
 So I mean, the charity corruption,
 I mean, obviously there's some.
 I don't think it's a huge issue,
 relative to just focusing on the wrong things.
 Prior to GiveWell, there were some organizations
 like Charity Navigator, which were more aimed at worrying
 about corruption and so on.
 So they weren't saying, these are the charities where
 you're going to do the most good.
 Instead, it was like, how good are the charities' financials?
 How good is this?
 How far are they transparent?
 And yeah, so that would be more useful for weeding out
 some of those worst charities.
 So GiveWell, just taking it a step further,
 sort of in this 21st century of data,
 it's actually looking at the effective part.
 Yeah, so it's like, if you know the wire cutter,
 if you want to buy a pair of headphones,
 they will just look at all the headphones and be like,
 these are the best headphones you can buy.
 And that's the idea with GiveWell.
 Okay, so do you think there's a bar of what suffering is
 and do you think one day we can eradicate suffering
 in our world?
 Yeah, exactly.
 Amongst humans?
 Let's talk humans for now.
 Talk humans, but in general, yeah, actually.
 So there's a colleague of mine calling the term abolitionism
 for the idea that we should just be trying
 to abolish suffering.
 And in the long run, I mean, I don't expect it anytime soon,
 but I think we can.
 I think that would require quite fantastic changes
 to the way society is structured and perhaps even the,
 in fact, even changes to human nature.
 But I do think that suffering,
 whenever it occurs is bad
 and we should want it to not occur.
 So there's a line, there's a gray area between suffering.
 Now I'm Russian, so I romanticize some aspects of suffering.
 There's a gray line between struggle, gray area
 between struggle and suffering.
 So one, do we want to eradicate all struggle in the world?
 So there's an idea that the human condition
 inherently has suffering in it.
 And it's a creative force.
 It's a struggle of our lives and we somehow grow from that.
 How do you think about that?
 I agree that's true.
 So, often, great artists can be also suffering
 from major health conditions or depression and so on.
 Or they come from abusive parents.
 Most great artists, they think come from abusive parents.
 Yeah, that seems to be at least commonly the case.
 But I want to distinguish between suffering
 as being instrumentally good,
 it causes people to produce good things
 and whether it's intrinsically good.
 And I think intrinsically, it's always bad.
 And so if we can produce these great achievements
 via some other means where,
 if we look at the scientific enterprise,
 we've produced incredible things,
 often from people who aren't suffering,
 who have pretty good lives.
 They're driven instead of being pushed
 by a certain sort of anguish,
 they're being driven by intellectual curiosity.
 If we can instead produce a society
 where it's all carrot and no stick,
 that's better from my perspective.
 Yeah, but I'm gonna have to disagree
 with the notion that that's possible.
 But I would say most of the suffering
 in the world is not productive.
 So I would dream of effective altruism
 curing that suffering.
 But then I would say that there is some suffering
 that is productive that we wanna keep.
 But that's not even the focus of,
 because most of the suffering is just absurd
 that needs to be eliminated.
 So let's not even romanticize this usual notion I have,
 but nevertheless, struggle has some kind of inherent value
 that to me at least,
 you're right, there's some elements of human nature
 that also have to be modified in order to cure all suffering.
 Yeah, I mean, there's an interesting question
 of whether it's possible.
 So at the moment, most of the time we're kind of neutral,
 and then we burn ourselves and that's negative.
 And that's really good that we get that negative signal
 because it means we won't burn ourselves again.
 There's a question like, could you design agents, humans,
 such that you're not hovering around the zero level,
 you're hovering at like bliss,
 and then you touch the flame and you're like,
 oh no, you're just slightly worse bliss.
 But that's really bad compared to the bliss
 you were normally in.
 So you can have like a gradient of bliss
 instead of like pain and pleasure.
 On that point, I think it's a really important point
 on the experience of suffering,
 the relative nature of it.
 Having grown up in the Soviet Union,
 we're quite poor by any measure
 and when I was in my childhood,
 but it didn't feel like you were poor
 because everybody around you were poor.
 And then in America, I feel,
 I for the first time begin to feel poor.
 Yeah, yeah.
 Because of the, there's different,
 there's some cultural aspects to it
 that really emphasize that it's good to be rich.
 And then there's just the notion
 that there is a lot of income inequality
 and therefore you experience that inequality.
 That's where suffering comes.
 So what do you think about the inequality of suffering
 that we have to think about?
 Do you think we have to think about that
 as part of effective altruism?
 Yeah, I think we're just, things vary
 in terms of whether you get benefits or costs from them
 just in relative terms or in absolute terms.
 So a lot of the time, yeah,
 there's this hedonic treadmill where if you get,
 you know, there's, money is useful
 because it helps you buy things
 or good for you because it helps you buy things.
 But there's also a status component too.
 And that status component is kind of zero sum.
 If you were saying like in Russia,
 you know, no one else felt poor
 because everyone around you was poor.
 Whereas now you've got this,
 these other people who are, you know, super rich
 and maybe that makes you feel, you know,
 less good about yourself.
 There are some other things however,
 which are just intrinsically good or bad.
 So commuting, for example, it's just people hate it.
 It doesn't really change.
 Knowing that other people are commuting too,
 doesn't make it any kind of less bad.
 But sort of to push back on that for a second, I mean, yes,
 but also if some people were, you know, on horseback,
 your commute on the train might feel a lot better.
 Yeah.
 You know, there is a relative,
 I mean, everybody's complaining about society today
 forgetting it's forgetting how much better it is,
 the better it is in the ages of our nature,
 how the technology is improved,
 fundamentally improving most of the world's lives.
 Yeah, and actually there's some psychological research
 on the wellbeing benefits of volunteering,
 where people who volunteer
 tend to just feel happier about their lives.
 And one of the suggested explanations is it because
 it extends your reference class.
 So no longer are you comparing yourself to the Joneses
 who have their slightly better car,
 but you realize that, you know,
 people in much worse conditions than you.
 And so now, you know, your life doesn't seem so bad.
 That's actually on a psychological level,
 one of the fundamental benefits of effective altruism.
 Yeah.
 Is, I mean, I guess it's the altruism part
 of effective altruism is exposing yourself
 to the suffering in the world allows you to be more,
 yeah, happier, and actually allows you
 in a sort of meditative introspective way
 realize that you don't need most of the wealth you have
 to be happy.
 Absolutely, I mean, I think effective altruism
 has been this huge benefit for me.
 And I really don't think that if I had more money
 that I was living on, that that would change
 my level of wellbeing at all.
 Whereas engaging in something that I think is meaningful,
 that I think is steering humanity in a positive direction,
 that's extremely rewarding.
 And so, yeah, I mean, despite my best attempts at sacrifice,
 I don't, you know, I think I've actually ended up happier
 as a result of engaging in effective altruism
 than I would have done.
 That's such an interesting idea.
 So let's talk about animal welfare.
 Sure.
 Easy question, what is consciousness?
 Especially as it has to do with the capacity to suffer.
 I think there seems to be a connection
 between how conscious something is,
 the amount of consciousness and its ability to suffer.
 And that all comes into play about us thinking
 how much suffering there is in the world
 with regard to animals.
 So how do you think about animal welfare and consciousness?
 Okay, well, consciousness, easy question.
 Yeah, I mean, I think we don't have a good understanding
 of consciousness.
 My best guess is it's got, and by consciousness,
 I'm meaning what it feels like to be you,
 the subjective experience that seems to be different
 from everything else we know about in the world.
 Yeah, I think it's clear it's very poorly understood
 at the moment.
 I think it has something to do with information processing.
 So the fact that the brain is a computer
 or something like a computer.
 So that would mean that very advanced AI could be conscious,
 information processors in general could be conscious
 with some suitable complexity.
 But that also, some suitable complexity,
 it's a question whether greater complexity
 creates some kind of greater consciousness,
 which relates to animals.
 Yeah.
 Right, is there, if it's an information processing system
 and it's smaller and smaller, is an ant less conscious
 than a cow, less conscious than a monkey?
 Yeah, and again, this super hard question,
 but I think my best guess is yes.
 Like if I think, well, consciousness,
 it's not some magical thing that appears out of nowhere.
 It's not, Descartes thought it was just,
 comes in from this other realm
 and then enters through the pineal gland in your brain
 and that's kind of soul in its conscious.
 So it's got something to do
 with what's going on in your brain.
 A chicken has one 300th of the size of the brain
 that you have.
 Ants, I don't know how small it is.
 Maybe it's a millionth the size.
 My best guess, which I may well be wrong about
 because this is so hard, is that in some relevant sense,
 the chicken is experiencing consciousness
 to a less degree than the human
 and the ant significantly less again.
 I don't think it's as little as 300th as much, I think.
 There's evolutionary reasons for thinking
 that the ability to feel pain comes on the scene
 relatively early on and we have lots of our brain
 that's dedicated to stuff that doesn't seem to have to do
 anything to do with consciousness,
 language processing and so on.
 So it seems like the easy,
 so there's a lot of complicated questions there
 that we can't ask the animals about,
 but it seems that there's easy questions
 in terms of suffering, which is things like factory farming
 that could be addressed.
 Is that the lowest hanging fruit,
 if I may use crude terms here, of animal welfare?
 Absolutely, I think that's the lowest hanging fruit.
 So at the moment we kill, we raise and kill
 about 50 billion animals every year.
 So how many?
 50 billion.
 Yeah, so for every human on the planet,
 several times that number are being killed.
 And the vast majority of them are raised in factory farms
 where basically whatever your view on animals,
 I think you should agree, even if you think,
 well, maybe it's not bad to kill an animal,
 maybe if the animal was raised in good conditions,
 that's just not the empirical reality.
 The empirical reality is that they are kept
 in incredible cage confinement,
 they are de-beaked or detailed without anesthetic,
 chickens often peck each other to death,
 like otherwise, because of such stress.
 It's really, I think when a chicken gets killed,
 that's the best thing that happened to the chicken
 in the course of its life.
 And it's also completely unnecessary.
 This is in order to save a few pence
 for the price of meat or price of eggs.
 And we have indeed found,
 it's also just inconsistent with consumer preference
 as well.
 People who buy the products, if they could,
 they, when you do surveys,
 are extremely against suffering in factory farms.
 It's just that they don't appreciate how bad it is
 and just tend to go with the easy options.
 And so then the best, the most effective programs
 I know of at the moment are nonprofits that go to companies
 and work with companies to get them to take a pledge
 to cut certain sorts of animal products
 like eggs from cage confinement out of their supply chain.
 And it's now the case that the top 50 food retailers
 and fast food companies have all made these
 kind of cage-free pledges.
 And when you do the numbers, you get the conclusion
 that every dollar you're giving to these nonprofits
 result in hundreds of chickens being spared
 from cage confinement.
 And then they're working to other types of animals,
 other products too.
 So is that the most effective way to do,
 have a ripple effect essentially?
 As opposed to directly having regulation
 from on top that says you can't do this?
 So I would be more open to the regulation approach,
 but at least in the US,
 there's quite intense regulatory capture
 from the agricultural industry.
 And so attempts that we've seen to try and change regulation
 have, it's been a real uphill struggle.
 There are some examples of ballot initiatives
 where the people have been able to vote in a ballot
 to say we wanna ban eggs from cage conditions.
 And that's been huge, that's been really good.
 But beyond that, it's much more limited.
 So I've been really interested in the idea of hunting
 in general and wild animals and seeing nature
 as a form of cruelty
 that I am ethically more okay with.
 Just from my perspective.
 And then I read about wild animal suffering.
 I'm just giving you the kind of notion of how I felt
 because animal factory farming is so bad
 that living in the woods seemed good.
 And yet when you actually start to think about it,
 all of the animals in the animal world
 are living in terrible poverty, right?
 So you have all the medical conditions, all of that.
 I mean, they're living horrible lives,
 it could be improved.
 That's a really interesting notion
 that I think may not even be useful to talk about
 because factory farming is such a big thing to focus on.
 But it's nevertheless an interesting notion
 to think of all the animals in the wild as suffering
 in the same way that humans and poverty are suffering.
 Yeah, I mean, and often even worse.
 So many animals we produce by our selection.
 So you have a very large number of children
 in the expectation that only a small number survive.
 And so for those animals, almost all of them
 just live short lives where they starve to death.
 So yeah, there's huge amounts of suffering in nature
 that I don't think we should pretend
 that it's this kind of wonderful paradise for most animals.
 Yeah, their life is filled with hunger and fear and disease.
 I agree with you entirely that when it comes
 to focusing on animal welfare,
 we should focus on factory farming.
 But we also, yeah, should be aware to the reality
 of what life for most animals is like.
 So let's talk about a topic I've talked a lot about
 and you've actually quite eloquently talked about,
 which is the third priority that effective altruism
 considers as really important is existential risks.
 When you think about the existential risks
 that are facing our civilization, what's before us?
 What concerns you?
 What should we be thinking about from the,
 especially from an effective altruism perspective?
 Great, so the reason I started getting concerned
 about this was thinking about future generations
 where the key idea is just,
 well, future people matter morally.
 There are vast numbers of future people.
 If we don't cause our own extinction,
 there's no reason why civilization
 might not last a million years.
 I mean, we last as long as a typical mammalian species.
 Or a billion years is when the earth is no longer habitable.
 Or if we can take to the stars,
 then perhaps it's trillions of years beyond that.
 So the future could be very big indeed.
 And it seems like we're potentially very early on
 in civilization.
 Then the second idea is just,
 well, maybe there are things
 that are gonna really derail that.
 Things that actually could prevent us
 from having this long, wonderful civilization.
 And instead, could cause our own extinction
 or otherwise perhaps like lock ourselves
 into a very bad state.
 And what ways could that happen?
 Well, causing our own extinction,
 the development of nuclear weapons in the 20th century,
 at least put on the table that we now had weapons
 that were powerful enough
 that you could very significantly destroy society.
 Perhaps an all-black nuclear war
 would cause a nuclear winter.
 Perhaps that would be enough
 for the human race to go extinct.
 When you think we haven't done it,
 sorry to interrupt,
 why do you think we haven't done it yet?
 Is it surprising to you
 that having always for the past few decades,
 several thousand of active,
 ready to launch nuclear weapons, warheads,
 and yet we have not launched them
 ever since the initial launch on Hiroshima and Nagasaki?
 I think it's a mix of luck.
 So I think it's definitely not inevitable
 that we haven't used them.
 So John F. Kennedy, general Cuban missile crisis,
 put the estimate of nuclear exchange
 between the US and USSR
 at somewhere between one and three and even.
 So we really did come close.
 At the same time,
 I do think mutually assured destruction
 is a reason why people don't go to war.
 It would be why nuclear powers don't go to war.
 Do you think that holds,
 if you can linger on that for a second,
 like my dad is a physicist amongst other things,
 and he believes that nuclear weapons
 are actually just really hard to build,
 which is one of the really big benefits of them currently.
 So that you don't have,
 it's very hard if you're crazy to acquire a nuclear weapon.
 So the mutually assured destruction really works
 when you talk, seems to work better when it's nation states,
 when it's serious people, even if they're a little bit,
 you know, dictatorial and so on.
 Do you think this mutually assured destruction idea
 will carry, how far will it carry us
 in terms of different kinds of weapons?
 Oh yeah, I think it's your point that nuclear weapons
 are very hard to build and relatively easy to control
 because you can control fissile material,
 is a really important one.
 And future technology that's equally destructive
 might not have those properties.
 So for example, if in the future,
 people are able to design viruses,
 perhaps using a DNA printing kit that's on,
 that, you know, one can just buy.
 In fact, there are companies in the process
 of creating home DNA printing kits.
 Well then perhaps that's just totally democratized,
 perhaps the power to reap huge destructive potential
 is in the hands of most people in the world,
 or certainly most people with effort.
 And then, yeah, I no longer trust mutually assured
 destruction because for some people,
 the idea that they would die is just not a disincentive.
 There was a Japanese cult, for example,
 Omshin Rikkyo in the 90s that had,
 but they what they believed was that Armageddon was coming.
 If you died before Armageddon, you would get good karma,
 you wouldn't go to hell.
 If you died during Armageddon, maybe you would go to hell.
 And they had a biological weapons program,
 a chemical weapons program.
 When they were finally apprehended,
 they hadn't stocks of saline gas
 that were sufficient to kill four million people,
 engaged in multiple terrorist acts.
 If they had had the ability to print a virus at home,
 that would have been very scary.
 So it's not impossible to imagine groups of people
 that hold that kind of belief of death and suicide
 as a good thing for passage into the next world and so on.
 And then connect them with some weapons.
 Then ideology and weaponry may create serious problems for us.
 Let me ask you a quick question on
 what do you think is the line between killing most humans
 and killing all humans?
 How hard is it to kill everybody?
 Have you thought about this?
 I've thought about it a bit.
 I think it is very hard to kill everybody.
 So in the case of, let's say, an all-out nuclear exchange,
 and let's say that leads to nuclear winter,
 we don't really know, but it might well happen.
 That would, I think, result in billions of deaths.
 Would it kill everybody?
 It's quite hard to see how it would kill everybody
 for a few reasons.
 One is just there's just so many people.
 7.5 billion people.
 So this bad event has to kill almost all of them.
 Secondly, live in such a diversity of locations.
 So a nuclear exchange or the virus,
 it has to kill people who live on the coast of New Zealand,
 which is gonna be climatically much more stable
 than other areas in the world.
 Or people who are on submarines.
 Or who have access to bunkers.
 So there's a very...
 I'm sure there's two guys in Siberia just badass.
 There's just human nature somehow just perseveres.
 Yeah, and then the second thing is just
 if there's some catastrophic event,
 people really don't want to die.
 So there's gonna be huge amounts of effort
 to ensure that it doesn't affect everyone.
 If you thought about what it takes to rebuild a society
 with smaller and smaller numbers,
 like how big of a setback these kinds of things are.
 Yeah, so then that's something
 where there's real uncertainty, I think.
 Where at some point you just lose
 sufficient genetic diversity such that you can't come back.
 It's unclear how small that population is,
 but if you've only got say a thousand people
 or fewer than a thousand, then maybe that's small enough.
 What about human knowledge?
 And then there's human knowledge.
 I mean, it's striking how short on geological timescales
 or evolutionary timescales the progress in
 or how quickly the progress in human knowledge has been.
 Agriculture we only invented in 10,000 BC.
 Cities were only 3000 BC.
 Whereas typical mammal species is half a million years
 to a million years.
 Do you think it's inevitable in some sense?
 Agriculture, everything that came,
 the Industrial Revolution, cars, planes, the internet,
 that level of innovation you think is inevitable?
 I think given how quickly it arose,
 so in the case of agriculture,
 I think that was dependent on climate.
 It was the kind of glacial period was over,
 the earth warmed up a bit.
 That made it much more likely
 that humans would develop agriculture.
 When it comes to the Industrial Revolution,
 it's just, again, it only took a few thousand years
 from cities to Industrial Revolution.
 If we think, okay, we've gone back to this,
 even let's say agricultural era,
 but there's no reason why we wouldn't go extinct
 in the coming tens of thousands of years
 or hundreds of thousands of years,
 it would be very surprising if we didn't rebound,
 unless there's some special reason
 that makes things different.
 So perhaps we just have a much greater disease burden now.
 So HIV exists, it didn't exist before.
 And perhaps that's latent in being suppressed
 by modern medicine and sanitation and so on,
 but would be a much bigger problem for some
 utterly destroyed society
 that was trying to rebound.
 Or there's just maybe there's something we don't know about.
 So another existential risk comes from the mysterious,
 the beautiful artificial intelligence.
 Yeah.
 So what's the shape of your concerns about AI?
 I think there are quite a lot of concerns about AI,
 and sometimes the different risks
 don't get distinguished enough.
 So the kind of classic worry,
 most is closely associated with Nick Bostrom
 and Elias Joukowsky is that we at some point
 move from having narrow AI systems
 to artificial general intelligence.
 You get this very fast feedback effect
 where AI is able to build,
 artificial intelligence helps you to build
 greater artificial intelligence.
 We have this one system that's suddenly very powerful,
 far more powerful than others
 than perhaps far more powerful than
 the rest of the world combined.
 And then secondly, it has goals
 that are misaligned with human goals.
 And so it pursues its own goals.
 It realized, hey, there's this competition,
 namely from humans, it would be better
 if we eliminated them.
 In just the same way as Homo sapiens
 eradicated the Neanderthals,
 in fact, it in fact killed off most large animals
 on the planet that walked the planet.
 So that's kind of one set of worries.
 I think that's not my,
 I think these shouldn't be dismissed as science fiction.
 I think it's something we should be taking very seriously.
 But it's not the thing you visualize
 when you're concerned about the biggest near term.
 Yeah, I think it's like one possible scenario
 that would be astronomically bad.
 I think there are other scenarios
 that would also be extremely bad, comparably bad,
 that are more likely to occur.
 So one is just we are able to control AI.
 So we're able to get it to do what we want it to do.
 And perhaps there's not like this fast takeoff
 of AI capabilities within a single system.
 It's distributed across many systems
 that do somewhat different things.
 But you do get very rapid economic
 and technological progress as a result.
 That concentrates power into the hands
 of a very small number of individuals,
 perhaps a single dictator.
 And secondly, that single individual is,
 or small group of individuals or single country,
 is then able to like lock in their values indefinitely
 via transmitting those values to artificial systems
 that have no reason to die.
 Like, their code is copyable.
 Perhaps Donald Trump or Xi Jinping
 creates their kind of AI progeny in their own image.
 And once you have a system that's,
 once you have a society that's controlled by AI,
 you no longer have one of the main drivers
 of change historically, which is the fact
 that human lifespans are only a hundred years,
 give or take.
 So that's really interesting.
 So as opposed to sort of killing off all humans,
 is locking in, like creating a hell on earth,
 basically a set of principles under which
 the society operates that's extremely undesirable.
 So everybody is suffering indefinitely.
 Or it doesn't, I mean, it also doesn't need
 to be hell on earth.
 It could just be the wrong values.
 So we talked at the very beginning
 about how I wanna see this kind of diversity
 of different values and exploration
 so that we can just work out what is kind of morally right,
 what is good, what is bad,
 and then pursue the thing that's best.
 So actually, so the idea of wrong values
 is actually probably, the beautiful thing is
 there's no such thing as right and wrong values
 because we don't know the right answer.
 We just kind of have a sense of which value is more right
 and which is more wrong.
 So any kind of lock-in makes a value wrong
 because it prevents exploration of this kind.
 Yeah, and just imagine if fascist,
 imagine if there was Hitler's utopia or Stalin's utopia
 or Donald Trump's or Xi Jinping's forever.
 Yeah.
 You know, how good or bad would that be
 compared to the best possible future we could create?
 And my suggestion is it would really suck
 compared to the best possible future we could create.
 And you're just one individual.
 There's some individuals for whom Donald Trump
 is perhaps the best possible future.
 And so that's the whole point of us individuals
 exploring this space together.
 Exactly, yeah.
 And what's trying to figure out,
 which is the path that will make America great again?
 Yeah, exactly.
 So how can effective altruism help?
 I mean, this is a really interesting notion
 they actually describe being of artificial intelligence
 being used as extremely powerful technology
 in the hands of very few, potentially one person,
 to create some very undesirable effect.
 So as opposed to AI, and again,
 the source of the undesirableness there is the human.
 AI is just a really powerful tool.
 So whether it's that or whether AI's AGI
 just runs away from us completely,
 how as individuals, as people in the effective
 altruism movement, how can we think about
 something like this?
 I understand poverty and welfare,
 but this is a far out incredibly mysterious
 and difficult problem.
 Great, well, I think there's three paths as an individual.
 So if you're thinking about career paths you can pursue,
 so one is going down the line of technical AI safety.
 So this is most relevant to the kind of AI winning,
 AI taking over scenarios where this is just technical work
 on current machine learning systems,
 often sometimes going more theoretical too,
 on how we can ensure that an AI is able to learn
 human values and able to act in the way
 that you want it to act.
 And that's a pretty mainstream issue and approach
 in machine learning today.
 So we definitely need more people doing that.
 Second is on the policy side of things,
 which I think is even more important at the moment,
 which is how should developments in AI be managed
 on a political level?
 How can you ensure that the benefits of AI
 are very distributed, power isn't being concentrated
 in the hands of a small set of individuals?
 How do you ensure that there aren't arms races
 between different AI companies that might result in them
 cutting corners with respect to safety?
 And so there the input as individuals who can have
 is this, we're not talking about money,
 we're talking about effort.
 We're talking about career choices.
 Yeah, we're talking about career choice, yeah.
 But then it is the case that supposing,
 you're like, I've already decided my career
 and I'm doing something quite different,
 you can contribute with money too,
 where at the Center for Effective Altruism,
 we set up the Long-Term Future Fund.
 So if you go onto effectivealtruism.org,
 you can donate where a group of individuals
 will then work out what's the highest value place
 they can donate to work on existential risk issues
 with a particular focus on AI.
 What's path number three?
 This was path number three.
 This was donations were the third option
 I was thinking of.
 And then yeah, you can also donate directly
 to organizations working on this
 like Center for Human Compatible AI at Berkeley,
 Future of Humanity Institute at Oxford,
 or other organizations too.
 Does AI keep you up at night, this kind of concern?
 Yeah, it's kind of a mix where I think it's very likely
 things are gonna go well.
 I think we're gonna be able to solve these problems.
 I think that's by far the most likely outcome,
 at least over the next. By far the most likely.
 So if you look at all the trajectories running away
 from our current moment in the next 100 years,
 you see AI creating destructive consequences
 as a small subset of those possible trajectories.
 Or at least yeah, kind of eternal destructive consequences.
 I think that being a small subset.
 At the same time, it still freaks me out.
 I mean, when we're talking about the entire future
 of civilization, then small probabilities,
 1% probability, that's terrifying.
 What do you think about Elon Musk's strong worry
 that we should be really concerned about existential risks
 of AI?
 Yeah, I mean, I think, broadly speaking, I think he's right.
 I think if we talked, we would probably have
 very different probabilities on how likely it is
 that we're doomed.
 But again, when it comes to talking about
 the entire future of civilization,
 it doesn't really matter if it's 1% or if it's 50%.
 We ought to be taking every possible safeguard we can
 to ensure that things go well rather than poorly.
 Last question, if you yourself could eradicate
 one problem from the world, what would that problem be?
 That's a great question.
 I don't know if I'm cheating in saying this,
 but I think the thing I would most want to change
 is just the fact that people don't actually care
 about ensuring the long run future goes well.
 People don't really care about future generations.
 They don't think about it.
 It's not part of their aims.
 Well, in some sense, you're not cheating at all,
 because in speaking the way you do,
 in writing the things you're writing,
 you're addressing exactly this aspect.
 Exactly.
 That is your input into the effect of altruism movement.
 So for that, Will, thank you so much.
 It's an honor to talk to you.
 I really enjoyed it.
 Thanks so much for having me on.
 Thanks for listening to this conversation
 with William McCaskill.
 Thank you to our presenting sponsor, Cash App.
 Please consider supporting the podcast
 by downloading Cash App and using code LexPodcast.
 If you enjoy this podcast, subscribe on YouTube,
 review it with five stars on Apple Podcasts,
 support it on Patreon,
 or simply connect with me on Twitter at Lex Friedman.
 And now let me leave you with some words
 from William McCaskill.
 One additional unit of income can do 100 times as much
 to benefit the extreme poor as it can to benefit you
 or I, earning the typical US wage of $28,000 a year.
 It's not often that you have two options,
 one of which is 100 times better than the other.
 Imagine a happy hour where you can either
 buy yourself a beer for $5
 or buy someone else a beer for 5 cents.
 If that were the case, we'd probably be pretty generous.
 Next round's on me.
 But that's effectively the situation we're in all the time.
 It's like a 99% off sale or buy one, get 99 free.
 Might be the most amazing deal you'll see in your life.
 Thank you for listening and hope to see you next time.
VoiceoverIf this video could gateetztthat it, thank you so much.