Pamela McCorduck: Machines Who Think and the Early Days of AI | Lex Fridman Podcast #34
 The following is a conversation with Pamela McCordick. She's an author who has written on the history and the philosophical significance of artificial intelligence. Her books include Machines Who Think in 1979, The Fifth Generation in 1983 with Ed Fagenbaum, who's considered to be the father of expert systems, The Edge of Chaos that features women, and many more books. I came across her work in an unusual way by stumbling in a quote from Machines Who Think that is something like, artificial intelligence began with the ancient wish to forge the gods. That was a beautiful way to draw a connecting line between our societal relationship with AI from the grounded day-to-day science, math, and engineering to popular stories and science fiction and myths of automatons that go back for centuries. Through her literary work, she has spent a lot of time with the seminal figures of artificial intelligence, including the founding fathers of AI from the 1956 Dartmouth summer workshop where the field was launched. I reached out to Pamela for a conversation in hopes of getting a sense of what those early days were like and how their dreams continued to reverberate through the work of our community today. I often don't know where the conversation may take us, but I jump in and see. Having no constraints, rules, or goals is a wonderful way to discover new ideas. This is the Artificial Intelligence Podcast. If you enjoy it, subscribe on YouTube, give it five stars on iTunes, support it on Patreon, or simply connect with me on Twitter at Lex Friedman, spelled F-R-I-D-M-A-N. And now here's my conversation with Pamela McCordick. In 1979, your book Machines Who Think was published. In it, you interviewed some of the early AI pioneers and explored the idea that AI was born not out of maybe math and computer science, but out of myth and legend. So tell me if you could the story of how you first arrived at the book, the journey of beginning to write it. I had been a novelist. I'd published two novels. And I was sitting under the portal at Stanford one day in the house we were renting for the summer. And I thought, I should write a novel about these weird people in AI I know. And then I thought, ah, don't write a novel, write a history. Simple, just go around, interview them, splice it together, voila, instant book. Ha, ha, ha. It was much harder than that. But nobody else was doing it. And so I thought, well, this is a great opportunity. And there were people who, John McCarthy, for example, thought it was a nutty idea. The field had not evolved yet, so on. And he had some mathematical thing he thought I should write instead. And I said, no, John, I am not a woman in search of a project. This is what I want to do. I hope you'll cooperate. And he said, oh, mutter, mutter, well, OK, it's your time. What was the pitch for the, I mean, such a young field at that point. How do you write a personal history of a field that's so young? I said, this is wonderful. The founders of the field are alive and kicking and able to talk about what they're doing. Did they sound or feel like founders at the time? Did they know that they know that they have founded something? Oh, yeah. They knew what they were doing was very important. What I now see in retrospect is that they were at the height of their research careers. And it's humbling to me that they took time out from all the things that they had to do as a consequence of being there and to talk to this woman who said, I think I'm going to write a book about you. No, it was amazing. Just amazing. So who stands out to you? Maybe looking 63 years ago, the Dartmouth Conference. So Marvin Minsky was there. McCarthy was there. Claude Shannon, Alan Newell, Herb Simon, some of the folks you've mentioned. Then there's other characters, right? One of your co-authors. He wasn't at Dartmouth. He wasn't at Dartmouth. No. But I mean, he was I think an undergraduate then. And of course, Joe Traub. I mean, all of these are players, not at Dartmouth, but in that era. Right. CMU and so on. So who are the characters, if you could paint a picture that stand out to you from memory? Those people you've interviewed and maybe not people that were just in the atmosphere. Of course, the four founding fathers were extraordinary guys. They really were. Who are the founding fathers? Alan Newell, Herbert Simon, Marvin Minsky, John McCarthy. They were the four who were not only at the Dartmouth Conference, but Newell and Simon arrived there with a working program called the Logic Theorist. Everybody else had great ideas about how they might do it, but they weren't going to do it yet. And you mentioned Joe Traub, my husband. I was immersed in AI before I met Joe because I had been Ed Feigenbaum's assistant at Stanford. And before that, I had worked on a book edited by Feigenbaum and Julian Feldman called Computers and Thought. It was the first textbook of readings of AI. And they only did it because they were trying to teach AI to people at Berkeley. And you'd have to send them to this journal and that journal. This was not the internet where you could go look at an article. So I was fascinated from the get-go by AI. I was an English major. What did I know? And yet I was fascinated. And that's why you saw that historical, that literary background, which I think is very much a part of the continuum of AI. That AI grew out of that same impulse. That traditional, what drew you to AI? How did you even think of it back then? What was the possibilities, the dreams? What was interesting to you? The idea of intelligence outside the human cranium, this was a phenomenal idea. And even when I finished Machines Who Think, I didn't know if they were going to succeed. In fact, the final chapter is very wishy-washy, frankly. I'll succeed the field did. Yeah. So was there the idea that AI began with the wish to forge the gods? So the spiritual component that we crave to create this other thing greater than ourselves? For those guys, I don't think so. Newell and Simon were cognitive psychologists. What they wanted was to simulate aspects of human intelligence and they found they could do it on the computer. Minsky just thought it was a really cool thing to do. Likewise, McCarthy. McCarthy had got the idea in 1949 when he was a Caltech student. And he listened to somebody's lecture. It's in my book. I forget who it was. And he thought, oh, that would be fun to do. How do we do that? And he took a very mathematical approach. Minsky was hybrid and Newell and Simon were very much cognitive psychology. How can we simulate various things about human cognition? What happened over the many years is, of course, our definition of intelligence expanded tremendously. These days, biologists are comfortable talking about the intelligence of a cell, the intelligence of the brain, not just human brain, but the intelligence of any kind of brain. Cephalopause, an octopus is really intelligent by any... We wouldn't have thought of that in the 60s, even the 70s. So all these things have worked in. And I did hear one behavioral primatologist, Franz de Waal, say AI taught us the questions to ask. Yeah, this is what happens, right? When you try to build it is when you start to actually ask questions, it puts a mirror to ourselves. Yeah, right. So you were there in the middle of it. It seems like not many people were asking the questions that you were, just trying to look at this field the way you were. I was solo. When I went to get funding for this because I needed somebody to transcribe the interviews and I needed travel expenses, I went to everything you could think of, the NSA, NSF, DARPA. There was an Air Force place that doled out money. And each of them said, well, that was very interesting. That's a very interesting idea, but we'll think about it. And the National Science Foundation actually said to me in plain English, hey, you're only a writer. You're not a historian of science. And I said, yeah, that's true. But the historians of science will be crawling all over this field. I'm writing for the general audience, so I thought. And they still wouldn't budge. I finally got a private grant without knowing who it was from, from Ed Fredkin at MIT. He was a wealthy man and he liked what he called crackpot ideas. And he considered- This a crackpot idea? This a crackpot idea and he was willing to support it. I am ever grateful. Let me say that. Some would say that a history of science approach to AI or even just a history or anything like the book that you've written hasn't been written since. Maybe I'm not familiar, but it's certainly not many. If we think about bigger than just these couple of decades, few decades, what are the roots of AI? Oh, they go back so far. Yes, of course, there's all the legendary stuff, the Golem and the early robots of the 20th century. But they go back much further than that. If you read Homer, Homer has robots in the Iliad. And a classical scholar was pointing out to me just a few months ago, well, you said you just read the Odyssey. The Odyssey is full of robots. It is, I said. Yeah. How do you think Odysseus' ship gets from one place to another? He doesn't have the crew people to do that, the crew men. Yeah, it's magic, it's robots. Oh, I thought. How interesting. So we've had this notion of AI for a long time. And then toward the end of the 19th century, the beginning of the 20th century, there were scientists who actually tried to make this happen some way or another, not successfully. They didn't have the technology for it. And of course, Babbage in the 1850s and 60s, he saw that what he was building was capable of intelligent behavior. And when he ran out of funding, the British government finally said, that's enough. He and Lady Lovelace decided, oh, well, why don't we play the ponies with this? He had other ideas for raising money too. But if we actually reach back once again, I think people don't actually really know that robots do appear or ideas of robots. You talk about the Hellenic and the Hebraic points of view. Oh, yes. Can you tell me about each? Tell me about each. I defined it this way. The Hellenic point of view is robots are great. They are party help. They help this guy Hephaestus, this God Hephaestus in his forge. I presume he made them to help him and so on and so forth. And they welcome the whole idea of robots. The Hebraic view has to do with, I think it's the second commandment, thou shalt not make any graven image. In other words, you better not start imitating humans because that's just forbidden. It's the second commandment. And a lot of the reaction to artificial intelligence has been in a sense that this is somehow wicked. This is somehow blasphemous. We shouldn't be going there. Now, you can say, yeah, but there are going to be some downsides. And I say, yes, there are, but blasphemy is not one of them. There is a kind of fear that feels to be almost primal. Is there religious roots to that? Because so much of our society has religious roots. And so there is a feeling of, like you said, blasphemy, of creating the other, of creating something. It doesn't have to be artificial intelligence. It's creating life in general. It's the Frankenstein idea. It was the annotated Frankenstein on my coffee table. It's a tremendous novel. It really is just beautifully perceptive. Yes, we do fear this and we have good reason to fear it, because it can get out of hand. Maybe you can speak to that fear, the psychology, if you've thought about it. There's a practical set of fears, concerns in the short term. If we actually think about artificial intelligence systems, you can think about bias of discrimination in algorithms. You can think about their social networks have algorithms that recommend the content you see. Thereby, these algorithms control the behavior of the masses. There's these concerns. But to me, it feels like the fear that people have is deeper than that. So have you thought about the psychology of it? I think in a superficial way, I have. There is this notion that if we produce a machine that can think, it will outthink us and therefore replace us. I guess that's a primal fear of almost kind of a kind of mortality. So around the time, you said you work with Stanford, with Ed Fagenbaum. So let's look at that one person throughout his history. Clearly a key person, one of the many in the history of AI. How has he changed in general around him? How has Stanford changed in the last, how many years are we talking about here? Oh, since 65. 65. So maybe it doesn't have to be about him. It could be bigger, but because he was a key person in expert systems, for example, how is that? How are these folks who you've interviewed in the 70s, 79 changed through the decades? In Ed's case, I know him well. We are dear friends. We see each other every month or so. He told me that when Machines Who Think first came out, he really thought all the front matter was kind of baloney. He said, well, all the front matter was kind of baloney. And 10 years later, he said, no, I see what you're getting at. Yes, this has been a human impulse for thousands of years to create something outside the human cranium that has intelligence. I think it's very hard when you're down at the algorithmic level and you're just trying to make something work, which is hard enough, just step back and think of the big picture. It reminds me of when I was in Santa Fe, I knew a lot of archeologists, which was a hobby of mine. And I would say, yeah, yeah, well, you can look at the shards and say, oh, this came from this tribe and this came from this trade route and so on. But what about the big picture? And a very distinguished archeologist said to me, they don't think that way. No, they're trying to match the shard to where it came from. Where did this corn, remainder of this corn come from? Was it grown here? Was it grown elsewhere? And I think this is part of the AI, any scientific field. You're so busy doing the hard work, and it is hard work, that you don't step back and say, oh, well, now let's talk about the general meaning of all this. Yeah. So none of the, even Minsky and McCarthy, they- Oh, those guys did. Yeah, the founding fathers did. Early on or- Pretty early on. But in a different way from how I looked at it, the two cognitive psychologists, Newell and Simon, they wanted to imagine reforming cognitive psychology so that we would really, really understand the brain. Minsky was more speculative, and John McCarthy saw it as, I think I'm doing him right by this, he really saw it as a great boon for human beings to have this technology, and that was reason enough to do it. And he had wonderful, wonderful fables about how, if you do the mathematics, you will see that these things are really good for human beings. And if you had a technological objection, he had an answer, a technological answer, but here's how we could get over that, and then blah, blah, blah, blah. And one of his favorite things was what he called the literary problem, which of course he presented to me several times. That is, everything in literature, there are conventions in literature. One of the conventions is that you have a villain and a hero, and the hero in most literature is human, and the villain in most literature is a machine. And he said, no, that's just not the way it's going to be, but that's the way we're used to it. So when we tell stories about AI, it's always with this paradigm. I thought, yeah, he's right. Looking back, the classics, RUR is certainly the machines trying to overthrow the humans. Frankenstein is different. Frankenstein is a creature. He never has a name. Frankenstein, of course, is the guy who created him, the human, Dr. Frankenstein. This creature wants to be loved, wants to be accepted, and it is only when Frankenstein turns his head, in fact, runs the other way, and the creature is without love that he becomes the monster that he later becomes. So who's the villain in Frankenstein? It's unclear, right? Oh, it is unclear, yeah. It's really the people who drive him. By driving him away, they bring out the worst. That's right. They give him no human solace, and he is driven away. You're right. He becomes, at one point, the friend of a blind man, and he serves this blind man, and they become very friendly. But when the sighted people of the blind man's family come in, ah, you've got a monster here. So it's very didactic in its way. What I didn't know is that Mary Shelley and Percy Shelley were great readers of the literature surrounding abolition in the United States, the abolition of slavery, and they picked that up wholesale. You know, you are making monsters of these people because you won't give them the respect and love that they deserve. Do you have, if we get philosophical for a second, do you worry that once we create machines that are a little bit more intelligent, let's look at Roomba, the vacuum cleaner, that this darker part of human nature where we abuse the other, the somebody who's different, will come out? I don't worry about it. I could imagine it happening. But I think that what AI has to offer the human race will be so attractive that people will be won over. So you have looked deep into these people, have deep conversations, and it's interesting to get a sense of stories of the way they were thinking and the way it was changed, the way your own thinking about AI has changed. So you mentioned McCarthy. What about the years at CMU, Carnegie Mellon with Joe? Sure. Joe was not in AI. He was in algorithmic complexity. Was there always a line between AI and computer science, for example? Is AI its own place of outcasts? Was that the feeling? There was a kind of outcast period for AI. For instance, in 1974, the new field was hardly 10 years old. The new field of computer science was asked by the National Science Foundation, I believe, but it may have been the National Academies, I can't remember, to tell your fellow scientists where computer science is and what it means. And they wanted to leave out AI. And they only agreed to put it in because Don Knuth said, hey, this is important. You can't just leave that out. Really? Don Knuth? Don Knuth, yes. I talked to him recently too. Out of all the people. Yes. But you see, an AI person couldn't have made that argument. He wouldn't have been believed, but Knuth was believed. Yes. So Joe Traub worked on the real stuff. Joe was working on algorithmic complexity, but he would say in plain English again and again, the smartest people I know are in AI. Really? Oh, yes. No question. Anyway, Joe loved these guys. What happened was that, I guess it was as I started to write Machines Who Think, Herb Simon and I both wrote it. Herb Simon and I became very close friends. He would walk past our house on Northumberland Street every day after work. And I would just be putting my cover on my typewriter. And I would lean out the door and say, Herb, would you like a sherry? And Herb almost always would like a sherry. So he'd stop in and we'd talk for an hour, two hours. My journal says we talked this afternoon for three hours. What was on his mind at the time in terms of on the AI side of things? Oh, we didn't talk too much about AI. We talked about other things. Just life. We both love literature. And Herb had read Proust in the original French twice all the way through. I can't, I've read it in English in translation. So we talked about literature. We talked about languages. We talked about music because he loved music. We talked about art because he was actually enough of a painter that he had to give it up because he was afraid it was interfering with his research and so on. So no, it was really just chat chat, but it was very warm. So one summer, I said to Herb, you know, my students have all the really interesting conversations. I was teaching at the University of Pittsburgh then in the English department. They get to talk about the meaning of life and that kind of thing. And what do I have? I have university meetings where we talk about the photocopying budget and whether the course on romantic poetry should be one semester or two. So Herb laughed. He said, yes, I know what you mean. He said, but you could do something about that. Dot, that was his wife, Dot and I used to have a salon at the University of Chicago every Sunday night. And we would have essentially an open house and people knew it wasn't for a small talk. It was really for some topic of depth. He said, but my advice would be that you choose the topic ahead of time. Fine. I said, so the following, we exchanged mail over the summer. That was US post in those days because you didn't have personal email. And I decided I would organize it and there would be eight of us, Alan Null and his wife, Herb Simon and his wife, Dorothea. There was a novelist in town, a man named Mark Harris. He had just arrived and his wife, Josephine. Mark was most famous then for a novel called Bang the Drum Slowly, which was about baseball and Joe and me, so eight people. And we met monthly and we just sank our teeth into really hard topics. And it was great fun. How have your own views around artificial intelligence changed through the process of writing Machines Who Think and afterwards the ripple effects? I was a little skeptical that this whole thing would work out. It didn't matter to me. It was so audacious. This whole thing being AI. AI generally, yeah. And in some ways, it hasn't worked out the way I expected so far. That is to say there's this wonderful lot of apps thanks to deep learning and so on, but those are algorithmic. And in the part of symbolic processing, there is very little yet. And that's a field that lies waiting for industrious graduate students. Maybe you can tell me some figures that popped up in your life in the 80s with expert systems where there was the symbolic AI possibilities of what most people think of as AI. If you dream of the possibilities, AI is really expert systems. And those hit a few walls and there were challenges there. And I think, yes, they will reemerge again with some new breakthroughs and so on. But what did that feel like, both the possibility and the winter that followed, the slowdown and research? Ah, you know, this whole thing about AI winter is to me a crock. It's no winters. Because I look at the basic research that was being done in the 80s, which is supposed to be, my God, it was really important. It was laying down things that nobody had thought about before, but it was basic research. You couldn't monetize it, hence the winter. Hence the winter. You know, research, scientific research goes and fits and starts. It isn't this nice, smooth, oh, this follows this, follows this. No, it just doesn't work that way. The interesting thing, the way winters happen, it's never the fault of the researchers. It's the some source of hype over promising. Well, no, let me take that back. Sometimes it is the fault of the researchers. Sometimes certain researchers might over promise the possibilities. They themselves believe that we're just a few years away, sort of just recently talked to Elon Musk and he believes he'll have an autonomous vehicle. We'll have autonomous vehicles in a year. And he believes it. A year? A year, yeah. We'll have mass deployment of our time. For the record, this is 2019 right now. So he's talking 2020. To do the impossible, you really have to believe it. And I think what's going to happen when you believe it, because there's a lot of really brilliant people around him, is some good stuff will come out of it. Some unexpected, brilliant breakthroughs will come out of it when you really believe it, when you work that hard. Yeah, I believe that. And I believe autonomous vehicles will come. I just don't believe it'll be in a year. I wish. But nevertheless, autonomous vehicles is a good example. There's a feeling many companies have promised by 2021, by 2022, Ford, GM, basically every single automotive company has promised they'll have autonomous vehicles. So that kind of over promise is what leads to the winter. Because we'll come to those dates. There won't be autonomous vehicles. And there'll be a feeling, well, wait a minute, if we took your word at that time, that means we just spent billions of dollars, had made no money, and there's a counter response to where everybody gives up on it. Sort of intellectually, at every level, the hope just dies. And all that's left is a few basic researchers. So you're uncomfortable with some aspects of this idea. Well, it's the difference between science and commerce. So you think science goes on the way it does? Science can really be killed by not getting proper funding or timely funding. I think Great Britain was a perfect example of that. The Lighthill report in, I don't remember the year, essentially said, there's no use Great Britain putting any money into this. It's going nowhere. And this was all about social factions in Great Britain. Edinburgh hated Cambridge and Cambridge hated Manchester. Somebody else can write that story. But it really did have a hard effect on research there. Now, they've come roaring back with deep mind, but that's one guy and his visionaries around him. But just to push on that, it's kind of interesting. You have this dislike of the idea of an AI winter. Where's that coming from? Where were you? Oh, because I just don't think it's true. There was a particular periods of time. It's a romantic notion, certainly. Yeah, well, I admire science, perhaps more than I admire commerce. Commerce is fine. Hey, you know, we all got to live. But science has a much longer view than commerce and continues almost regardless. It can't continue totally regardless, but it's almost regardless of what's saleable and what's not, what's monetizable and what's not. So the winter is just something that happens on the commerce side and the science marches. That's a beautifully optimistic, inspiring message. I agree with you. I think if we look at the key people that work in AI, they work in key scientists, most disciplines, they continue working out of the box. They continue working out of the box. They work in key scientists, most disciplines, they continue working out of the love for science. No matter the, you can always scrape up some funding to stay alive and they continue working diligently, but there certainly is a huge amount of funding now and there's a concern on the AI side and deep learning. There's a concern that we might with over-promising hit another slowdown in funding, which does affect the number of students, that kind of thing. Yeah, it does. So the kind of ideas you had in Michigan to think, did you continue that curiosity through the decades that followed? Yes, I did. What was your view, historical view of how AI community evolved, the conversations about it, the work? Has it persisted the same way from its birth? No, of course not. It's just, as we were just talking, the symbolic AI really kind of dried up and it all became algorithmic. I remember a young AI student telling me what he was doing and I had been away from the field long enough. I'd gotten involved with complexity at the Santa Fe Institute. I thought, algorithms, yeah, they're in the service of, but they're not the main event. No, they became the main event. That surprised me. And we all know the downside of this. We all know that if you're using an algorithm to make decisions based on a gazillion human decisions, baked into it are all the mistakes that humans make, the bigotries, the short-sightedness, so on and so on. So you mentioned Santa Fe Institute. So you've written the novel Edge of Chaos, but it's inspired by the ideas of complexity, a lot of which have been extensively explored at the Santa Fe Institute. It's another fascinating topic of just sort of emergent complexity from chaos. Nobody knows how it happens really, but it seems to wear all the interesting stuff that does happen. So how did first, not your novel, but just complexity in general in the work at Santa Fe fit into the bigger puzzle of the history of AI, or maybe even your personal journey through that? One of the last projects I did concerning AI in particular was looking at the work of Harold Cohen, the painter. And Harold was deeply involved with AI. He was a painter first, and what his project, Aran, which was a lifelong project, did was reflect his own cognitive processes. Harold and I, even though I wrote a book about it, we had a lot of friction between us. And I went, I thought, this is it, the book died, it was published and fell into a ditch. This is it, I'm finished, it's time for me to do something different. By chance, this was a sabbatical year for my husband, and we spent two months at the Santa Fe Institute and two months at Caltech, and then the spring semester in Munich, Germany. Okay, those two months at the Santa Fe Institute were so restorative for me. The institute was very small then. It was in some kind of office complex on old Santa Fe Trail. Everybody kept their door open. So you could crack your head on a problem, and if you finally didn't get it, you could walk in to see Stuart Kaufman or any number of people and say, I don't get this, can you explain? And one of the people that I was talking to about complex adaptive systems was Murray Gell-Mann. And I told Murray what Harold Cohen had done. And I said, you know, this sounds to me like a complex adaptive system. And he said, yeah, it is. Well, what do you know? Harold's errand had all these kissin' cousins all over the world. In science and in economics and so on and so forth. I was so relieved. I thought, okay, your instincts are okay. You're doing the right thing. I didn't have the vocabulary. And that was one of the things that the Santa Fe Institute gave me. If I could have rewritten that book, no, it had just come out. I couldn't rewrite it. I would have had a vocabulary to explain what Aaron was doing. Okay. So I got really interested in what was going on at the Institute. The people were, again, bright and funny and willing to explain anything to this amateur. George Cowan, who was then the head of the Institute, said he thought it might be a nice idea if I wrote a book about the Institute. And I thought about it. And I had a book called The God Knows What. And I said, I'm sorry, George. Yeah, I'd really love to do it, but, you know, just not going to work for me at this moment. And he said, oh, too bad. I think it would make an interesting book. Well, he was right and I was wrong. I wish I'd done it. But that's interesting. I hadn't thought about that. That that was a road not taken that I wish I'd taken. Well, you know what? That's just on that point. It's quite brave for you as a writer, as sort of coming from a world of literature, the literary thinking and historical thinking. I mean, just from that world and bravely talking to quite, I assume, large egos in AI or in complexity and so on. How'd you do it? Like, where did you? I mean, I suppose they could be intimidated of you as well, because it's two different worlds. I don't think I never picked up that anybody was intimidated by me. But how were you brave enough? Where did you find the guts? God, just dumb, dumb luck. I mean, this is an interesting rock to turn over. I'm going to write a book about it. And, you know, people have enough patience with writers if they think they're going to end up in a book that they let you flail around and so on. Well, but they also look if the writer has, there's like, if there's a sparkle in their eye, if they get it. Yeah, sure. Right. When were you at the Santa Fe Institute? The time I'm talking about is 1990, yeah, 1990, 1991, 92. But we then, because Joe was an external faculty member, we're in Santa Fe every summer. We bought a house there and I didn't have that much to do with the Institute anymore. I was writing my novels. I was doing whatever I was doing. But I loved the Institute and I loved the, again, the audacity of the ideas. That really appeals to me. I think that there's this feeling, much like in great, great institutes of neuroscience, for example, that they're in it for the long game of understanding something fundamental about reality and nature. And that's really exciting. So if we start to not to look a little bit more recently, how AI is really popular today. How is this world, you mentioned algorithmic, but in general, is the spirit of the people, the kind of conversations you hear through the gray virus on, is that different than the roots that you remember? No, the same kind of excitement, the same kind of, this is really going to make a difference in the world. And it will, it has. A lot of folks, especially young, 20 years old or something, they think we've just found something special here. We're going to change the world tomorrow. On a time scale, do you have a sense of what, of the time scale at which breakthroughs in AI happen? I really don't. I don't think so. I really don't. Because look at deep learning. That was a, Jeffrey Hinton came up with the algorithm in 86. But it took all these years for the technology to be good enough to actually be applicable. So no, I can't predict that at all. I can't, I wouldn't even try. Well, let me ask you to not to try to predict, but to speak to the, I'm sure in the 60s, as it continues now, there's people that think, let's call it, we can call it this fun word, the singularity. When there's a phase shift, there's some profound feeling where we're all really surprised by what's able to be achieved. I'm sure those dreams are there. I remember reading quotes in the 60s and those continued. How have your own views, maybe if you look back about the timeline of a singularity changed? Well, I'm not a big fan of the singularity as Ray Kurzweil has presented it. How would you define the Ray Kurzweil? How do you think of singularity in those? If I understand Kurzweil's view, it's sort of, there's going to be this moment when machines are smarter than humans and game over. However, the game over is, I mean, do they put us on a reservation? Do they, et cetera, et cetera. And first of all, machines are sort of, first of all, machines are smarter than humans in some ways all over the place. And they have been since adding machines were invented. So it's not going to come like some great eatable crossroads where they meet each other and our offspring, Oedipus says, you're dead. It's just not going to happen. Yeah. So it's already game over with calculators, right? They're already out to do much better at basic arithmetic than us. But there's a human-like intelligence and that's not the ones that destroy us, but somebody that you can have as a friend, you can have deep connections with that kind of passing the touring test and beyond those kinds of ideas. Have you dreamt of those? Oh, yes. Yes. Yes. Those possibilities. In a book I wrote with Ed Feigenbaum, there's a little story called The Geriatric Robot. And how I came up with The Geriatric Robot is a story in itself, but here's what The Geriatric Robot does. It doesn't just clean you up and feed you and wheel you out into the sun. It's great advantages. It listens. It says, tell me again about the great coup of 73. Tell me again about how awful or how wonderful your grandchildren are and so on and so forth. And it isn't hanging around to inherit your money. It isn't hanging around because it can't get any other job. This is its job and so on and so forth. Well, I would love something like that. Yeah. I mean, for me, that deeply excites me. So I think there's a lot of us- Like you gotta know, it was a joke. I dreamed it up because I needed to talk to college students and I needed to give them some idea of what AI might be. And they were rolling in the aisles as I elaborated and elaborated and elaborated. When it went into the book, they took my hide off in the New York Review of Books. This is just what we've thought about these people in AI. They're inhuman. Oh, come on, get over it. Don't you think that's a good thing for the world that AI could potentially- I do, absolutely. And furthermore, I'm pushing 80 now. By the time I need help like that, I also want it to roll itself in a corner and shut the fuck up. Let me linger on that point. Do you really though? Yeah, I do. Here's why. Don't you want it to push back a little bit? A little, but I have watched my friends go through the whole issue around having help in the house. And some of them have been very lucky and had fabulous help. And some of them have had people in the house who want to keep the television going on all day, who want to talk on their phones all day. No. So basically- Just roll yourself in the corner and shut up. Yeah. Unfortunately, us humans, when we're assistants, we still, even when we're assisting others, we care about ourselves more. Of course. And so you create more frustration. And a robot, AI assistant can really optimize the experience for you. I was just speaking to the point- you actually bring up a very, very good point. But I was speaking to the fact that us humans are a little complicated, that we don't necessarily want a perfect servant. I don't- maybe you disagree with that, but there's a- I think there's a push and pull with humans. A little tension, a little mystery that, of course, that's really difficult for AI to get right. But I do sense, especially in today with social media, that people are getting more and more lonely, even young folks. And sometimes especially young folks, that loneliness, there's a longing for connection and AI can help alleviate some of that loneliness. Some, just somebody who listens, like in person. So to speak. Yeah. So to speak. Yeah. That to me is really exciting. But so if we look at that level of intelligence, which is exceptionally difficult to achieve actually, as the singularity or whatever, that's the human level bar, that people have dreamt of that too. Turing dreamt of it. He had a date timeline. Do you have, how have your own timeline evolved on past? I don't even think about it. You don't even think? No. Just this field has been so full of surprises for me. You just take it in and see. Yeah. Oh, that's great. Yeah. I just can't. Maybe that's because I've been around the field long enough to think, you know, don't go that way. Herb Simon was terrible about making these predictions of when this and that would happen. And he was a sensible guy. His quotes are often used, right? As a legend. Yeah. Do you have concerns about AI, the existential threats of many people like Elon Musk and Sam Harris and others that are thinking about? Oh, yeah. That takes up half a chapter in my book. I call it the male gaze. Well, you hear me out. The male gaze is actually a term from film criticism. And I'm blocking on the women who dream this up, but she pointed out how most movies were made from the male point of view, that women were objects, not subjects. They didn't have any agency, so on and so forth. So when Elon and his pals Hawking and so on came, AI is going to eat our lunch and our dinner and our midnight snack too, I thought, what? And I said to Ed Feigenbaum, oh, these guys have always been the smartest guy on the block, and here comes something that might be smarter. Ooh, let's stamp it out before it takes over. And Ed laughed. He said, I didn't think about it that way. But I did. I did. And it is the male gaze. Okay, suppose these things do have agency. Well, let's wait and see what happens. Can we imbue them with ethics? Can we imbue them with a sense of empathy? Or are they just going to be, I don't know, we've had centuries of guys like that. That's interesting that the ego, the male gaze is immediately threatened. And so you can't think in a patient, calm way of how the tech could evolve. Speaking of which, you're a 96 book, The Future of Women. I think at the time and now, certainly now, I mean, I'm sorry, maybe at the time, but I'm more cognizant of now, is extremely relevant. You and Nancy Ramsey talk about four possible futures of women in science and tech. So if we look at the decades before and after the book was released, can you tell a history, sorry, of women in science and tech and how it has evolved? How have things changed? Where do we stand? Not enough. They have not changed enough. The way that women are ground down in computing is simply unbelievable. But what are the four possible futures for women in tech from the book? What you're really looking at are various aspects of the present. So for each of those, you could say, oh yeah, we do have backlash. Look at what's happening with abortion and so on and so forth. We have one step forward, one step back. The golden age of equality was the hardest chapter to write. And I used something from the Santa Fe Institute, which is the sandpile effect that you drop sand very slowly onto a pile and it grows and it grows and it grows until suddenly it just breaks apart. And in a way, me too has done that. That was the last drop of sand that broke everything apart. That was a perfect example of the sandpile effect. And that made me feel good. It didn't change all of society, but it really woke a lot of people up. But are you in general optimistic about maybe after me too? I mean, me too is about a very specific kind of thing. Boy, solve that and you solve everything. But are you in general optimistic about the future? Yes, I'm a congenital optimistic. I can't help it. What about AI? What are your thoughts about the future of AI? Of course I get asked what do you worry about? And the one thing I worry about is the things that we can't anticipate. There's going to be something out of left field that we will just say, we weren't prepared for that. I am generally optimistic. When I first took up being interested in AI, like most people in the field, more intelligence was like more virtue. What could be bad? And in a way, I still believe that, but I realized that my notion of intelligence has broadened. There are many kinds of intelligence and we need to imbue our machines with those many kinds. So you've now just finished or in the process of finishing the book that you've been working on, the memoir. How have you changed? I know it's just writing, but how have you changed the process? If you look back, what kind of stuff did it bring up to you that surprised you looking at the entirety of it all? The biggest thing, and it really wasn't a surprise, is how lucky I was. Oh my, to have access to the beginning of a scientific field that is going to change the world. How did I luck out? And yes, of course, my view of things has widened a lot. If I can get back to one feminist part of our conversation, without knowing it, it really was subconscious. I wanted AI to succeed because I was so tired of hearing that intelligence was inside the male cranium. And I thought if there was something out there that wasn't a male thinking and doing well, then that would put a lie to this whole notion of intelligence resides in the male cranium. I did not know that until one night Harold Cohen and I were having a glass of wine, maybe two, and he said, what drew you to AI? And I said, oh, you know, smartest people I knew, great project, blah, blah, blah. And I wanted something besides male smarts. And it just bubbled up out of me, Lex. And I, what? What's this? It's kind of brilliant, actually. So AI really humbles all of us and humbles the people that need to be humbled the most. Let's hope. Oh, wow. That is so beautiful. Pamela, thank you so much for talking to me. Oh, it's been a great pleasure. Thank you.
 The following is a conversation with Pamela McCordick. She's an author who has written on
 the history and the philosophical significance of artificial intelligence. Her books include
 Machines Who Think in 1979, The Fifth Generation in 1983 with Ed Fagenbaum, who's considered to be
 the father of expert systems, The Edge of Chaos that features women, and many more books.
 I came across her work in an unusual way by stumbling in a quote from Machines Who Think
 that is something like, artificial intelligence began with the ancient wish to forge the gods.
 That was a beautiful way to draw a connecting line between our societal relationship with AI
 from the grounded day-to-day science, math, and engineering to popular stories and science fiction
 and myths of automatons that go back for centuries. Through her literary work, she has spent a lot of
 time with the seminal figures of artificial intelligence, including the founding fathers of
 AI from the 1956 Dartmouth summer workshop where the field was launched. I reached out to Pamela
 for a conversation in hopes of getting a sense of what those early days were like and how their
 dreams continued to reverberate through the work of our community today. I often don't know where
 the conversation may take us, but I jump in and see. Having no constraints, rules, or goals is a
 wonderful way to discover new ideas. This is the Artificial Intelligence Podcast. If you enjoy it,
 subscribe on YouTube, give it five stars on iTunes, support it on Patreon, or simply connect with me
 on Twitter at Lex Friedman, spelled F-R-I-D-M-A-N. And now here's my conversation with Pamela McCordick.
 In 1979, your book Machines Who Think was published. In it, you interviewed some of
 the early AI pioneers and explored the idea that AI was born not out of maybe math and computer
 science, but out of myth and legend. So tell me if you could the story of how you first arrived at
 the book, the journey of beginning to write it. I had been a novelist. I'd published two novels.
 And I was sitting under the portal at Stanford one day in the house we were renting for the summer.
 And I thought, I should write a novel about these weird people in AI I know. And then I thought,
 ah, don't write a novel, write a history. Simple, just go around, interview them, splice it together,
 voila, instant book. Ha, ha, ha. It was much harder than that. But nobody else was doing it.
 And so I thought, well, this is a great opportunity. And there were people who, John McCarthy,
 for example, thought it was a nutty idea. The field had not evolved yet, so on. And he had some
 mathematical thing he thought I should write instead. And I said, no, John, I am not a woman
 in search of a project. This is what I want to do. I hope you'll cooperate. And he said,
 oh, mutter, mutter, well, OK, it's your time. What was the pitch for the, I mean, such a young
 field at that point. How do you write a personal history of a field that's so young? I said,
 this is wonderful. The founders of the field are alive and kicking and able to talk about what
 they're doing. Did they sound or feel like founders at the time? Did they know that they
 know that they have founded something? Oh, yeah. They knew what they were doing was very important.
 What I now see in retrospect is that they were at the height of their research careers.
 And it's humbling to me that they took time out from all the things that they had to do as a
 consequence of being there and to talk to this woman who said, I think I'm going to write a book
 about you. No, it was amazing. Just amazing. So who stands out to you? Maybe looking 63 years ago,
 the Dartmouth Conference. So Marvin Minsky was there. McCarthy was there. Claude Shannon,
 Alan Newell, Herb Simon, some of the folks you've mentioned. Then there's other characters, right?
 One of your co-authors. He wasn't at Dartmouth. He wasn't at Dartmouth. No. But I mean,
 he was I think an undergraduate then. And of course, Joe Traub. I mean, all of these are players,
 not at Dartmouth, but in that era. Right. CMU and so on. So who are the characters,
 if you could paint a picture that stand out to you from memory? Those people you've interviewed
 and maybe not people that were just in the atmosphere. Of course, the four founding fathers
 were extraordinary guys. They really were. Who are the founding fathers? Alan Newell, Herbert Simon,
 Marvin Minsky, John McCarthy. They were the four who were not only at the Dartmouth Conference,
 but Newell and Simon arrived there with a working program called the Logic Theorist. Everybody else
 had great ideas about how they might do it, but they weren't going to do it yet.
 And you mentioned Joe Traub, my husband. I was immersed in AI before I met Joe
 because I had been Ed Feigenbaum's assistant at Stanford. And before that, I had worked on a book
 edited by Feigenbaum and Julian Feldman called Computers and Thought. It was the first textbook
 of readings of AI. And they only did it because they were trying to teach AI to people at Berkeley.
 And you'd have to send them to this journal and that journal. This was not the internet where you
 could go look at an article. So I was fascinated from the get-go by AI. I was an English major.
 What did I know? And yet I was fascinated. And that's why you saw that historical,
 that literary background, which I think is very much a part of the continuum of AI.
 That AI grew out of that same impulse. That traditional, what drew you to AI?
 How did you even think of it back then? What was the possibilities, the dreams? What was interesting to you?
 The idea of intelligence outside the human cranium, this was a phenomenal idea.
 And even when I finished Machines Who Think, I didn't know if they were going to succeed.
 In fact, the final chapter is very wishy-washy, frankly. I'll succeed the field did.
 Yeah. So was there the idea that AI began with the wish to forge the gods? So the spiritual
 component that we crave to create this other thing greater than ourselves?
 For those guys, I don't think so. Newell and Simon were cognitive psychologists.
 What they wanted was to simulate aspects of human intelligence and they found they could do it on the computer.
 Minsky just thought it was a really cool thing to do. Likewise, McCarthy. McCarthy had got the idea
 in 1949 when he was a Caltech student. And he listened to somebody's lecture. It's in my book.
 I forget who it was. And he thought, oh, that would be fun to do. How do we do that?
 And he took a very mathematical approach. Minsky was hybrid and Newell and Simon were very much
 cognitive psychology. How can we simulate various things about human cognition?
 What happened over the many years is, of course, our definition of intelligence expanded tremendously.
 These days, biologists are comfortable talking about the intelligence of a cell,
 the intelligence of the brain, not just human brain, but the intelligence of any kind of brain.
 Cephalopause, an octopus is really intelligent by any... We wouldn't have thought of that
 in the 60s, even the 70s. So all these things have worked in. And I did hear one behavioral
 primatologist, Franz de Waal, say AI taught us the questions to ask.
 Yeah, this is what happens, right? When you try to build it is when you start to actually
 ask questions, it puts a mirror to ourselves. Yeah, right.
 So you were there in the middle of it. It seems like not many people were asking the questions
 that you were, just trying to look at this field the way you were.
 I was solo. When I went to get funding for this because I needed somebody to transcribe the
 interviews and I needed travel expenses, I went to everything you could think of, the NSA,
 NSF, DARPA. There was an Air Force place that doled out money. And each of them said,
 well, that was very interesting. That's a very interesting idea, but we'll think about it.
 And the National Science Foundation actually said to me in plain English, hey, you're only a writer.
 You're not a historian of science. And I said, yeah, that's true. But the historians of science
 will be crawling all over this field. I'm writing for the general audience, so I thought. And they
 still wouldn't budge. I finally got a private grant without knowing who it was from, from Ed
 Fredkin at MIT. He was a wealthy man and he liked what he called crackpot ideas. And he considered-
 This a crackpot idea?
 This a crackpot idea and he was willing to support it. I am ever grateful. Let me say that.
 Some would say that a history of science approach to AI or even just a history or anything like the
 book that you've written hasn't been written since. Maybe I'm not familiar, but it's certainly not
 many. If we think about bigger than just these couple of decades, few decades, what are the
 roots of AI?
 Oh, they go back so far. Yes, of course, there's all the legendary stuff, the Golem and
 the early robots of the 20th century. But they go back much further than that. If you read Homer,
 Homer has robots in the Iliad. And a classical scholar was pointing out to me just a few months
 ago, well, you said you just read the Odyssey. The Odyssey is full of robots. It is, I said.
 Yeah. How do you think Odysseus' ship gets from one place to another? He doesn't have the crew
 people to do that, the crew men. Yeah, it's magic, it's robots. Oh, I thought. How interesting.
 So we've had this notion of AI for a long time. And then toward the end of the 19th century,
 the beginning of the 20th century, there were scientists who actually tried to make this happen
 some way or another, not successfully. They didn't have the technology for it. And of course,
 Babbage in the 1850s and 60s, he saw that what he was building was capable of intelligent behavior.
 And when he ran out of funding, the British government finally said, that's enough.
 He and Lady Lovelace decided, oh, well, why don't we play the ponies with this?
 He had other ideas for raising money too. But if we actually reach back once again,
 I think people don't actually really know that robots do appear or ideas of robots.
 You talk about the Hellenic and the Hebraic points of view. Oh, yes. Can you tell me about each?
 Tell me about each. I defined it this way. The Hellenic point of view is robots are great.
 They are party help. They help this guy Hephaestus, this God Hephaestus in his forge.
 I presume he made them to help him and so on and so forth. And they welcome the whole idea of robots.
 The Hebraic view has to do with, I think it's the second commandment, thou shalt not make
 any graven image. In other words, you better not start imitating humans because that's just
 forbidden. It's the second commandment. And a lot of the reaction to artificial intelligence has been
 in a sense that this is somehow wicked. This is somehow blasphemous. We shouldn't be going there.
 Now, you can say, yeah, but there are going to be some downsides. And I say, yes, there are,
 but blasphemy is not one of them. There is a kind of fear that feels to be almost primal.
 Is there religious roots to that? Because so much of our society has religious roots. And so there
 is a feeling of, like you said, blasphemy, of creating the other, of creating something.
 It doesn't have to be artificial intelligence. It's creating life in general. It's the Frankenstein
 idea. It was the annotated Frankenstein on my coffee table. It's a tremendous novel. It really
 is just beautifully perceptive. Yes, we do fear this and we have good reason to fear it,
 because it can get out of hand. Maybe you can speak to that fear,
 the psychology, if you've thought about it. There's a practical set of fears, concerns in
 the short term. If we actually think about artificial intelligence systems, you can think
 about bias of discrimination in algorithms. You can think about their social networks have
 algorithms that recommend the content you see. Thereby, these algorithms control the behavior
 of the masses. There's these concerns. But to me, it feels like the fear that people have
 is deeper than that. So have you thought about the psychology of it?
 I think in a superficial way, I have. There is this notion that if we
 produce a machine that can think, it will outthink us and therefore replace us.
 I guess that's a primal fear of almost kind of a kind of mortality. So around the time,
 you said you work with Stanford, with Ed Fagenbaum. So let's look at that one person
 throughout his history. Clearly a key person, one of the many in the history of AI. How has he
 changed in general around him? How has Stanford changed in the last, how many years are we talking
 about here? Oh, since 65. 65. So maybe it doesn't have to be about him. It could be bigger,
 but because he was a key person in expert systems, for example, how is that? How are these folks who
 you've interviewed in the 70s, 79 changed through the decades? In Ed's case, I know him well. We are
 dear friends. We see each other every month or so. He told me that when Machines Who Think first
 came out, he really thought all the front matter was kind of baloney. He said, well,
 all the front matter was kind of baloney. And 10 years later, he said, no, I see what you're
 getting at. Yes, this has been a human impulse for thousands of years to create something outside
 the human cranium that has intelligence. I think it's very hard when you're down
 at the algorithmic level and you're just trying to make something work, which is hard enough,
 just step back and think of the big picture. It reminds me of when I was in Santa Fe, I knew a
 lot of archeologists, which was a hobby of mine. And I would say, yeah, yeah, well, you can look
 at the shards and say, oh, this came from this tribe and this came from this trade route and so
 on. But what about the big picture? And a very distinguished archeologist said to me, they don't
 think that way. No, they're trying to match the shard to where it came from. Where did this corn,
 remainder of this corn come from? Was it grown here? Was it grown elsewhere? And I think this
 is part of the AI, any scientific field. You're so busy doing the hard work, and it is hard work,
 that you don't step back and say, oh, well, now let's talk about the general meaning of all this.
 Yeah. So none of the, even Minsky and McCarthy, they-
 Oh, those guys did. Yeah, the founding fathers did.
 Early on or-
 Pretty early on. But in a different way from how I looked at it, the two cognitive psychologists,
 Newell and Simon, they wanted to imagine reforming cognitive psychology so that we would really,
 really understand the brain. Minsky was more speculative, and John McCarthy saw it as,
 I think I'm doing him right by this, he really saw it as a great boon for human beings to have this
 technology, and that was reason enough to do it. And he had wonderful, wonderful fables about how,
 if you do the mathematics, you will see that these things are really good for human beings.
 And if you had a technological objection, he had an answer, a technological answer,
 but here's how we could get over that, and then blah, blah, blah, blah. And one of his favorite
 things was what he called the literary problem, which of course he presented to me several times.
 That is, everything in literature, there are conventions in literature. One of the conventions
 is that you have a villain and a hero, and the hero in most literature is human, and the villain
 in most literature is a machine. And he said, no, that's just not the way it's going to be,
 but that's the way we're used to it. So when we tell stories about AI, it's always with this
 paradigm. I thought, yeah, he's right. Looking back, the classics, RUR is certainly the machines
 trying to overthrow the humans. Frankenstein is different. Frankenstein is a creature. He never
 has a name. Frankenstein, of course, is the guy who created him, the human, Dr. Frankenstein.
 This creature wants to be loved, wants to be accepted, and it is only when Frankenstein
 turns his head, in fact, runs the other way, and the creature is without love
 that he becomes the monster that he later becomes.
 So who's the villain in Frankenstein? It's unclear, right?
 Oh, it is unclear, yeah. It's really the people who drive him. By driving him away,
 they bring out the worst. That's right. They give him no human solace, and he is driven away.
 You're right. He becomes, at one point, the friend of a blind man, and he serves this blind man,
 and they become very friendly. But when the sighted people of the blind man's family come in,
 ah, you've got a monster here. So it's very didactic in its way. What I didn't know is that
 Mary Shelley and Percy Shelley were great readers of the literature surrounding abolition in the
 United States, the abolition of slavery, and they picked that up wholesale. You know, you are making
 monsters of these people because you won't give them the respect and love that they deserve.
 Do you have, if we get philosophical for a second, do you worry that once we create machines that
 are a little bit more intelligent, let's look at Roomba, the vacuum cleaner, that this darker
 part of human nature where we abuse the other, the somebody who's different, will come out?
 I don't worry about it. I could imagine it happening. But I think that what AI has to offer
 the human race will be so attractive that people will be won over.
 So you have looked deep into these people, have deep conversations, and it's interesting to get a
 sense of stories of the way they were thinking and the way it was changed, the way your own
 thinking about AI has changed. So you mentioned McCarthy. What about the years at CMU, Carnegie
 Mellon with Joe? Sure. Joe was not in AI. He was in algorithmic complexity.
 Was there always a line between AI and computer science, for example?
 Is AI its own place of outcasts? Was that the feeling?
 There was a kind of outcast period for AI. For instance, in 1974,
 the new field was hardly 10 years old. The new field of computer science was asked by the
 National Science Foundation, I believe, but it may have been the National Academies, I can't remember,
 to tell your fellow scientists where computer science is and what it means. And they wanted
 to leave out AI. And they only agreed to put it in because Don Knuth said,
 hey, this is important. You can't just leave that out.
 Really? Don Knuth? Don Knuth, yes.
 I talked to him recently too. Out of all the people.
 Yes. But you see, an AI person couldn't have made that argument. He wouldn't have been believed,
 but Knuth was believed. Yes.
 So Joe Traub worked on the real stuff. Joe was working on algorithmic complexity,
 but he would say in plain English again and again, the smartest people I know are in AI.
 Really? Oh, yes. No question. Anyway, Joe loved these guys. What happened was that,
 I guess it was as I started to write Machines Who Think, Herb Simon and I both wrote it.
 Herb Simon and I became very close friends. He would walk past our house on Northumberland Street
 every day after work. And I would just be putting my cover on my typewriter.
 And I would lean out the door and say, Herb, would you like a sherry?
 And Herb almost always would like a sherry. So he'd stop in and we'd talk for an hour,
 two hours. My journal says we talked this afternoon for three hours.
 What was on his mind at the time in terms of on the AI side of things?
 Oh, we didn't talk too much about AI. We talked about other things.
 Just life. We both love literature.
 And Herb had read Proust in the original French twice all the way through. I can't, I've read it
 in English in translation. So we talked about literature. We talked about languages. We talked
 about music because he loved music. We talked about art because he was actually enough of a painter
 that he had to give it up because he was afraid it was interfering with his research
 and so on. So no, it was really just chat chat, but it was very warm. So one summer,
 I said to Herb, you know, my students have all the really interesting conversations.
 I was teaching at the University of Pittsburgh then in the English department.
 They get to talk about the meaning of life and that kind of thing. And what do I have? I have
 university meetings where we talk about the photocopying budget and whether the
 course on romantic poetry should be one semester or two. So Herb laughed. He said,
 yes, I know what you mean. He said, but you could do something about that.
 Dot, that was his wife, Dot and I used to have a salon at the University of Chicago every Sunday
 night. And we would have essentially an open house and people knew it wasn't for a small talk. It was
 really for some topic of depth. He said, but my advice would be that you choose the topic ahead
 of time. Fine. I said, so the following, we exchanged mail over the summer. That was US post
 in those days because you didn't have personal email. And I decided I would organize it and there
 would be eight of us, Alan Null and his wife, Herb Simon and his wife, Dorothea. There was a novelist
 in town, a man named Mark Harris. He had just arrived and his wife, Josephine. Mark was most
 famous then for a novel called Bang the Drum Slowly, which was about baseball and Joe and me,
 so eight people. And we met monthly and we just sank our teeth into really hard topics. And it
 was great fun. How have your own views around artificial intelligence changed through the
 process of writing Machines Who Think and afterwards the ripple effects? I was a little
 skeptical that this whole thing would work out. It didn't matter to me. It was so audacious.
 This whole thing being AI. AI generally, yeah. And in some ways, it hasn't worked out the way
 I expected so far. That is to say there's this wonderful lot of apps thanks to deep learning
 and so on, but those are algorithmic. And in the part of symbolic processing, there is very little
 yet. And that's a field that lies waiting for industrious graduate students. Maybe you can
 tell me some figures that popped up in your life in the 80s with expert systems where there was
 the symbolic AI possibilities of what most people think of as AI. If you dream of the possibilities,
 AI is really expert systems. And those hit a few walls and there were challenges there.
 And I think, yes, they will reemerge again with some new breakthroughs and so on. But what did
 that feel like, both the possibility and the winter that followed, the slowdown and research?
 Ah, you know, this whole thing about AI winter is to me a crock.
 It's no winters.
 Because I look at the basic research that was being done in the 80s, which is supposed to be,
 my God, it was really important. It was laying down things that nobody had thought about before,
 but it was basic research. You couldn't monetize it, hence the winter.
 Hence the winter.
 You know, research, scientific research goes and fits and starts. It isn't this nice, smooth,
 oh, this follows this, follows this. No, it just doesn't work that way.
 The interesting thing, the way winters happen, it's never the fault of the researchers.
 It's the some source of hype over promising. Well, no, let me take that back. Sometimes it
 is the fault of the researchers. Sometimes certain researchers might over promise the
 possibilities. They themselves believe that we're just a few years away, sort of just recently talked
 to Elon Musk and he believes he'll have an autonomous vehicle. We'll have autonomous
 vehicles in a year. And he believes it.
 A year?
 A year, yeah. We'll have mass deployment of our time.
 For the record, this is 2019 right now. So he's talking 2020.
 To do the impossible, you really have to believe it. And I think what's going to happen when you
 believe it, because there's a lot of really brilliant people around him, is some good stuff
 will come out of it. Some unexpected, brilliant breakthroughs will come out of it when you really
 believe it, when you work that hard.
 Yeah, I believe that. And I believe autonomous vehicles will come. I just don't believe it'll
 be in a year. I wish.
 But nevertheless, autonomous vehicles is a good example. There's a feeling many companies have
 promised by 2021, by 2022, Ford, GM, basically every single automotive company has promised
 they'll have autonomous vehicles. So that kind of over promise is what leads to the winter.
 Because we'll come to those dates. There won't be autonomous vehicles. And there'll be a feeling,
 well, wait a minute, if we took your word at that time, that means we just spent billions of dollars,
 had made no money, and there's a counter response to where everybody gives up on it.
 Sort of intellectually, at every level, the hope just dies. And all that's left is a few basic
 researchers. So you're uncomfortable with some aspects of this idea.
 Well, it's the difference between science and commerce.
 So you think science goes on the way it does?
 Science can really be killed by not getting proper funding or timely funding. I think
 Great Britain was a perfect example of that. The Lighthill report in,
 I don't remember the year, essentially said, there's no use Great Britain putting any money
 into this. It's going nowhere. And this was all about social factions in Great Britain.
 Edinburgh hated Cambridge and Cambridge hated Manchester. Somebody else can write that story.
 But it really did have a hard effect on research there. Now, they've come roaring back with deep
 mind, but that's one guy and his visionaries around him.
 But just to push on that, it's kind of interesting. You have this dislike of the idea of an AI winter.
 Where's that coming from? Where were you? Oh, because I just don't think it's true.
 There was a particular periods of time. It's a romantic notion, certainly.
 Yeah, well, I admire science, perhaps more than I admire commerce. Commerce is fine. Hey, you know,
 we all got to live. But science has a much longer view than commerce and continues almost regardless.
 It can't continue totally regardless, but it's almost regardless of what's saleable and what's
 not, what's monetizable and what's not. So the winter is just something that happens
 on the commerce side and the science marches. That's a beautifully optimistic, inspiring message.
 I agree with you. I think if we look at the key people that work in AI, they work in key scientists,
 most disciplines, they continue working out of the box. They continue working out of the box.
 They work in key scientists, most disciplines, they continue working out of the love for science.
 No matter the, you can always scrape up some funding to stay alive and they continue working
 diligently, but there certainly is a huge amount of funding now and there's a concern on the AI side
 and deep learning. There's a concern that we might with over-promising hit another slowdown in funding,
 which does affect the number of students, that kind of thing.
 Yeah, it does.
 So the kind of ideas you had in Michigan to think, did you continue that curiosity through
 the decades that followed?
 Yes, I did.
 What was your view, historical view of how AI community evolved, the conversations about it,
 the work? Has it persisted the same way from its birth?
 No, of course not. It's just, as we were just talking, the symbolic AI really kind of dried up
 and it all became algorithmic. I remember a young AI student telling me what he was doing
 and I had been away from the field long enough. I'd gotten involved with complexity at the Santa
 Fe Institute. I thought, algorithms, yeah, they're in the service of, but they're not the main event.
 No, they became the main event. That surprised me.
 And we all know the downside of this. We all know that if you're using an algorithm to make decisions
 based on a gazillion human decisions, baked into it are all the mistakes that humans make, the bigotries,
 the short-sightedness, so on and so on.
 So you mentioned Santa Fe Institute. So you've written the novel Edge of Chaos,
 but it's inspired by the ideas of complexity, a lot of which have been extensively explored
 at the Santa Fe Institute. It's another fascinating topic of just sort of
 emergent complexity from chaos. Nobody knows how it happens really, but it seems to wear all the
 interesting stuff that does happen. So how did first, not your novel, but just complexity in
 general in the work at Santa Fe fit into the bigger puzzle of the history of AI, or maybe even
 your personal journey through that? One of the last projects I did
 concerning AI in particular was looking at the work of Harold Cohen, the painter. And Harold was
 deeply involved with AI. He was a painter first, and what his project, Aran, which was a lifelong
 project, did was reflect his own cognitive processes. Harold and I, even though I wrote
 a book about it, we had a lot of friction between us. And I went, I thought, this is it, the book
 died, it was published and fell into a ditch. This is it, I'm finished, it's time for me to do
 something different. By chance, this was a sabbatical year for my husband, and we spent two months at
 the Santa Fe Institute and two months at Caltech, and then the spring semester in Munich, Germany.
 Okay, those two months at the Santa Fe Institute were so restorative for me.
 The institute was very small then. It was in some kind of office complex on old Santa Fe Trail.
 Everybody kept their door open. So you could crack your head on a problem, and if you finally
 didn't get it, you could walk in to see Stuart Kaufman or any number of people and say,
 I don't get this, can you explain? And one of the people that I was talking to about complex
 adaptive systems was Murray Gell-Mann. And I told Murray what Harold Cohen had done. And I said,
 you know, this sounds to me like a complex adaptive system. And he said, yeah, it is.
 Well, what do you know? Harold's errand had all these kissin' cousins all over the world.
 In science and in economics and so on and so forth. I was so relieved. I thought, okay,
 your instincts are okay. You're doing the right thing. I didn't have the vocabulary. And that
 was one of the things that the Santa Fe Institute gave me. If I could have rewritten that book,
 no, it had just come out. I couldn't rewrite it. I would have had a vocabulary to explain
 what Aaron was doing. Okay. So I got really interested in what was going on at the Institute.
 The people were, again, bright and funny and willing to explain anything to this amateur.
 George Cowan, who was then the head of the Institute, said he thought it might be a nice
 idea if I wrote a book about the Institute. And I thought about it. And I had a book called
 The
 God Knows What. And I said, I'm sorry, George. Yeah, I'd really love to do it, but, you know,
 just not going to work for me at this moment. And he said, oh, too bad. I think it would make an
 interesting book. Well, he was right and I was wrong. I wish I'd done it. But that's interesting.
 I hadn't thought about that. That that was a road not taken that I wish I'd taken.
 Well, you know what? That's just on that point. It's quite brave for you as a writer, as sort
 of coming from a world of literature, the literary thinking and historical thinking. I mean,
 just from that world and bravely talking to quite, I assume, large egos in AI or in complexity and
 so on. How'd you do it? Like, where did you? I mean, I suppose they could be intimidated of
 you as well, because it's two different worlds. I don't think I never picked up that anybody was
 intimidated by me. But how were you brave enough? Where did you find the guts? God, just dumb,
 dumb luck. I mean, this is an interesting rock to turn over. I'm going to write a book about it.
 And, you know, people have enough patience with writers if they think they're going to
 end up in a book that they let you flail around and so on.
 Well, but they also look if the writer has, there's like, if there's a sparkle in their eye,
 if they get it.
 Yeah, sure.
 Right. When were you at the Santa Fe Institute?
 The time I'm talking about is 1990, yeah, 1990, 1991, 92. But we then, because Joe was an external
 faculty member, we're in Santa Fe every summer. We bought a house there and I didn't have that
 much to do with the Institute anymore. I was writing my novels. I was doing whatever I was doing.
 But I loved the Institute and I loved the, again, the audacity of the ideas. That really appeals to
 me.
 I think that there's this feeling, much like in great, great institutes of neuroscience, for
 example, that they're in it for the long game of understanding something fundamental about reality
 and nature. And that's really exciting. So if we start to not to look a little bit more recently,
 how AI is really popular today. How is this world, you mentioned algorithmic, but in general,
 is the spirit of the people, the kind of conversations you hear through the gray virus on,
 is that different than the roots that you remember?
 No, the same kind of excitement, the same kind of, this is really going to make a difference in the
 world. And it will, it has.
 A lot of folks, especially young, 20 years old or something, they think we've just found something
 special here. We're going to change the world tomorrow. On a time scale, do you have a sense
 of what, of the time scale at which breakthroughs in AI happen?
 I really don't. I don't think so.
 I really don't. Because look at deep learning. That was a, Jeffrey Hinton
 came up with the algorithm in 86. But it took all these years for the technology to be good enough
 to actually be applicable. So no, I can't predict that at all. I can't, I wouldn't even try.
 Well, let me ask you to not to try to predict, but to speak to the, I'm sure in the 60s,
 as it continues now, there's people that think, let's call it, we can call it this
 fun word, the singularity. When there's a phase shift, there's some profound feeling where
 we're all really surprised by what's able to be achieved. I'm sure those dreams are there.
 I remember reading quotes in the 60s and those continued. How have your own views,
 maybe if you look back about the timeline of a singularity changed?
 Well, I'm not a big fan of the singularity as Ray Kurzweil has presented it.
 How would you define the Ray Kurzweil? How do you think of singularity in those?
 If I understand Kurzweil's view, it's sort of, there's going to be this moment when machines
 are smarter than humans and game over. However, the game over is, I mean, do they put us on a
 reservation? Do they, et cetera, et cetera. And first of all, machines are sort of,
 first of all, machines are smarter than humans in some ways all over the place.
 And they have been since adding machines were invented. So it's not going to come
 like some great eatable crossroads where they meet each other and our offspring, Oedipus says,
 you're dead. It's just not going to happen.
 Yeah. So it's already game over with calculators, right? They're already out to do much better at
 basic arithmetic than us. But there's a human-like intelligence and that's not the ones that destroy
 us, but somebody that you can have as a friend, you can have deep connections with that kind of
 passing the touring test and beyond those kinds of ideas. Have you dreamt of those?
 Oh, yes. Yes. Yes. Those possibilities. In a book I wrote with Ed Feigenbaum,
 there's a little story called The Geriatric Robot. And how I came up with The Geriatric Robot is a
 story in itself, but here's what The Geriatric Robot does. It doesn't just clean you up and feed
 you and wheel you out into the sun. It's great advantages. It listens. It says, tell me again
 about the great coup of 73. Tell me again about how awful or how wonderful your grandchildren are
 and so on and so forth. And it isn't hanging around to inherit your money. It isn't hanging
 around because it can't get any other job. This is its job and so on and so forth. Well,
 I would love something like that. Yeah. I mean, for me, that deeply excites me. So I think
 there's a lot of us- Like you gotta know, it was a joke. I dreamed it up because I needed to talk
 to college students and I needed to give them some idea of what AI might be. And they were rolling
 in the aisles as I elaborated and elaborated and elaborated. When it went into the book,
 they took my hide off in the New York Review of Books. This is just what we've thought about
 these people in AI. They're inhuman. Oh, come on, get over it. Don't you think that's a good thing
 for the world that AI could potentially- I do, absolutely. And furthermore, I'm pushing 80 now.
 By the time I need help like that, I also want it to roll itself in a corner and shut the fuck up.
 Let me linger on that point. Do you really though? Yeah, I do. Here's why. Don't you want it to push
 back a little bit? A little, but I have watched my friends go through the whole issue around
 having help in the house. And some of them have been very lucky and had fabulous help. And some
 of them have had people in the house who want to keep the television going on all day, who want to
 talk on their phones all day. No. So basically- Just roll yourself in the corner and shut up.
 Yeah. Unfortunately, us humans, when we're assistants, we still, even when we're assisting
 others, we care about ourselves more. Of course. And so you create more frustration. And a robot,
 AI assistant can really optimize the experience for you. I was just speaking to the point-
 you actually bring up a very, very good point. But I was speaking to the fact that
 us humans are a little complicated, that we don't necessarily want a perfect servant.
 I don't- maybe you disagree with that, but there's a- I think there's a push and pull
 with humans. A little tension, a little mystery that, of course, that's really difficult for AI
 to get right. But I do sense, especially in today with social media, that people are getting more
 and more lonely, even young folks. And sometimes especially young folks, that loneliness, there's a
 longing for connection and AI can help alleviate some of that loneliness. Some, just somebody who
 listens, like in person. So to speak. Yeah. So to speak. Yeah. That to me is really exciting. But
 so if we look at that level of intelligence, which is exceptionally difficult to achieve actually,
 as the singularity or whatever, that's the human level bar, that people have dreamt of that too.
 Turing dreamt of it. He had a date timeline. Do you have, how have your own timeline
 evolved on past? I don't even think about it. You don't even think? No. Just this field has been so
 full of surprises for me. You just take it in and see. Yeah. Oh, that's great. Yeah. I just can't.
 Maybe that's because I've been around the field long enough to think, you know, don't go that way.
 Herb Simon was terrible about making these predictions of when this and that would happen.
 And he was a sensible guy. His quotes are often used, right? As a legend. Yeah. Do you have
 concerns about AI, the existential threats of many people like Elon Musk and Sam Harris and others
 that are thinking about? Oh, yeah. That takes up half a chapter in my book. I call it the male gaze.
 Well, you hear me out. The male gaze is actually a term from film criticism.
 And I'm blocking on the women who dream this up, but she pointed out how most movies were made
 from the male point of view, that women were objects, not subjects. They didn't have any
 agency, so on and so forth. So when Elon and his pals Hawking and so on came, AI is going to eat
 our lunch and our dinner and our midnight snack too, I thought, what? And I said to Ed Feigenbaum,
 oh, these guys have always been the smartest guy on the block, and here comes something that might
 be smarter. Ooh, let's stamp it out before it takes over. And Ed laughed. He said, I didn't
 think about it that way. But I did. I did. And it is the male gaze. Okay, suppose these things do
 have agency. Well, let's wait and see what happens. Can we imbue them with ethics? Can we imbue them
 with a sense of empathy? Or are they just going to be, I don't know, we've had centuries of guys
 like that. That's interesting that the ego, the male gaze is immediately threatened. And so you
 can't think in a patient, calm way of how the tech could evolve. Speaking of which, you're a 96 book,
 The Future of Women. I think at the time and now, certainly now, I mean, I'm sorry, maybe at the
 time, but I'm more cognizant of now, is extremely relevant. You and Nancy Ramsey talk about four
 possible futures of women in science and tech. So if we look at the decades before and after the book
 was released, can you tell a history, sorry, of women in science and tech and how it has evolved?
 How have things changed? Where do we stand? Not enough. They have not changed enough. The
 way that women are ground down in computing is simply unbelievable. But what are the four possible
 futures for women in tech from the book? What you're really looking at are various
 aspects of the present. So for each of those, you could say, oh yeah, we do have backlash. Look at
 what's happening with abortion and so on and so forth. We have one step forward, one step back.
 The golden age of equality was the hardest chapter to write. And I used something from the Santa Fe
 Institute, which is the sandpile effect that you drop sand very slowly onto a pile and it grows
 and it grows and it grows until suddenly it just breaks apart. And in a way, me too has done that.
 That was the last drop of sand that broke everything apart. That was a perfect example
 of the sandpile effect. And that made me feel good. It didn't change all of society,
 but it really woke a lot of people up. But are you in general optimistic about
 maybe after me too? I mean, me too is about a very specific kind of thing.
 Boy, solve that and you solve everything. But are you in general optimistic about the future?
 Yes, I'm a congenital optimistic. I can't help it.
 What about AI? What are your thoughts about the future of AI?
 Of course I get asked what do you worry about? And the one thing I worry about is the things
 that we can't anticipate. There's going to be something out of left field that we will just say,
 we weren't prepared for that. I am generally optimistic. When I first took up being interested
 in AI, like most people in the field, more intelligence was like more virtue. What could be
 bad? And in a way, I still believe that, but I realized that my notion of intelligence
 has broadened. There are many kinds of intelligence and we need to imbue our machines with those many
 kinds. So you've now just finished or in the process of finishing the book that you've been
 working on, the memoir. How have you changed? I know it's just writing, but how have you changed
 the process? If you look back, what kind of stuff did it bring up to you that surprised you looking
 at the entirety of it all? The biggest thing, and it really wasn't a surprise, is how lucky I was.
 Oh my, to have access to the beginning of a scientific field that is going to change the world.
 How did I luck out? And yes, of course, my view of things has widened a lot.
 If I can get back to one feminist part of our conversation, without knowing it,
 it really was subconscious. I wanted AI to succeed because I was so tired of hearing that
 intelligence was inside the male cranium. And I thought if there was something out there
 that wasn't a male thinking and doing well, then that would put a lie to this whole notion of
 intelligence resides in the male cranium. I did not know that until one night Harold Cohen and I
 were having a glass of wine, maybe two, and he said, what drew you to AI? And I said, oh,
 you know, smartest people I knew, great project, blah, blah, blah. And I wanted something besides
 male smarts. And it just bubbled up out of me, Lex. And I, what? What's this?
 It's kind of brilliant, actually. So AI really humbles all of us and humbles the
 people that need to be humbled the most. Let's hope.
 Oh, wow. That is so beautiful. Pamela, thank you so much for talking to me.
 Oh, it's been a great pleasure. Thank you.