Dileep George: Brain-Inspired AI | Lex Fridman Podcast #115
dileep george artificial intelligence agi ai ai podcast artificial intelligence podcast lex fridman lex podcast lex mit lex ai lex jre mit ai
Dileep George is a researcher at the intersection of neuroscience and artificial intelligence, co-founder of Vicarious, formerly co-founder of Numenta. From the early work on Hierarchical temporal memory to Recursive Cortical Networks to today, Dileep's always sought to engineer intelligence that is closely inspired by the human brain.

Support this channel by supporting our sponsors. Click links, get discount:
- Babbel: https://babbel.com and use code LEX
- MasterClass: https://masterclass.com/lex
- Raycon: https://buyraycon.com/lex

EPISODE LINKS:
Dileep's Twitter: https://twitter.com/dileeplearning
Vicarious Research: https://www.vicarious.com/science

PODCAST INFO:
Podcast website:
https://lexfridman.com/podcast
Apple Podcasts:
https://apple.co/2lwqZIr
Spotify:
https://spoti.fi/2nEwCF8
RSS:
https://lexfridman.com/feed/podcast/
Full episodes playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist:
https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
4:50 - Building a model of the brain
17:11 - Visual cortex
27:50 - Probabilistic graphical models
31:35 - Encoding information in the brain
36:56 - Recursive Cortical Network
51:09 - Solving CAPTCHAs algorithmically
1:06:48 - Hype around brain-inspired AI
1:18:21 - How does the brain learn?
1:21:32 - Perception and cognition
1:25:43 - Open problems in brain-inspired AI
1:30:33 - GPT-3
1:40:41 - Memory
1:45:08 - Neuralink
1:51:32 - Consciousness
1:57:59 - Book recommendations
2:06:49 - Meaning of life

CONNECT:
- Subscribe to this YouTube channel
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/LexFridmanPage
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman
 The following is a conversation with Dalip George, a researcher at the intersection of neuroscience and artificial intelligence, co-founder of Vicarious with Scott Phoenix, and formerly co-founder of Numenta with Jeff Hawkins, who's been on this podcast, and Donna Dubinsky. From his early work on hierarchical temporal memory to recursive cortical networks to today, the Leaps always sought to engineer intelligence that is closely inspired by the human brain. As a side note, I think we understand very little about the fundamental principles underlying the function of the human brain, but the little we do know gives hints that may be more useful for engineering intelligence than any idea in mathematics, computer science, physics, and scientific fields outside of biology. And so the brain is a kind of existence proof that says it's possible. Keep at it. I should also say that brain-inspired AI is often overhyped, and use this fodder, just this quantum computing, for a marketing speak. But I'm not afraid of exploring these sometimes overhyped areas since where there's smoke, there's sometimes fire. Quick summary of the ads. Three sponsors, Babbel, Raycon Earbuds, and Masterclass. Please consider supporting this podcast by clicking the special links in the description to get the discount. It really is the best way to support this podcast. If you enjoy this thing, subscribe on YouTube, review the Five Stars on Apple Podcasts, support on Patreon, or connect with me on Twitter at Lex Friedman. As usual, I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation. This show is sponsored by Babbel, an app and website that gets you speaking in a new language within weeks. Go to babbel.com and use code Lex to get three months free. They offer 14 languages, including Spanish, French, Italian, German, and yes, Russian. Daily lessons are 10 to 15 minutes, super easy, effective, designed by over 100 language experts. Let me read a few lines from the Russian poem, Noch, Ulitsa, Fanar, Abteka, by Alexander Bloch, that you'll start to understand if you sign up to Babbel. Now, I say that you'll only start to understand this poem because Russian starts with the language and ends with the vodka. Now, the latter part is definitely not endorsed or provided by Babbel and will probably lose me the sponsorship. But once you graduate from Babbel, you can enroll my advanced course of late night Russian conversation over vodka. I have not yet developed an app for that. It's in progress. So get started by visiting babbel.com and use code Lex to get three months free. This show is sponsored by Raycon Earbuds. Get them at buyraycon.com slash Lex. They become my main method of listening to podcasts, audiobooks, and music when I run, do pushups and pull ups, or just living life. In fact, I often listen to brown noise with them when I'm thinking deeply about something, it helps me focus. They're super comfortable, pair easily, great sound, great bass, six hours of playtime. I've been putting in a lot of miles to get ready for a potential ultra marathon and listening to audio books on World War II. The sound is rich and really comes in clear. So again, get them at buyraycon.com slash Lex. This show is sponsored by Masterclass. Sign up at masterclass.com slash Lex to get a discount and to support this podcast. When I first heard about Masterclass, I thought it was too good to be true. I still think it's too good to be true. For 180 bucks a year, you get an all access pass to watch courses from to list some of my favorites. Chris Hatfield on space exploration, Neil deGrasse Tyson on scientific thinking and communication, Will Wright, creator of SimCity and Sims and game design. Every time I do this read, I really want to play a city builder game. Carlos Santana on guitar, Garry Kasparov on chess, Daniel Legano on poker and many more. Chris Hatfield explaining how rockets work and the experience of being launched into space alone is worth the money. By the way, you can watch it on basically any device. Once again, sign up at masterclass.com to get a discount and to support this podcast. And now here's my conversation with Daleep George. Do you think we need to understand the brain in order to build it? Yes. If you want to build the brain, we definitely need to understand how it works. So Blue Brain or Henry Markram's project is trying to build a brain without understanding it, just trying to put details of the brain from neuroscience experiments into a giant simulation by putting more and more neurons, more and more details. But that is not going to work because when it doesn't perform as what you expect it to do, then what do you do? You just keep adding more details. How do you debug it? So unless you understand, unless you have a theory about how the system is supposed to work, how the pieces are supposed to fit together, what they're going to contribute, you can't build it. At the functional level, understand. So can you actually linger on and describe the Blue Brain project? It's kind of a fascinating principle and idea to try to simulate the brain. We're talking about the human brain, right? Right. Human brains and rat brains or cat brains have lots in common that the cortex, the neocortex structure is very similar. So initially they were trying to just simulate a cat brain. To understand the nature of evil. To understand the nature of evil. Or as it happens in most of these simulations, you easily get one thing out, which is oscillations. Yeah, if you simulate a large number of neurons, they oscillate. And you can adjust the parameters and say that, oh, oscillations match the rhythm that we see in the brain, et cetera. Oh, I see. So the idea is, is the simulation at the level of individual neurons? Yeah. So the Blue Brain project, the original idea as proposed was, you put very detailed biophysical models of neurons and you interconnect them according to the statistics of connections that we have found from real neuroscience experiments. And then turn it on and see what happens. And these neural models are incredibly complicated in themselves, right? Because these neurons are modeled using this idea called Hodgkin-Huxley models, which are about how signals propagate in a cable. And there are active dendrites, all those phenomena, which those phenomena themselves, we don't understand that well. And then we put in connectivity, which is part guesswork, part observed. And of course, if you do not have any theory about how it is supposed to work, we just have to take whatever comes out of it as, okay, this is something interesting. But in your sense, like these models of the way signal travels along, like with the axons and all the basic models, they're too crude. Oh, well, actually, they are pretty detailed and pretty sophisticated. And they do replicate the neural dynamics. If you take a single neuron and you try to turn on the different channels, the calcium channels and the different receptors, and see what the effect of turning off those channels are in the neurons spike output, people have built pretty sophisticated models of that. And they are, I would say, in the regime of correct. Well, see, the correctness, that's interesting, because you mentioned it at several levels. The correctness is measured by looking at some kind of aggregate statistics? It would be more the spiking dynamics of a single neuron. Spiking dynamics of a single neuron, okay. And yeah, these models, because they are going to the level of mechanism, right? So they are basically looking at, okay, what is the effect of turning on an ion channel? And you can model that using electric circuits. So it is not just a function fitting. People are looking at the mechanism underlying it and putting that in terms of electric circuit theory, signal propagation theory, and modeling that. And so those models are sophisticated, but getting a single neuron model 99% right does not still tell you how to... It would be the analog of getting a transistor model right and now trying to build a microprocessor. And if you did not understand how a microprocessor works, but you say, oh, I now can model one transistor well, and now I will just try to interconnect the transistors according to whatever I could guess from the experiments and try to simulate it, then it is very unlikely that you will produce a functioning microprocessor. When you want to produce a functioning microprocessor, you want to understand Boolean logic, how do the gates work, all those things, and then understand how do those gates get implemented using transistors. Yeah. There is actually... This reminds me, this is a paper, maybe you are familiar with it, that I remember going through in a reading group that approaches a microprocessor from a perspective of a neuroscientist. I think it basically uses all the tools that we have of neuroscience to try to understand, like as if we just aliens showed up to study computers and to see if those tools could be used to get any kind of sense of how the microprocessor works. I think the final takeaway from at least this initial exploration is that we are screwed. There is no way that the tools of neuroscience would be able to get us to anything, like not even Boolean logic. I mean, it is just any aspect of the architecture of the function of the processes involved, the clocks, the timing, all that, you can't figure that out from the tools of neuroscience. Yes, I am very familiar with this particular paper. I think it was called, Can a Neuroscientist Understand a Microprocessor? Something like that. Following the methodology in that paper, even an electrical engineer would not understand microprocessors. I don't think it is that bad in the sense of saying, neuroscientists do find valuable things by observing the brain. They do find good insights, but those insights cannot be put together just as a simulation. You have to investigate what are the computational underpinnings of those findings. How do all of them fit together from an information processing perspective? Somebody has to painstakingly put those things together and build hypotheses. So, I don't want to have all of these neuroscientists saying, oh, they are not finding anything. No, that paper almost went to that level of neuroscientists will never understand. That's not true. I think they do find lots of useful things, but it has to be put together in a computational framework. The AI systems will be listening to this podcast a hundred years from now. There is some non-zero probability they will find your words laughable. I remember humans thought they understood something about the brain that are totally clueless. There is a sense about neuroscience that we may be in the very early days of understanding the brain. But, I mean, that's one perspective. In your perspective, how far are we into understanding any aspect of the brain? So, the dynamics of the individual neuron communication to how in a collective sense, how they are able to store information, transfer information, how the intelligence then emerges, all that kind of stuff. Where are we on that timeline? Yeah. So, timelines are very, very hard to predict. And you can, of course, be wrong. And you can be wrong on either side. We know that when we look back, the first flight was in 1903. In 1900, there was a New York Times article on flying machines that do not fly. And humans might not fly for another hundred years. That was what that article stated. But no, they flew three years after that. So, it's very hard to... Well, and on that point, one of the Wright brothers, I think two years before, said that, like he said, some number, like 50 years, he has become convinced that it's impossible. Even during their experimentation. Yeah. I mean, that's a tribute to when... That's like the entrepreneurial battle of depression, of going through just thinking, this is impossible. But there, yeah, there's something... Even the person that's in it is not able to see, estimate correctly. Exactly. But I can tell from the point of objectively what are the things that we know about the brain and how that can be used to build AI models, which can then go back and inform how the brain works. So, my way of understanding the brain would be to basically say, look at the insights neuroscientists have found, understand that from a computational angle, information processing angle, build models using that. And then building that model, which is a functional model, which is doing the task that we want the model to do. It is not just trying to model a phenomena in the brain. It is trying to do what the brain is trying to do on the whole functional level. And building that model will help you fill in the missing pieces that biology just gives you the hints. And building the model fills in the rest of the pieces of the puzzle. And then you can go and connect that back to biology and say, okay, now it makes sense that this part of the brain is doing this, or this layer in the cortical circuit is doing this. And then continue this iteratively, because now that will inform new experiments in neuroscience. And of course, building the model and verifying that in the real world will also tell you more about does the model actually work? And you can refine the model, find better ways of putting these neuroscience insights together. So, I would say it is... Neuroscientists alone, just from experimentation, will not be able to build a functional model of the brain. So, there's lots of efforts, which are very impressive efforts in collecting more and more connectivity data from the brain. How are the microcircuits of the brain connected with each other? Those are beautiful, by the way. Those are beautiful. And at the same time, those do not itself, by themselves, convey the story of how does it work? And somebody has to understand, okay, why are they connected like that? And what are those things doing? And we do that by building models in AI, using hints from neuroscience, and repeat the cycle. So, what aspect of the brain are useful in this whole endeavor? Which, by the way, I should say, you're both a neuroscientist and an AI person. I guess the dream is to both understand the brain and to build AGI systems. So, it's like an engineer's perspective of trying to understand the brain. So, what aspects of the brain functionally speaking, like you said, do you find interesting? Yeah, quite a lot of things. All right. So, one is, if you look at the visual cortex, and the visual cortex is a large part of the brain, I forget the exact fraction, but a huge part of our brain area is occupied by just vision. So, visual cortex is not just a feed-forward cascade of neurons. There are a lot more feedback connections in the brain compared to the feed-forward connections. And it is surprising to the level of detail neuroscientists have actually studied this. If you go into neuroscience literature and poke around and ask, have they studied what will be the effect of poking a neuron in level IT in level V1? And have they studied that? And you will say, yes, they have studied that. So, every possible combination. I mean, it's not a random exploration at all. It's very hypothesis-driven. Experimental neuroscientists are very, very systematic in how they probe the brain because experiments are very costly to conduct. They take a lot of preparation. They need a lot of control. So, they are very hypothesis-driven in how they probe the brain. And often what I find is that when we have a question in AI about, has anybody probed how lateral connections in the brain works? And when you go and read the literature, yes, people have probed it and people have probed it very systematically. And they have hypotheses about how those lateral connections are supposedly contributing to visual processing. But of course, they haven't built very, very functional detailed models of it. By the way, how do the studies, sorry to interrupt, do they stimulate like a neuron in one particular area of the visual cortex and then see how the signal travels kind of thing? Very, very fascinating experiments. So, I can give you one example I was impressed with. So, before going to that, let me give you overview of how the layers in the cortex are organized. Visual cortex is organized into roughly four hierarchical levels. So, V1, V2, V4, IT. And in V1- What happened to V3? Well, yeah, there's another pathway. Okay. So, I'm talking about just object recognition pathway. All right, cool. And then in V1 itself, there is a very detailed micro circuit in V1 itself. There is organization within a level itself. The cortical sheet is organized into multiple layers and there are columnar structure. And this layer-wise and columnar structure is repeated in V1, V2, V4, IT, all of them. And the connections between these layers within a level, in V1 itself, there are six layers roughly, and the connections between them, there is a particular structure to them. And now, so one example of an experiment people did is when you present a stimulus, which is, let's say, requires separating the foreground from the background of an object. So, it's a textured triangle on a textured background. And you can check, does the surface settle first, or does the contour settle first? Settle? Settle in the sense that when you finally form the percept of the triangle, you understand where the contours of the triangle are, and you also know where the inside of the triangle is, right? That's when you form the final percept. Now, you can ask, what is the dynamics of forming that final percept? Do the neurons first find the edges and converge on where the edges are, and then they find the inner surfaces, or does it go the other way around? The other way around. So, what's the answer? In this case, it turns out that it first settles on the edges, it converges on the edge hypothesis first, and then the surfaces are filled in from the edges to the inside. That's fantastic. And the detail to which you can study this, it's amazing that you can actually not only find the temporal dynamics of when this happens, and then you can also find which layer in V1, which layer is encoding the edges, which layer is encoding the surfaces, and which layer is encoding the feedback, which layer is encoding the feedforward, and what's the combination of them that produces the final person. And these kinds of experiments stand out when you try to explain illusions. One example of a favorite illusion of mine is the Kinesa triangle. I don't know that you are familiar with this one. So, this is an example where it's a triangle, but only the corners of the triangle are shown in the stimulus. So, they look like kind of Pac-Man. Oh, the black Pac-Man. Yeah. And then you start to see. Your visual system hallucinates the edges. And when you look at it, you will see a faint edge. And you can go inside the brain and look, do actually neurons signal the presence of this edge? And if they signal, how do they do it? Because they are not receiving anything from the input. The input is black for those neurons. So, how do they signal it? When does the signaling happen? So, if a real contour is present in the input, then the neurons immediately signal, oh, okay, there is an edge here. When it is an illusory edge, it is clearly not in the input. It is coming from the context. So, those neurons fire later. And you can say that, okay, it's the feedback connections that is causing them to fire. And they happen later. And you can find the dynamics of them. So, these studies are pretty impressive and very detailed. So, by the way, just to step back, you said that there may be more feedback connections than feedforward connections. Yeah. First of all, just for like a machine learning folks, I mean, that's crazy that there's all these feedback connections. We often think about, thanks to deep learning, you start to think about the human brain as a kind of feedforward mechanism. So, what the heck are these feedback connections? Yeah. What's the dynamics? What are we supposed to think about them? Yeah. So, this fits into a very beautiful picture about how the brain works. So, the beautiful picture of how the brain works is that our brain is building a model of the world. So, our visual system is building a model of how objects behave in the world. And we are constantly projecting that model back onto the world. So, what we are seeing is not just a feedforward thing that just gets interpreted in a feedforward part. We are constantly projecting our expectations onto the world. And what the final percept is a combination of what we project onto the world combined with what the actual sensory input is. Almost like trying to calculate the difference and then trying to interpret the difference. Yeah. I wouldn't put it as calculating the difference. It's more like what is the best explanation for the input stimulus based on the model of the world I have? Got it. Got it. And that's where all the illusions come in. But that's an incredibly efficient process. So, the feedback mechanism, it just helps you constantly, yeah. So, hallucinate how the world should be based on your world model and then just looking at if there's novelty, like trying to explain it. Yeah. Hence, that's why movement, we detect movement really well. There's all these kinds of things. And this is like at all different levels of the cortex you're saying. Every. This happens at the lowest level, at the highest level. Exactly. Yes. Yeah. In fact, feedback connections are more prevalent everywhere in the cortex. And so, one way to think about it, and there's a lot of evidence for this, is inference. So, basically, if you have a model of the world and when some evidence comes in, what you are doing is inference. You are trying to now explain this evidence using your model of the world. And this inference includes projecting your model onto the evidence and taking the evidence back into the model and doing an iterative procedure. And this iterative procedure is what happens using the feed-forward feedback propagation. And feedback affects what you see in the world and it also affects feed-forward propagation. And examples are everywhere. We see these kinds of things everywhere. The idea that there can be multiple competing hypotheses in our model trying to explain the same evidence. And then you have to kind of make them compete. And one hypothesis will explain away the other hypothesis through this competition process. So, you have competing models of the world that try to explain. What do you mean by explain away? So, this is a classic example in graphical models, probabilistic models. What are those? I think it's useful to mention it because we'll talk about them more. Yeah. So, neural networks are one class of machine learning models. You have a distributed set of nodes which are called the neurons. Each one is doing a dot product and you can approximate any function using this multi-level network of neurons. So, that's a class of models which are useful for function approximation. There is another class of models in machine learning called probabilistic graphical models. And you can think of them as each node in that model is variable, which is talking about something. It can be a variable representing is an edge present in the input or not. And at the top of the network, a node can be representing is there an object present in the world or not. And then, so it is another way of encoding knowledge. And then, once you encode the knowledge, you can do inference in the right way. What is the best way to explain some set of evidence using this model that you encoded? So, when you encode the model, you are encoding the relationship between these different variables. How is the edge connected to the model of the object? How is the surface connected to the model of the object? And then, of course, this is a very distributed complicated model. And inference is, how do you explain a piece of evidence when a set of stimulus comes in? If somebody tells me there is a 50% probability that there is an edge here in this part of the model, how does that affect my belief on whether I should think that there should be a square present in the image? So, this is the process of inference. So, one example of inference is having this experience of effect between multiple causes. So, graphical models can be used to represent causality in the world. So, let's say your alarm at home can be triggered by a burglar getting into your house, or it can be triggered by an earthquake. Both can be causes of the alarm going off. So, now you are in your office, you heard burglar alarm going off, you are heading home, thinking that there's a burglar. But while driving home, if you hear on the radio that there was an earthquake in the vicinity, now your strength of evidence for a burglar getting into their house is diminished. Because now that piece of evidence is explained by the earthquake being present. So, if you think about these two causes explaining at lower level variable, which is alarm, now what we are seeing is that increasing the evidence for some cause, there is evidence coming in from below for alarm being present. And initially it was flowing to a burglar being present. But now, since there is side evidence for this other cause, it explains away this evidence and evidence will now flow to the other cause. These are two competing causal things trying to explain the same evidence. And the brain has a similar kind of mechanism for doing so. That's kind of interesting. How is that all encoded in the brain? Where is the storage of information? Are we talking just maybe to get it a little bit more specific? Is it in the hardware of the actual connections? Is it in chemical communication? Is it electrical communication? Do we know? So, this is a paper that we are bringing out soon. Which one is this? This is the cortical microcircuits paper that I sent you a draft of. Of course, a lot of it is still hypothesis. One hypothesis is that you can think of a cortical column as encoding a concept. Think of it as an example of a concept. Is an edge present or not? Or is an object present or not? So, you can think of it as a binary variable, a binary random variable. The presence of an edge or not or the presence of an object or not. So, each cortical column can be thought of as representing that one concept, one variable. And then the connections between these cortical columns are basically encoding the relationship between these random variables. And then there are connections within the cortical column. Each cortical column is implemented using multiple layers of neurons with very, very rich structure there. There are thousands of neurons in a cortical column. But that structure is similar across the different cortical columns. Correct. And also these cortical columns connect to a substructure called thalamus in the structure. So, all cortical columns pass through this substructure. So, our hypothesis is that the connections between the cortical columns implement this. That's where the knowledge is stored about how these different concepts connect to each other. And then the neurons inside this cortical column and in thalamus in combination implement these actual computations needed for inference, which includes explaining away and competing between the different hypothesis. So, what is amazing is that neuroscientists have actually done experiments to the tune of showing these things. They might not be putting it in the overall inference framework, but they will show things like if I poke this higher level neuron, it will inhibit through this complicated loop through the thalamus, it will inhibit this other column. So, they will do such experiments. Do they use terminology of concepts, for example? Is it something where it's easy to anthropomorphize and think about concepts like you start moving into logic-based reasoning systems? So, are we just thinking of concepts in that kind of way? Or is it a lot messier, a lot more gray area, even more gray, even more messy than the artificial neural network kinds, kinds of abstractions? It's the easiest way to think of it as a variable. It's a binary variable, which is showing the presence or absence of something. But I guess what I'm asking is, is that something that we're supposed to think of something that's human interpretable, that something? It doesn't need to be. It doesn't need to be human interpretable. There's no need for it to be human interpretable. But it's almost like you will be able to find some interpretation of it because it is connected to the other things that you know about. And the point is it's useful somehow. It's useful as an entity in the graph, in connecting to the other entities that are, let's call them concepts. By the way, are these the cortical microcircuits? Correct. These are the cortical microcircuits. That's what neuroscientists use to talk about the circuits within a level of the cortex. So you can think of, let's think of a neural network, artificial neural network terms. People talk about the architecture of the, how many layers they build, what is the fan in, fan out, etc. That is the macro architecture. And then within a layer of the neural network, the cortical neural network is much more structured within a level. There's a lot more intricate structure there. But even within an artificial neural network, you can think of feature detection plus pooling as one level. And so that is kind of a microcircuit. It's much more complex in the real brain. And so within a level, whatever is that circuitry within a column of the cortex and between the layers of the cortex, that's the microcircuitry. I love that terminology. Machine learning people don't use the circuit terminology, but they should. That's nice. So okay. So that's the cortical microcircuit. So what's interesting about, what can we say, what is the paper that you're working on, propose about the ideas around these cortical microcircuits? So this is a fully functional model for the microcircuits of the visual cortex. So the paper focuses on your idea and our discussion now is focusing on vision, the visual cortex. Okay. This is a model. This is a full model. This is how vision works. But this is a model of- A hypothesis. Okay. So let me step back a bit. So we looked at neuroscience for insights on how to build a vision model. And we synthesized all those insights into a computational model. This is called the recursive cortical network model that we used for breaking captures. And we are using the same model for robotic picking and tracking of objects. And that again is a vision system. That's a vision system. Computer vision system. That's a computer vision system. Takes in images and outputs what? On one side, it outputs the class of the image and also segments the image. And you can also ask it further queries. Where is the edge of the object? Where is the interior of the object? So it's a model that you build to answer multiple questions. So you're not trying to build a model for just classification or just segmentation, et cetera. It's a joint model that can do multiple things. So that's the model that we built using insights from neuroscience. And some of those insights are what is the role of feedback connections? What is the role of lateral connections? So all those things went into the model. The model actually uses feedback connections. All these ideas from your science. So what the heck is a recursive cortical network? Like what are the architecture approaches, interesting aspects here, which is essentially a brain-inspired approach to a computer vision? Yeah. So there are multiple layers to this question. I can go from the very top and then zoom in. Okay. So one important thing, constraint that went into the model is that you should not think of vision as something in isolation. We should not think perception as something as a pre-processor for cognition. Perception and cognition are interconnected. And so you should not think of one problem in separation from the other problem. And so that means if you finally want to have a system that understands concepts about the world and can learn in a very conceptual model of the world and can reason and connect to language, all of those things, you need to think all the way through and make sure that your perception system is compatible with your cognition system and language system and all of them. And one aspect of that is top-down controllability. What does that mean? So that means... In this context. So think of... You can close your eyes and think about the details of one object. I can zoom in further and further. So think of the bottle in front of me. And now you can think about, okay, what the cap of that bottle looks. We can think about what's the texture on that bottle of the cap. You can think about what will happen if something hits that. So you can manipulate your visual knowledge in cognition-driven ways. And so this top-down controllability and being able to simulate scenarios in the world. So you're not just a passive player in this perception game. You can control it. You have imagination. Correct. So basically having a generative network, which is a model, and it is not just some arbitrary generated network. It has to be built in a way that it is controllable top-down. It is not just trying to generate a whole picture at once. It's not trying to generate photorealistic things of the world. You don't have good photorealistic models of the world. Human brains do not have. If I, for example, ask you the question, what is the color of the letter E in the Google logo? You have no idea. Although you have seen it millions of times, hundreds of times. Our model is not photorealistic, but it has other properties that we can manipulate it. You can think about filling in a different color in that logo. You can think about expanding the letter E. You can imagine the consequence of actions that you have never performed. These are the kind of characteristics the generative model needs to have. This is one constraint that went into our model. When you read just the perception side of the paper, it is not obvious that this was a constraint that went into the model, this top-down controllability of the generative model. What does top-down controllability in a model look like? It's a really interesting concept, fascinating concept. Is that recursive? Quite a few things. It's like what does the model factorize? What is the model representing as different pieces in the puzzle? In the RCN network, it thinks of the world. For example, the background of an image is modeled separately from the foreground of the image. The objects are separate from the background. They are different entities. There's a kind of segmentation that's built in fundamentally. Then even that object is composed of parts. Another one is the shape of the object is differently modeled from the texture of the object. Got it. You know who Francois Chalet is? Yes. He developed this IQ test type of thing for Arc Challenge. It's kind of cool that there's these concepts, priors that he defines that you bring to the table in order to be able to reason about basic shapes and things in an IQ test. Here you're making it quite explicit that these are distinct things that you should be able to model in this. Yes. Keep in mind that you can derive these from much more general principles. You don't need to explicitly put it as objects versus foreground versus background, the surface versus texture. These are derivable from more fundamental principles of what's the property of continuity of natural signals. What's the property of continuity of natural signals? By the way, that sounds very poetic. You're saying there's some low-level properties from which emerges the idea that shapes should be different. There should be parts of an object. There should be objects. There's objectness. There's all these things that it's kind of crazy that we humans evolved to have because it's useful. Correct. It derives mostly from the properties of natural signals. Yeah. Natural signals are the kind of things we'll perceive in the natural world. Correct. I don't know why that sounds so beautiful. Natural signals, yeah. As opposed to a QR code, which is an artificial signal that we created, humans are not very good at classifying QR codes. We are very good at saying something is a cat or a dog, but not very good at classifying. Computers are very good at classifying QR codes. Our visual system is tuned for natural signals. There are fundamental assumptions in the architecture that are derived from natural signals properties. I wonder when you take hallucinogenic drugs, does that go into natural or is that closer to the QR code? It's still natural. It's still natural. Yeah, because it is still operating using our brains. By the way, on that topic, I haven't been following. I think they're becoming legalized. I can't wait until they become legalized to a degree that vision science researchers could study it. Just through medical chemical ways, modify it. There could be ethical concerns, but that's another way to study. The brain is to be able to chemically modify it. There's probably very long a way to figure out how to do it ethically. Yeah, but I think there are studies on that already. I think so, because it's not unethical to give it to rats. Oh, that's true. That's true. There's a lot of drugged up rats out there. Okay, cool. Sorry. Sorry. It's okay. There's these low level things from natural signals that... From which these properties will emerge. Yes. But it is still a very hard problem on how to encode that. You mentioned the priors Franchot wanted to encode in the abstract reasoning challenge, but it is not straightforward how to encode those priors. Some of those challenges, the object completion challenges are things that we purely use our visual system to do. It looks like abstract reasoning, but it is purely an output of the vision system. For example, completing the corners of that Kansar triangle, completing the lines of that Kansar triangle. It's a purely visual system property. There is no abstract reasoning involved. It uses all these priors, but it is stored in our visual system in a particular way that is amenable to inference. That is one of the things that we tackled. It's basically saying, these are the prior knowledge which will be derived from the world. But then how is that prior knowledge represented in the model such that inference, when some piece of evidence comes in, can be done very efficiently and in a very distributed way? There are so many ways of representing knowledge which is not amenable to very quick inference, quick lookups. That's one core part of what we tackled in the RCN model. How do you encode visual knowledge to do very quick inference? Can you maybe comment on folks listening to this and in general may be familiar with different kinds of architectures of neural networks. What are we talking about with RCN? What does the architecture look like? What are the different components? Is it close to neural networks? Is it far away from neural networks? What does it look like? You can think of the delta between the model and a convolutional neural network if people are familiar with convolutional neural networks. Convolutional neural networks have this feed-forward processing cascade which is called feature detectors and pooling. That is repeated in a multi-level system. If you want an intuitive idea of what is happening, feature detectors are detecting interesting co-occurrences in the input. It can be a line, a corner, an eye, or a piece of texture etc. The pooling neurons are doing some local transformation of that and making it invariant to local transformations. This is what the structure of convolutional neural network is. Recursive cortical network has a similar structure when you look at just the feed-forward pathway. But in addition to that, it is also structured in a way that it is generative so that it can run it backward and combine the forward with the backward. Another aspect that it has is it has lateral connections. These lateral connections, so if you have an edge here and an edge here, it has connections between these edges. It is not just feed-forward connections. It is something between these edges which is the nodes are presenting these edges which is to enforce compatibility between them. Otherwise, what will happen is that- Like constraints? It is a constraint. Basically, if you do just feature detection followed by pooling, then your transformations in different parts of the visual field are not coordinated. So, when you generate from the model, you will create jagged things and uncoordinated transformations. So, these lateral connections are enforcing the transformations. Is the whole thing still differentiable? No. It is not trained using backprop. Okay, that is really important. So, there is this feed-forward, there is feedback mechanisms, there are some interesting connectivity things. It is still layered? Yes, there are multiple layers. Okay, very interesting. The interconnection between adjacent connections across service constraints that keep the thing stable. Correct. Okay, so what else? Then there is this idea of doing inference. A neural network does not do inference on the fly. An example of why this inference is important is one of the first applications that we showed in the paper was to crack text-based captures. What are captures, by the way? By the way, one of the most awesome- People do not use this term anymore. It is human computation, I think. I love this term. The guy who created captures, I think, came up with this term. Yeah. I love it. Anyway, what are captures? So, captures are those strings that you fill in. If you are opening a new account in Google, they show you a picture. It used to be a set of garbled letters that you have to figure out what is that string of characters and type it. The reason captures exist is because Google or Twitter do not want automatic creation of accounts. You can use a computer to create millions of accounts and use that for nefarious purposes. So, you want to make sure that, to the extent possible, the interaction that their system is having is with a human. So, it's called a human interaction proof. A capture is a human interaction proof. So, this captures are, by design, things that are easy for humans to solve, but hard for computers. Hard for robots, yeah. Yeah. Text-based captures was the one which is prevalent around 2014, because at that time, text-based captures were hard for computers to crack. Even now, they are actually, in the sense of an arbitrary text-based capture will be unsolvable even now. But with the techniques that we have developed, you can quickly develop a mechanism that solves the capture. They've probably gotten a lot harder too. People have been getting clever and clever at generating these text captures. Yeah. Correct. Okay. So, that was one of the things you've tested on is these kinds of captures in 2014, 2015, that kind of stuff. So, by the way, why captures? Yeah. Even now, I would say capture is a very, very good challenge problem if you want to understand how human perception works and if you want to build systems that work like the human brain. And I wouldn't say capture is a solved problem. We have cracked the fundamental defense of captures, but it is not solved in the way that humans solve it. So, I can give an example. I can take a five-year-old child who has just learned characters and show them any new capture that we create. They will be able to solve it. I can show you pretty much any new capture from any new website. You'll be able to solve it without getting any training examples from that particular style of capture. You're assuming I'm human, yeah. Yes. Yeah. That's right. So, if you are human, otherwise I will be able to figure that out using this one. But... This whole podcast is just a touring test. That's right. A long touring test. Anyway, I'm sorry. So, yeah. So, humans can figure it out with very few examples. Or no training examples. No training examples from that particular style of capture. Even now, this is unreachable for the current deep learning system. So, basically, I don't think a system exists where you can basically say, train on whatever you want. And then now say, hey, I will show you a new capture, which I did not show you in the training setup. Will the system be able to solve it? It still doesn't exist. So, that is the magic of human perception. And Doug Huffstater put this very beautifully in one of his talks. The central problem in AI is what is the letter A. If you can build a system that reliably can detect all the variations of the letter A, you don't even need to go to the... The B and the C. Yeah. You don't even need to go to the B and the C or the strings of characters. And so, that is the spirit with which we tackle that problem. What does it mean by that? I mean, is it like without training examples, try to figure out the fundamental elements that make up the letter A in all of its forms? In all of its forms. A can be made with two humans standing leaning against each other, holding the hands. And it can be made of leaves. It can be... Yeah. You might have to understand everything about this world in order to understand the letter A. Yeah. Exactly. So, it's common sense reasoning, essentially. Yeah. Right. So, to finally, to really solve, finally to say that you have solved capture, you have to solve the whole problem. Yeah. Okay. So, how does this kind of the RCN architecture help us to get... Do a better job of that kind of thing? So, as I mentioned, one of the important things was being able to do inference, being able to dynamically do inference. Can you clarify what you mean? Because you said neural networks don't do inference. Yeah. So, what do you mean by inference in this context? So, okay. So, in captures, what they do to confuse people is to make these characters crowd together. Yes. Okay. And when you make the characters crowd together, what happens is that you will now start seeing combinations of characters as some other new character or an existing character. So, you would put an R and N together, it will start looking like an M. And so, locally, there is very strong evidence for it being some incorrect character. But globally, the only explanation that fits together is something that is different from what you can find locally. So, this is inference. You are basically taking local evidence and putting it in the global context and often coming to a conclusion locally, which is conflicting with the local information. So, you mean inference in the way it's used when you talk about reasoning, for example, as opposed to inference with artificial neural networks, which is a single pass through the network. Okay. So, you're basically doing some basic forms of reasoning, like integration of how local things fit into the global picture. Right. And things like explaining away coming into this one, because you are explaining that piece of evidence as something else, because globally, that's the only thing that makes sense. So, now, you can amortize this inference in a neural network. If you want to do this, you can brute force it. You can just show it all combinations of things that you want your reasoning to work over. And you can just train the help out of that neural network. And it will look like it is doing inference on the fly, but it is really just doing amortized inference. It is because you have shown it a lot of these combinations during training time. So, what you want to do is be able to do dynamic inference rather than just being able to show all those combinations in the training time. And that's something we emphasized in the model. What does it mean dynamic inference? Does that have to do with the feedback thing? Yes. I'm trying to visualize what dynamic inference would be in this case. What is it doing with the input? It's showing the input the first time. What's changing over temporally? What's the dynamics of this inference process? So, you can think of it as you have, at the top of the model, the characters that you are trained on. They are the causes. You're trying to explain the pixels using the characters as the causes. The characters are the things that cause the pixels. Yeah. So, there's this causality thing. So, the reason you mentioned causality, I guess, is because there's a temporal aspect to this whole thing. In this particular case, the temporal aspect is not important. It is more like when if I turn the character on, the pixels will turn on. Yeah. It will be after there's a little bit, but yeah. So, it's causality in the sense of like a logic causality, like hence inference. Okay. The dynamics is that even though locally it will look like, okay, this is an A. And locally, when I look at just that patch of the image, it looks like an A. But when I look at it in the context of all the other causes, it might not, A is not something that makes sense. So, that is something you have to kind of recursively figure out. Yeah. So, okay. And this thing performed pretty well on the captures. Correct. And I mean, is there some kind of interesting intuition you can provide? Why it did well? What did it look like? Is there visualizations that could be human interpretable to us humans? Yes. Yeah. So, the good thing about the model is that it is extremely... So, it is not just doing a classification. It is providing a full explanation for the scene. So, when it operates on a scene, it is coming back and saying, look, this is the part is the A, and these are the pixels that turned on. These are the pixels in the input that makes me think that it is an A. And also, these are the portions I hallucinated. It provides a complete explanation of that form. And then, these are the contours. This is the interior, and this is in front of this other object. So, that's the kind of explanation the inference network provides. So, that is useful and interpretable. And then, the kind of errors it makes are also... I don't want to read too much into it, but the kind of errors the network makes are very similar to the kinds of errors humans would make in a similar situation. So, there's something about the structure that feels reminiscent of the way humans visual system works. Well, I mean, how hard-coded is this to the capture problem, this idea? Not really hard-coded because it's the assumptions, as I mentioned, are general, right? And those themselves can be applied in many situations which are natural signals. So, it's the foreground versus background factorization, and the factorization of the surfaces versus the contours. So, these are all generally applicable assumptions. In all vision. So, why attack the capture problem, which is quite interesting. Why attack the capture problem, which is quite unique in the computer vision context versus the traditional benchmarks of ImageNet and all those kinds of image classification, or even segmentation tasks and all that kind of stuff. Do you feel like that's... I mean, what's your thinking about those kinds of benchmarks in this context? I mean, those benchmarks are useful for algorithms where you... So, the settings that deep learning works in are, here is my huge training set, and here is my test set. So, the training set is almost 100x, 1000x bigger than the test set in many cases. What we wanted to do was invert that. The training set is way smaller than the test set. Yes. And capture is a problem that is by definition hard for computers, and it has these good properties of strong generalization, strong out of training distribution generalization. If you are interested in studying that and having your model have that property, then it's a good data set to tackle. So, have you attempted to, which I think I believe there's quite a growing body of work on looking at MNIST and ImageNet without training? So, taking... The basic challenge is what tiny fraction of the training set can we take in order to do a reasonable job of the classification task? Have you explored that angle in these classic benchmarks? Yes. So, we did do MNIST. So, it's not just capture. So, there was also multiple versions of MNIST, including the standard version, where we inverted the problem, which is basically saying rather than train on 60,000 training data, how quickly can you get to high level accuracy with very little training data? Is there some performance that you remember? How well did it do? How many examples did it need? Yeah. I remember that it was on the order of tens or hundreds of examples to get into 95% accuracy, and it was definitely better than the other systems out there at that time. At that time. Yeah. Yeah. They're really pushing it. I think that's a really interesting space, actually. I think there's an actual name for MNIST that... There's different names to the different sizes of training sets. I mean, people are attacking this problem. I think it's super interesting. It's funny how the MNIST will probably be with us all the way to AGI as a data set that just sticks by. It's a clean, simple data set to study the fundamentals of learning with. Just like CAPTCHAs, it's interesting. Correct. Not enough people... I don't know, maybe you can correct me, but I feel like CAPTCHAs don't show up as often in papers as they probably should. That's correct. Yeah. Because usually these things have a momentum. Once something gets established as a standard benchmark, there is a dynamics of how graduate students operate and how the academic system works that pushes people to track that benchmark. Nobody wants to think outside the box. Okay. Good performance on the CAPTCHAs. What else is there interesting on the RCN side before we talk about the cortical microswitch? Yeah. The same model. The important part of the model was that it trains very quickly with very little training data. It's quite robust to out of distribution perturbations. We are using that very fruitfully in... Advocatious in many of the robotic tasks we are solving. Let me ask you this touchy question. I have spoken with your friend, colleague, Jeff Hawkins, too. I have to ask, whenever you have brain-inspired stuff and you make big claims, big sexy claims, there's critics, machine learning subreddit. Don't give me that name. Machine learning subreddit. Don't give me started on those people. Criticism is good, but they're a bit over the top. There is quite a bit of skepticism and criticism. Is this work really as good as it promises to be? Do you have thoughts on that kind of skepticism? Do you have comments on the kind of criticism we might have received about, is this approach legit? Is this a promising approach? Or at least as promising as it seems to be advertised as? Yeah, I can comment on it. Our Arsene paper is published in Science, which I would argue is a very high quality journal, very hard to publish in. Usually it is indicative of the quality of the work. I am very, very certain that the ideas that we brought together in that paper, in terms of the importance of feedback connections, recursive inference, lateral connections, coming to best explanation of the scene as the problem to solve, trying to solve recognition, segmentation, all jointly in a way that is compatible with higher level cognition, top-down attention, all those ideas that we brought together into something coherent and workable in the world and tackling a challenging problem. I think that will stay and that contribution I stand by. Now, I can tell you a story which is funny in the context of this. If you read the abstract of the paper and the argument we are putting in, look, current deep learning systems take a lot of training data. They don't use these insights. Here is our new model, which is not a deep neural network. It's a graphical model. It does inference. This is how the paper is. Once the paper was accepted and everything, it went to the press department in science, AAAS, science office. We didn't do any press release when it was published. It went to the press department. What was the press release that they wrote up? A new deep learning model. Solves captures. Solves captures. You can see what was being hyped in that thing. There is a dynamic in the community. That especially happens when there are lots of new people coming into the field and they get attracted to one thing. Some people are trying to think different compared to that. I think skepticism is important in science and it is very much required. But it's also not skepticism. Usually, it's mostly bandwagon effect that is happening. But that's not even that. I'll tell you what they react to, which I'm sensitive to as well. If you look at just companies, OpenAI, DeepMind, Vicarious, there's a little bit of a race to the top and hype. It doesn't pay off to be humble. And the press is just irresponsible often. Don't get me started on the state of journalism today. It seems like the people who write articles about these things, they literally have not even spent an hour on the Wikipedia article about what is neural networks. They haven't invested just even the language to laziness. It's like robots beat humans. They write this kind of stuff. The researchers are quite sensitive to that because it gets a lot of attention. They're like, why did this work get so much attention? That's over the top and people get really sensitive. The same kind of criticism with OpenAI did work with Rubik's Cube, with the robot that people criticized. Same with GPT-2 and 3. They criticized the same thing with DeepMinds, with AlphaZero. I'm sensitive to it. And of course, with your work, you mentioned deep learning, but there's something super sexy to the public about brain inspired. That immediately grabs people's imagination. Not even neural networks, but really brain inspired, brain like neural networks. That seems really compelling to people and to me as well, to the world as a narrative. People hook onto that and sometimes the skepticism engine turns on in the research community and they're skeptical. But I think putting aside the ideas of the actual performance on CAPTCHAs or performance on any data set, to me, all these data sets are useless anyway. It's nice to have them, but in the grand scheme of things, they're silly toy examples. The point is, is there intuition about the ideas, just like you mentioned, bringing the ideas together in a unique way? Is there something there? Is there some value there? And is it going to stand the test of time? My confidence there is very high. I don't treat brain inspired as a marketing term. I am looking into the details of biology and puzzling over those things and I am grappling with those things. It is not a marketing term at all. You can use it as a marketing term and people often use it and you can get combined with them. When people don't understand how we're approaching the problem, it is easy to be misunderstood and think of it as purely marketing. But that's not the way we are. As a scientist, you believe that if we just stick to really understanding the brain, you should constantly meditate on how does the brain do this? Because that's going to be really helpful for engineering intelligent systems. I think it's one input and it is helpful, but you should know when to deviate from it too. An example is convolutional neural networks. Convolution is not an operation brain implements. The visual cortex is not convolutional. The visual cortex has local receptive fields, local connectivity, but there is no translation invariance in the network weights in the visual cortex. That is a computational trick, which is a very good engineering trick that we use for sharing the training between the different nodes. That trick will be with us for some time. It will go away when we have robots with eyes and heads that move. Then that trick will go away. It will not be useful at that time. The brain doesn't have translational invariance. It has a focal point. It has a thing it focuses on. It has a fovea. Because of the fovea, the receptive fields are not like the copying of the weights. The weights in the center are very different from the weights in the periphery. I did this. I actually wrote a paper and just got an chance to really study peripheral vision, which is a fascinating thing. Very under-understood thing of what the brain does at every level with the periphery. It does some funky stuff. It's another kind of trick than convolutional. Convolution in neural networks is a trick for efficiency. The brain does a whole other kind of thing, I guess. You need to understand the principles or processing so that you can still apply engineering tricks where you want to do. You don't want to be slavishly mimicking all the things of the brain. It should be one input. I think it is extremely helpful. It should be the point of really understanding so that you know when to deviate from it. Okay. That's really cool. That's work from a few years ago. You did work in Umenta with Jeff Hawkins with hierarchical temporal memory. If you could give a brief history, how has your view of the way the models of the brain changed over the past few years leading up to now? Is there some interesting aspects where there was an adjustment to your understanding of the brain, or is it all just building on top of each other? In terms of the higher level ideas, especially the ones Jeff wrote about in the book, if you blur out the details and if you just zoom out and at the higher level idea, things are, I would say, consistent with what he wrote about. But many things will be consistent with that because deep learning systems are also multi-level, hierarchical, all of those things. But in terms of the detail, a lot of things are different, and those details matter a lot. One point of difference I had with Jeff was how much of biological plausibility and realism do you want in the learning algorithms? So when I was there, this was almost 10 years ago now. That flies when you're having fun. I don't know what Jeff thinks now, but 10 years ago, the difference was that I did not want to be so constrained on saying my learning algorithms need to be biologically plausible based on some filter of biological plausibility available at that time. To me, that is a dangerous cut to make because we are discovering more and more things about the brain all the time. New biophysical mechanisms, new channels are being discovered all the time. So I don't want to up front kill off a learning algorithm just because we don't really understand the full biophysics of how the brain learns. Exactly. Let me ask Sergeant Chaurabh, what's your sense? What's our best understanding of how the brain learns? Things like back propagation, credit assignment. Many of these learning algorithms have things in common. Back propagation is one way of credit assignment. There is another algorithm called expectation maximization, which is another weight adjustment algorithm. But is your sense the brain does something like this? Has to. There is no way around it in the sense of saying that you do have to adjust the connections. So you're saying credit assignment, you have to reward the connections that were useful in making a correct prediction. I guess it doesn't have to be differentiable. Yeah, it doesn't have to be differentiable. But you have a model that you start with, you have data comes in, and you have to have a way of adjusting the model such that it better fits the data. So that is all of learning. And some of them can be using backprop to do that. Some of it can be using very local graph changes to do that. Many of these learning algorithms have similar update properties locally in terms of what the neurons need to do locally. I wonder if small differences in learning algorithms can have huge differences in the actual effect. So the dynamics of sort of the reverse spiking, if credit assignment is like a lightning versus a rainstorm or something, whether there's a looping local type of situation with the credit assignment, whether there is regularization, how it injects robustness into the whole thing, whether it's chemical or electrical or mechanical, all those kinds of things. I feel like those differences could be essential, right? It could be. It's just that you don't know enough to, on the learning side, you don't know enough to say that is definitely not the way the brain does it. Got it. So you don't want to be stuck to it. So you've been open-minded on that side of things. Correct. On the inference side, on the recognition side, I am much more amenable to being constrained because it's much easier to do experiments because it's like, okay, here's the stimulus. How many steps did it get to take the answer? I can trace it back. I can understand the speed of that computation, et cetera, much more readily on the inference side. Got it. And then you can't do good experiments on the learning side. Correct. So let's go right into the cortical microcircuits right back. So what are these ideas beyond recursive cortical network that you're looking at now? So we have made a pass through multiple of the steps that, as I mentioned earlier, we were looking at perception from the angle of cognition. It was not just perception for perception sake. How do you connect it to cognition? How do you learn concepts? And how do you learn abstract reasoning similar to some of the things Francois talked about? So we have taken one pass through it, basically saying, what is the basic cognitive architecture that you need to have, which has a perceptual system, which has a system that learns dynamics of the world, and then has something like a routine program learning system on top of it to learn concepts. So we have built the version point one of that system, this was another science robotics paper. The title of that paper was something like cognitive programs. How do you build cognitive programs? And the application there was on manipulation, robotic manipulation? It was, so think of it like this. Suppose you wanted to tell a new person that you met, you don't know the language that person uses, you want to communicate to that person to achieve some task. So I want to say, hey, you need to pick up all the red cups from the kitchen counter and put it here. How do you communicate that? You can show pictures, you can basically say, look, this is the starting state, the things are here, this is the ending state, how do you communicate that? And what does the person need to understand from that? The person needs to understand what conceptually happened in those pictures from the input to the output. So we are looking at pre-verbal conceptual understanding. Without language, how do you have a set of concepts that you can manipulate in your head? And from a set of images of input and output, can you infer what is happening in those images? Got it. With concepts that are pre-language, okay. So what does it mean for a concept to be pre-language? Yeah. Why is language so important here? So I want to make a distinction between concepts that are just learned from text by just feeding brute force text. You can start extracting things like, okay, cow is likely to be on grass. So those kinds of things you can extract purely from text. But that's kind of a simple association thing rather than a concept as an abstraction of something that happens in the real world in a grounded way that I can simulate it in my mind and then I can use it to make it in my mind and connect it back to the real world. And you think kind of the visual world, concepts in the visual world are somehow lower level than just the language? The lower level kind of makes it feel like, okay, that's like an unimportant, like it's more like I would say the concepts in the visual and the motor system and the concept learning system, which if you cut off the language part, just what we learn by interacting with the world and abstractions from that, that is a prerequisite for any real language understanding. So you disagree with Chomsky because he says language is at the bottom of everything. No, I disagree with Chomsky completely on how many levels from universal grammar to yeah. So that was a paper in science beyond the recursive cortical network. What other interesting problems are there, the open problems and brain inspired approaches that you're thinking about? I mean, everything is open, right? Like no problem is solved, solved. I think of perception as kind of the first thing that you have to build, but the last thing that you will be actually solved. Because if you do not build perception system in the right way, you cannot build concept system in the right way. So you have to build a perception system, however wrong that might be, you have to still build that and learn concepts from there and then keep iterating. And finally, perception will get solved fully when perception, cognition, language, all those things work together finally. So great, we've talked a lot about perception, but then maybe on the concept side and like common sense or just general reasoning side, is there some intuition you can draw from the brain about how we could do that? So I have this classic example I give. So suppose I give you a few sentences and then ask you a question following that sentence. This is a natural language processing problem, right? So here it goes. I'm telling you, Sally pounded a nail on the ceiling. Okay, that's a sentence. Now I'm asking you a question, was the nail horizontal or vertical? Vertical. Okay, how did you answer that? Well, I imagined Sally, it was kind of hard to imagine what the hell she was doing, but I imagined a visual of the whole situation. Exactly, exactly. So here, I post a question in natural language. The answer to that question was you got the answer from actually simulating the scene. Now I can go more and more detail about, okay, was Sally standing on something while doing this? Could she have been standing on a light bulb to do this? I could ask more and more questions about this and I can ask, make you simulate the scene in more and more detail, right? Where is all that knowledge that you're accessing stored? It is not in your language system. It was not just by reading text you got that knowledge. It is stored from the everyday experiences that you have had. By the age of five, you have pretty much all of this, right? It is stored in your visual system, motor system, in a way such that it can be accessed through language. Got it. I mean, right. So the language is just almost sort of the query into the whole visual cortex and that does the whole feedback thing. But I mean, it is all reasoning kind of connected to the perception system in some way. You can do a lot of it. You can still do a lot of it by quick associations without having to go into the depth. And most of the time you will be right, right? You can just do quick associations, but I can easily create tricky situations for you where that quick association is wrong and you have to actually run the simulation. So figuring out how these concepts connect, do you have a good idea of how to do that? That's exactly what one of the problems that we are working on. And the way we are approaching that is basically saying, okay, you need to... So the takeaway is that language is simulation control and your perceptual plus motor system is building a simulation of the world. And so that's basically the way we are approaching it. And the first thing that we built was a controllable perceptual system and we built a schema networks, which was a controllable dynamic system. Then we built a concept learning system that puts all these things together into programs or subtractions that you can run and simulate. And now we are taking the step of connecting it to language. And it will be very simple examples initially. It will not be the GPT-3 like examples, but it will be grounded simulation-based language. And for like the querying would be like question answering kind of thing? Correct. And it will be in some simple world initially, but it will be about, okay, can the system connect the language and grounded in the right way and run the right simulations to come up with the answer. And the goal is to try to do things that, for example, GPT-3 couldn't do. Correct. Speaking of which, if we could talk about GPT-3 a little bit, I think it's an interesting thought-provoking set of ideas that OpenAI is pushing forward. I think it's good for us to talk about the limits and the possibilities in neural networks. So in general, what are your thoughts about this recently released very large 175 billion parameter language model? So I haven't directly evaluated it yet. From what I have seen on Twitter and other people evaluating it, it looks very intriguing. I am very intrigued by some of the properties it is displaying. And of course, the text generation part of that was already evident in GPT-2, that it can generate coherent text over long distances. But of course, the weaknesses are also pretty visible in saying that, okay, it is not really carrying a world state around. And sometimes you get sentences like, I went up the hill to reach the valley or something like that, some completely incompatible statements. Or when you're traveling from one place to the other, it doesn't take into account the time of travel, things like that. So those things I think will happen less in GPT-3 because it is trained on even more data and it can do even more longer distance coherence. But it will still have the fundamental limitations that it doesn't have a world model and it can't run simulations in its head to find whether something is true in the world or not. Do you think within, so it's taking a huge amount of text from the internet and forming a compressed representation. Do you think in that could emerge something that's an approximation of a world model, which essentially could be used for reasoning? I'm not talking about GPT-3, I'm talking about GPT-4, 5 and GPT-10. Yeah, I mean, they will look more impressive than GPT-3. So if you take that to the extreme, then a Markov chain of just first order, if you go to, I'm taking it the other extreme, if you read Shannon's book, he has a model of English text, which is based on first-order Markov chains, second-order Markov chains, third-order Markov chains and think that, okay, third-order Markov chains look better than first-order Markov chains. So does that mean a first-order Markov chain has a model of the world? Yes, it does. So yes, in that level, when you go higher-order models or more sophisticated structure in the model like the transformer networks have, yes, they have a model of the text world. But that is not a model of the world. It's a model of the text world and it will have interesting properties and it will be useful, but just scaling it up is not going to give us AGI or natural language understanding or meaning. The question is whether being forced to compress a very large amount of text forces you to construct things that are very much like... Because the ideas of concepts and meaning is a spectrum. So in order to form that kind of compression, maybe it will be forced to figure out abstractions, which look awfully a lot like the kind of things that we think about as concepts, as world models, as common sense. Is that possible? No, I don't think it is possible because the information is not there. The information is there behind the text, right? No, unless somebody has written down all the details about how everything works in the world to the absurd amounts like, okay, it is easier to walk forward than backward, that you have to open the door to go out of the thing, doctors wear underwear, unless all these things somebody has written down somewhere or somehow the program found it to be useful for compression from some other text, the information is not there. So that's an argument that text is a lot lower fidelity than the experience of our physical world. Right, right. It's worth a thousand words. Well, in this case, pictures aren't really so the richest aspect of the physical world isn't even just pictures, it's the interactivity with the world. Exactly. Yeah, it's being able to interact. It's almost like if you could interact. So I disagree. Well, maybe I agree with you that pictures were a thousand words, but a thousand is still. Yeah, you could say you could capture it with the GPT X. So I wonder if there's some interactive element where a system could live in text world where it could be part of the chat, be part of talking to people. It's interesting. I mean, fundamentally, so you're making a statement about the limitation of text. Okay. So let's say we have a text corpus that includes basically every experience we could possibly have. I mean, just a very large corpus of text and also interactive components. I guess the question is whether the neural network architecture, these very simple transformers, but if they had like hundreds of trillions or whatever comes after a trillion parameters, whether that could store the information needed that's architecturally. Do you have thoughts about the limitation on that side of things with neural networks? I mean, so transformer is still a feed forward neural network. It has a very interesting architecture, which is good for text modeling and probably some aspects of video modeling, but it is still a feed forward architecture. You believe in the feedback mechanism, the recursion. Oh, and also causality, being able to do counterfactual reasoning, being able to do interventions, which is actions in the world. So all those things require different kinds of models to be built. I don't think transformers captures that family. It is very good at statistical modeling of text and it will become better and better with more data, bigger models, but that is only going to get so far. So I had this joke on Twitter saying that, hey, this is a model that has read all of quantum mechanics and theory of relativity and we are asking you to do text completion or we are asking you to solve simple puzzles. When you have AGI, that's not what you ask a system to do. We'll ask the system to do experiments and come up with hypothesis and revise the hypothesis based on evidence from experiments, all those things. Those are the things that we want the system to do when we have AGI, not solve the simple puzzles. Next impressive demo, somebody generating a red button in HTML. Right. Which are all useful. There is no dissing the usefulness of it. By the way, I'm playing a little bit of a devil's advocate, so calm down, Internet. So I'm curious almost in which ways will a dumb but large neural network will surprise us. So like, um, it's kind of your, I completely agree with your intuition is just that I don't want to dogmatically like a hundred percent put all the chips there. We've been surprised so much. Even the current GPT two and three are so surprising. Yeah. The self play mechanisms of alpha zero are really surprising. And I, the reinforcement, the fact that reinforcement learning works at all to me is really surprising. The fact that neural networks work at all is quite surprising given how nonlinear the spaces, the fact that it's able to find local minima that are at all reasonable. It's very surprising. So it, uh, I wonder sometimes whether us humans just want it to not, for AGI, not to be such a dumb thing. So I just, cause exactly what you're saying is like, um, the ideas of concepts and be able to reasonable those concepts and, and connect those concepts in a, like hierarchical ways. And then to be able to have a world models. I mean, just everything we're describing in human language in this poetic way, uh, seems to make sense that that is what intelligence, the reasoning are like. I wonder if at the core of it, it could be much dumber. Uh, well finally it is still connections and messages passing over them. Right. Right. So in that way it is done. So I guess the recursion, the, the feedback mechanism that does seem to be a fundamental kind of thing. Um, yeah, yeah. The idea of concepts, also memory. Correct. Yeah. Like having an episodic memory. Yeah. Yeah. That seems to be an important thing. So how do we get memory? So yeah, we have another piece of work that, which came on recently on how do you form episodic memories and, uh, and form abstractions from them. Uh, and we haven't figured out a, you know, all the connections of that to the overall cognitive architecture, but, um, yeah, what are your ideas about how you could have episodic memory? So at least it's very clear that there, you need to have two kinds of memory, right? That's very, very clear, right? We chose, there are things that happen, uh, as statistical patterns in the world. Uh, but then there is the, the, the one timeline of things that happen only once in your life, right? Uh, uh, and this day is not going to happen ever again. And, and so, and that needs to be stored as a, as a, you know, just a stream of, uh, string, right? This is, this is my experience. And then, then the question is about how do you take that experience and connect it to the statistical part of it? How do you now say that, okay, I experienced this thing. Now I want to be careful about similar situations. Uh, and so, so you need to be able to index that similarity using your other giants that is, you know, the, the model of the world that you have learned, although the situation came from the episode, you need to be able to index the other one. So, uh, the episodic memory being implemented as an indexing over the other, uh, model that you're building. So the memories remain and they, uh, they, they're an index into this, like the statistical thing that you formed. Yeah. Statistical causal structural model that you built over, over time. So, so it's basically the idea is that, uh, the hippocampus is, uh, just storing or sequencing, uh, a, in a set of pointers that happens over time. And then whenever you want to reconstitute that memory and evaluate the different, uh, uh, aspects of it, whether it was good, bad, do I need to encounter the situation again? You need the cortex, uh, to reinstantiate, to replay that memory. So how do you find that memory? Like which direction is the important direction? Both directions are, you know, it's again bidirectional. So I guess how do you retrieve the memory? So this is again hypothesis, right? We're making this up. So when you, uh, when you come to a, uh, new situation, right? Uh, your, your cortex is doing inference, uh, over in the new situation. And then, uh, of course hippocampus is connected to different parts of the cortex. Um, and, and you have this deja vu situation, right? Okay. I have seen this thing before. And, uh, and then in the hippocampus, you can have an index of, okay, this is when it happened as a, as a, as a timeline. Uh, and, and, and then then you can use the hippocampus to drive the similar timelines to say, now I am, I am rather than being driven by my current input stimuli, I am going back in time and rewinding my experience from there, putting back into the cortex and then putting it back into the cortex, of course affects what you're going to see next in your current situation. Got it. Yeah. So that's, that's the whole thing. Having a world model and then the, yeah. Uh, connecting to the perception. Yeah. It does seem to be that that's what's happening and be, uh, on the neural network side, it's, um, it's interesting to think of how we actually do that. Yeah. Yeah. So I have a knowledge base. Yes. It is possible that you can put many of these structures into, uh, neural networks and we will find ways of combining properties of neural networks and graphical models. So, uh, I mean, it's already started happening. Uh, graph neural networks are kind of a merge between them. Uh, and there will be more of that thing. So, but to me it is the direction is pretty clearly, you know, looking at biology and the history of, uh, uh, evolutionary history of intelligence, it is pretty clear that, okay, what we need is more structure in the models and modeling of the world and supporting dynamic inference. Well, let me ask you, uh, there's a guy named Elon Musk. There's a company called Neuralink and there's a general field called brain computer interfaces. Yeah. Um, it's kind of, uh, interface between your two loves the brain and the intelligence. Uh, so there's like very direct applications of brain computer interfaces for, um, people with different conditions, more in the short term, but there's also these sci-fi futuristic kinds of ideas of, uh, AI systems being able to, uh, communicate in a high bandwidth way with the brain, bi-directional. Yeah. Uh, what are your thoughts about, uh, Neuralink and BCI in general as a, as a possibility? So I think BCI is a cool research area. And in fact, um, when I got interested in brains initially, when, you know, so I was enrolled at Stanford and when I got interested in brains, it was, it was through a brain, uh, computer interface talk that Krishna Shainoy gave. That's when I even started thinking about the problem. So, uh, so it is definitely a fascinating research idea and it is, the applications are enormous, right? Um, so, you know, there's a science fiction scenario of, you know, brains directly communicating. Let's, you know, let's keep that aside for the time being. Uh, uh, even just the intermediate milestones that we're pursuing, which are very reasonable as far as I can see, uh, being able to control an external limb using, uh, uh, in a direct connections from the brain and being able to write things into the brain. Uh, so, so those are all, uh, good steps to take and they have enormous applications, you know, people losing limbs, being able to control prosthetics, uh, quadriplegics, being able to control something, uh, so, and therapeutics. And you know, I also know about another company working in this space, uh, called Paradromics. Uh, they're doing, you know, it's based on a different, uh, electrode array, but trying to attack some of the same problems. So I think it's a very- Also surgery. Correct. Surgically implanted electrodes. Yeah. Um, so, uh, yeah, I think of it as a very, very promising field, especially when it is helping people overcome some limitations. You know, at some point, of course it will advance the level of being able to communicate. Uh, How hard is that problem do you think? Like, so, so, okay, let's say we magically solve what I think is a really hard problem of doing all of this safely. Yeah. So, so like being able to connect electrodes and not just thousands, but like millions to the brain. I think it's very, very hard because you also do not know what the, what will happen to the brain with that, right? In the sense of how does the brain adapt to something like that? And it's, you know, as we were learning, it's the brain is quite, uh, in terms of neuroplasticity is pretty malleable. So it's going to adjust. So the machine learning side, the computer side is going to adjust and then the brain is going to adjust. Exactly. And then what soup does this land us into is the kind of hallucinations you might get from it. From this. That might be pretty intense. Yeah. Yeah. So, uh, just connecting to all of Wikipedia. It's interesting whether we need to be able to figure out the basic protocol of the brain's communication schemes in order to get them to the machine and the brain to talk. Cause another possibility is the brain actually just adjust to whatever the heck the computer is doing. Exactly. That's the way I think that I find that to be a more promising way. It's basically saying, you know, okay, attach electrodes to some part of the cortex. Okay. And make sure maybe if it is done from birth, the brain will adapt. It says that, you know, that part is not damaged. It was not used for anything. These electrodes are attached there. Right. And now you, you train that part of the brain to do this high bandwidth communication between something else. Right. And, and, uh, if you do it like that, then it is brain adapting to, and of course your external system is designed such that it is adaptable. You know, just like we, you know, design computers or mouse keyboard, all of them to be, uh, uh, interacting with humans. So, uh, of course that feedback system is designed to be, uh, human compatible, but, um, now it is not trying to record from the, all of the brain and, uh, uh, you know, now, you know, two system trying to adapt to each other. It's a brain adapting into one way. That's fascinating. The brain is connected to like the internet. It's connected. Just imagine just connecting it to Twitter and just, just, just taking that stream of information. Um, yeah. But again, if we take a step back, I don't know what your intuition is. I feel like that is not as hard of a problem as the doing it safely. There's, there's a huge barrier to surgery that cause it, cause the biological system, it's a mush of like weird stuff. Correct. So that the surgery part of it, biology part of it, the long-term repercussions part of it, again, I don't know what else will, uh, you know, we, we often find, uh, after a long time, uh, in biology that, okay, that idea was wrong, right. You know, so people used to cut off this, uh, uh, the gland called the thymus or something. Uh, and, uh, then they found that, oh no, that actually causes cancer. And then there's a subtle, like millions of variables involved, but this whole process, the nice thing, just like again, with the Elon, just like colonizing Mars seems like a ridiculously difficult idea, but in the process of doing it, we might learn a lot about the biology of the neurobiology of the brain, the neuroscience side of things. It's like, if you want to learn something, do the most difficult version of it and see what you learn. The intermediate steps that they are taking sounded already reasonable to me. Yeah, it's great. Well, but like everything with Elon is the timeline seems insanely fast. So that's, that's the only awful question. Uh, well, one, we've been talking about cognition a little bit, so like reasoning, we haven't mentioned the other C word, which is consciousness. Uh, do you ever think about that one? Do you, is that useful at all? Uh, in this whole context of what it takes to create an intelligent reasoning being, or is that completely outside of, uh, your, uh, like the engineering perspective of intelligence? It is not outside the realm, but it doesn't on a day to day basis inform what we do, but it's more so in, in many ways, the company name is connected to this idea of consciousness. What's the company name? Vicarious, you know, so Vicarious is the company name. And, uh, and so what does Vicarious mean, right? It's, um, uh, at the first level, it is about modeling the world and, uh, and it is internalizing the external actions. So, so you interact with the world and learn a lot about the world. And now after having learned a lot about the world, you can run those things in your mind without actually having to act in the world. So you can run, uh, things Vicariously just in your, in your, in your brain. And similarly, you can experience another person's thoughts by, you know, having a model of how that person works and, uh, and running there, you know, putting yourself in some other person's shoes. So that is being Vicarious. Now, it's the same modeling apparatus that you're using to model the external world or some other person's thoughts. You can turn it to yourself. You can up, you know, if that same modeling thing is applied to your own modeling apparatus, uh, then that is what gives rise to consciousness, I think. Well, that's more like self-awareness. There's the hard problem of consciousness, which is like, when the model becomes, when, when the model feels like something, when this whole process is like, it act, it's like, you really are in it. It, you feel like an entity in this world, not just, you know, that you're an entity, but it feels like something to be that entity. It, um, it, you know, and thereby we attribute this, you know, then this starts to be where something that has consciousness can suffer. You start to have these kinds of things that we can reason about that is much, uh, much heavier. It seems like there's much greater cost of your, your decisions and like mortality is tied up into that. Like the fact that these things end, right? First of all, I end at some point and then other things end. And you know that, that somehow seems to be at least for us humans, a deep motivator. Yes. And that, you know, that, that idea of motivation in general, we talk about goals and AI, but the goals aren't quite the same thing as like the, our mortality. It feels like, it feels like, first of all, humans don't have a goal and they just kind of create goals at different levels. They like make up goals because we're terrified by the mystery of the thing that gets us all. So we make these goals up. So we're like a goal generation machine as opposed to a machine which optimizes the trajectory towards a singular goal. So it feels like that's an important part of, uh, cognition, that whole mortality thing. Well, it is, it is a part of human, uh, cognition, uh, but there is no, uh, reason for, uh, that mortality to come to the question for a, uh, artificial system because we can, uh, copy the artificial system. The, the, the problem with humans is that we can't, I can't clone you. I can't, I, I, you know, I can, I can clone, even if I clone you as a, uh, you know, the, the hardware, your experience, uh, that was stored in your brain, uh, your, uh, episodic memory, all those will not be captured in the, in the new clone. Um, so, um, but that's not the same with an AI system, right? So, but it's also possible that the, the thing that you mentioned with us humans is actually fundamental, of fundamental importance for intelligence. So like the fact that you can copy an AI system means that that AI system is not yet an, um, AGI. So like, so if you look at existence proof, if we reason based on existence proof, you could say that it doesn't feel like death is a fundamental property of an intelligent system, but we don't yet give me an example of an immortal intelligent being. Uh, we don't have those. It could, it's very possible that, you know, that's, it is a fundamental property of intelligence is a thing that has a deadline for itself. You can think of it like this. So suppose you invent a way to freeze people, uh, for a long time, it's not dying, right? Uh, so, so you can be frozen and woken up, uh, thousands of years from now. Uh, so it's no fear of death. So no, the, you're still, it's not, it's not about time. It's about the knowledge that it's temporary. Uh, the, that aspect of it, the finiteness of it, I think, um, creates a kind of urgency. Correct. For us, for humans. Yeah. For humans. Yes. Uh, and that, that is part of our drives. Uh, but, um, and that's why I'm not too worried about AI, uh, you know, uh, having motivations to kill all humans and those kinds of things. Why just wait? You know, so why do you need to do that? Yeah. I've never heard that before. That's a good, that's a good point. Yeah. Just murder seems like a lot of work. We'll just wait, wait it out. That'll probably hurt themselves. Let me ask you, um, people often kind of wonder world-class researchers such as yourself, what kind of books, technical fiction, philosophical were, um, uh, had an impact on you in your life and, uh, maybe ones you could probably possibly recommend that others read. Maybe if you have three books that pop into mind. Yeah. So I definitely liked, uh, Judy Apple's book, uh, probabilistic reasoning and intelligence systems. It's, um, it's a very deep technical book, but what I liked is that in, so there are many places where you can learn about probabilistic graphical models from, but throughout this book, Judy Apple kind of sprinkles his philosophical observations and, and he thinks about, connects us to how the brain thinks and attentions and resources, all those things. So, so that whole thing makes it more interesting to read. Uh, he emphasizes the importance of causality. So that was in his later book. So this was the first book probabilistic reasoning and intelligence systems. He mentions causality, but he hadn't really sunk his teeth into like, you know, how do you actually formalize that? Yeah. And, uh, the second book causality is the 2000, uh, the one in 2000, that one is really hard. So I wouldn't recommend that. Oh yeah. So that looks at the, like the mathematical, like his model of, uh, do calculus. Yeah. It was pretty dense mathematical. Right. Right. Right. Uh, the book of why is definitely more enjoyable. Oh, for sure. Yeah. Um, so yeah, so I would, I would recommend probabilistic reasoning in intelligence systems. Another book I liked, uh, was, uh, one from Doug Huffstater. Uh, this was a long time ago. He has a book, he had a book, I think called it was called the mind's eye. It was, um, uh, probably Huffstater and Daniel Dennett together. Uh, yeah. So, and it actually was, uh, I bought that book. It's in my show. I haven't read it yet, but I, uh, I couldn't get an electronic version of it, which is annoying because I'm, you read everything on Kindle. Uh, you had to actually purchase the physical. It's like one of the only physical books I have. Cause anyway, there's a lot of people who recommended it highly. So yeah. And the third one, uh, I would definitely recommend reading is, um, uh, this is not a technical book. It is history. It's called it's the name of the book. I think it's Bishop's boys. It's about pride brothers and, uh, and their, their, their path and how it was, uh, it's, there are multiple books on this topic and all of them are great. It's, uh, fascinating how, uh, flight was, uh, you know, treated as an unsolvable problem. And, and, and also, you know, what aspects did people emphasize? Uh, you know, people thought, oh, it is all about, uh, just powerful engines, you know, just need to have powerful lightweight engines. Uh, and, um, so, you know, some people thought of it as how far can we just throw the thing, you know, throw it. So, uh, so it's a, it's a very fascinating. And even after they, uh, made the invention, like, you know, people not believing it and, uh, uh, the social aspect of it. It's a different, you know, very passionate. Do you, do you draw any parallels between, you know, birds fly? So there's the natural approach to, uh, to flight and then there's the engineered approach. Do you, uh, do you see the same kind of thing with the brain and our trying to engineer intelligence? Yeah, it's, it's a good analogy to have. Uh, of course, all analogies have their, you know, so, so, so people in, uh, you know, AI often, uh, use airplanes as an example of, Hey, we didn't learn anything from birds. Look, but the funny thing is that, uh, and, and the, the saying is, uh, airplanes don't flap wings. Right. This is what they say. The funny thing and ironic thing is that, that you don't need to flap to fly is something Wright brothers found by observing birds. So they have in their notebook, you know, in some of these books, they show their, their notebook drawings, right? They, they make detailed notes about buzzards, uh, just soaring over thermals. And they basically say, look, flapping is not the important proportion is not the important problem to solve here. We want to solve control. Uh, and, uh, once you solve control, propulsion will fall into place. All of, all of these are people, uh, you know, they realize this by observing birds. Beautiful report. That's actually brilliant. Uh, cause people do use that analogy. I'm gonna have to remember that one. Do you have advice for people interested in artificial intelligence, like young folks today? I talked to undergraduate students all the time, uh, interested in neuroscience, interested in, uh, understanding how the brain works. Is there advice you would give them about their career, maybe about their life? Uh, sure. I think every, you know, every piece of advice should be taken with a pinch of salt because each person is different. Their motivations are different. Uh, but I can, I can definitely say if your goal is to understand the brain from the angle of wanting to build one, you know, uh, then, uh, being an experimental neuroscientist might not be the way to go about it. Um, uh, it might be a better way to pursue it might be through, um, computer science, electrical engineering, machine learning and AI. And of course you have to study up the neuroscience, but that you can do on your own. Um, if you are more, uh, attracted by finding something intriguing about the discovering something intriguing about the brain, uh, then of course it is better to be an experimentalist. Uh, so find that motivation. What are you intrigued by? And of course, find your strengths too. Some people are very good experimentalists, uh, and, and they enjoy doing that. Uh, and it's interesting to see which department, if you're, if you're picking, uh, in terms of like your education path, whether to, um, uh, to, to go with like an MIT it's brain and computer, uh, no, uh, BCS brain and cognitive sciences. Yeah. Uh, or, or the CS side of things. And actually, uh, the brain folks, the neuroscience folks are more and more now embracing of, uh, you know, learning TensorFlow and PyTorch, right there. They, they see the power of, uh, trying to engineer ideas, uh, that, uh, that they get from the brain into, and then explore how those could be used to, uh, to create intelligent systems. So that might be the right department actually. Yeah. To, uh, so this was a question in, uh, uh, you know, one of the Redwood neuroscience Institute workshops, uh, that Jeff Hawkins organized, uh, uh, almost 10 years ago. This question was put to a panel, right? What, what should be the undergrad major, uh, you should take if you want to understand the brain. And, uh, and the majority of opinion in that one was, uh, electrical engineering. Interesting. Uh, because I mean, I'm a doubly undergrad, so I got lucky in that way. Uh, um, but it, I think it does have some of the right ingredients because you learn about circuits, you, you learn about how you can construct circuits to, uh, you know, approach, you know, do functions, uh, you learn about microprocessors, um, you learn information theory, you learn signal processing, uh, you learn continuous math. So, um, so in that way, it's, it's a good step to, if you want to go to computer science or neuroscience, you can, it's, it's a good step. The downside, you're more likely to be forced to use MATLAB. So, so, so one of the, one of the interesting things about, I mean, this is changing, the world is changing, but, uh, like certain departments lagged on the programming side of things on developing good, uh, good habits and software engineering. But I think that's more and more changing and, and students can take the answer of their own hands, like learn to program. I feel like everybody should learn to program because it, uh, it's like everyone in the sciences, because it empowers, it puts the data at your fingertips. So you can organize it. You can find all kinds of things in the data. And then you can also for the appropriate sciences, build systems that like based on that. So like then engineer intelligence systems, uh, we already talked about mortality, so we hit a ridiculous point, but let me ask you the, uh, you know, one of the things about intelligence is it's goal-driven and, uh, you study the brain. So the question is like, what's the goal that the brain is operating under? What's, what's the meaning of it all for us humans in your view? What's the meaning of learning? What's the meaning of it all for us humans in your view? What's the meaning of life? The meaning of life is what you construct out of it. It's completely open. It's open. So there's no, there's nothing, uh, uh, like you mentioned, you like constraints. So there's what's, uh, it's, it's, it's wide open. Is there, is there some useful aspect that you think about in terms of like the openness of it and just the basic mechanisms of generating goals, uh, in studying cognition in the brain that you think about, or is it just about, cause everything we've talked about, kind of the perception system is to understand the environment. That's like to be able to like not die, like not fall over and like be able to, uh, you don't think we need to, um, think about anything bigger than that. Yeah, I think so because it's, it's basically being able to understand the machinery of the world, uh, such that you can pursue whatever goals you want. Right. So the machinery of the world is, is really ultimately what we should be striving to understand. The rest is just, the rest is just whatever the heck you want to do or whatever, whatever. I think that's, uh, I, that's beautifully put. I don't think there's a better way to end it. I'm so honored that you, you would show up here and waste your time with me. It's been an awesome conversation. Thanks so much for talking today. Oh, thank you so much. This was, this was so much more fun than I expected. Thank you. Thanks for listening to this conversation with Dilip George. And thank you to our sponsors, Babbel, Raycon Earbuds and Masterclass. Please consider supporting this podcast by going to babbel.com and use code Lex, going to buy Raycon.com slash Lex and signing up at masterclass.com slash Lex. Click the links, get the discount. It really is the best way to support this podcast. If you enjoy this thing, subscribe to our channel, and we'll see you next time. Thank you for listening and hope to see you next time.
 The following is a conversation with Dalip George, a researcher at the intersection of
 neuroscience and artificial intelligence, co-founder of Vicarious with Scott Phoenix,
 and formerly co-founder of Numenta with Jeff Hawkins, who's been on this podcast, and Donna
 Dubinsky. From his early work on hierarchical temporal memory to recursive cortical networks
 to today, the Leaps always sought to engineer intelligence that is closely inspired by the
 human brain. As a side note, I think we understand very little about the fundamental principles
 underlying the function of the human brain, but the little we do know gives hints that may be more
 useful for engineering intelligence than any idea in mathematics, computer science, physics, and
 scientific fields outside of biology. And so the brain is a kind of existence proof that says
 it's possible. Keep at it. I should also say that brain-inspired AI is often overhyped,
 and use this fodder, just this quantum computing, for a marketing speak. But I'm not afraid of
 exploring these sometimes overhyped areas since where there's smoke, there's sometimes fire.
 Quick summary of the ads. Three sponsors, Babbel, Raycon Earbuds, and Masterclass. Please consider
 supporting this podcast by clicking the special links in the description to get the discount.
 It really is the best way to support this podcast. If you enjoy this thing, subscribe on YouTube,
 review the Five Stars on Apple Podcasts, support on Patreon, or connect with me on Twitter at Lex
 Friedman. As usual, I'll do a few minutes of ads now and never any ads in the middle that can break
 the flow of the conversation. This show is sponsored by Babbel, an app and website that gets you speaking
 in a new language within weeks. Go to babbel.com and use code Lex to get three months free. They
 offer 14 languages, including Spanish, French, Italian, German, and yes, Russian. Daily lessons
 are 10 to 15 minutes, super easy, effective, designed by over 100 language experts. Let me
 read a few lines from the Russian poem, Noch, Ulitsa, Fanar, Abteka, by Alexander Bloch,
 that you'll start to understand if you sign up to Babbel.
 Now, I say that you'll only start to understand this poem because Russian starts with the language
 and ends with the vodka. Now, the latter part is definitely not endorsed or provided by Babbel
 and will probably lose me the sponsorship. But once you graduate from Babbel, you can enroll
 my advanced course of late night Russian conversation over vodka. I have not yet developed an app for
 that. It's in progress. So get started by visiting babbel.com and use code Lex to get three months
 free. This show is sponsored by Raycon Earbuds. Get them at buyraycon.com slash Lex. They become
 my main method of listening to podcasts, audiobooks, and music when I run, do pushups and pull ups,
 or just living life. In fact, I often listen to brown noise with them when I'm thinking deeply
 about something, it helps me focus. They're super comfortable, pair easily, great sound, great bass,
 six hours of playtime. I've been putting in a lot of miles to get ready for a potential ultra
 marathon and listening to audio books on World War II. The sound is rich and really comes in clear.
 So again, get them at buyraycon.com slash Lex. This show is sponsored by Masterclass. Sign up
 at masterclass.com slash Lex to get a discount and to support this podcast. When I first heard
 about Masterclass, I thought it was too good to be true. I still think it's too good to be true.
 For 180 bucks a year, you get an all access pass to watch courses from to list some of my favorites.
 Chris Hatfield on space exploration, Neil deGrasse Tyson on scientific thinking and communication,
 Will Wright, creator of SimCity and Sims and game design. Every time I do this read, I really want
 to play a city builder game. Carlos Santana on guitar, Garry Kasparov on chess, Daniel Legano
 on poker and many more. Chris Hatfield explaining how rockets work and the experience of being
 launched into space alone is worth the money. By the way, you can watch it on basically any device.
 Once again, sign up at masterclass.com to get a discount and to support this podcast.
 And now here's my conversation with Daleep George. Do you think we need to understand the brain in
 order to build it? Yes. If you want to build the brain, we definitely need to understand how it
 works. So Blue Brain or Henry Markram's project is trying to build a brain without understanding it,
 just trying to put details of the brain from neuroscience experiments into a giant simulation
 by putting more and more neurons, more and more details. But that is not going to work because
 when it doesn't perform as what you expect it to do, then what do you do? You just keep adding more
 details. How do you debug it? So unless you understand, unless you have a theory about
 how the system is supposed to work, how the pieces are supposed to fit together, what they're going
 to contribute, you can't build it. At the functional level, understand. So can you actually
 linger on and describe the Blue Brain project? It's kind of a fascinating principle and idea
 to try to simulate the brain. We're talking about the human brain, right?
 Right. Human brains and rat brains or cat brains have lots in common that the cortex,
 the neocortex structure is very similar. So initially they were trying to just simulate
 a cat brain. To understand the nature of evil.
 To understand the nature of evil. Or as it happens in most of these simulations,
 you easily get one thing out, which is oscillations. Yeah, if you simulate a large
 number of neurons, they oscillate. And you can adjust the parameters and say that, oh,
 oscillations match the rhythm that we see in the brain, et cetera.
 Oh, I see. So the idea is, is the simulation at the level of individual neurons?
 Yeah. So the Blue Brain project, the original idea as proposed was, you put very detailed
 biophysical models of neurons and you interconnect them according to the statistics of connections
 that we have found from real neuroscience experiments. And then turn it on and see what
 happens. And these neural models are incredibly complicated in themselves, right? Because these
 neurons are modeled using this idea called Hodgkin-Huxley models, which are about how
 signals propagate in a cable. And there are active dendrites, all those phenomena,
 which those phenomena themselves, we don't understand that well. And then we put in
 connectivity, which is part guesswork, part observed. And of course, if you do not have any
 theory about how it is supposed to work, we just have to take whatever comes out of it as, okay,
 this is something interesting. But in your sense, like these models of the way signal travels
 along, like with the axons and all the basic models, they're too crude.
 Oh, well, actually, they are pretty detailed and pretty sophisticated. And they do replicate
 the neural dynamics. If you take a single neuron and you try to turn on the different channels,
 the calcium channels and the different receptors, and see what the effect of turning off those
 channels are in the neurons spike output, people have built pretty sophisticated models of that.
 And they are, I would say, in the regime of correct.
 Well, see, the correctness, that's interesting, because you mentioned it at several levels.
 The correctness is measured by looking at some kind of aggregate statistics?
 It would be more the spiking dynamics of a single neuron.
 Spiking dynamics of a single neuron, okay.
 And yeah, these models, because they are going to the level of mechanism, right? So they are
 basically looking at, okay, what is the effect of turning on an ion channel? And you can model that
 using electric circuits. So it is not just a function fitting. People are looking at the
 mechanism underlying it and putting that in terms of electric circuit theory, signal propagation
 theory, and modeling that. And so those models are sophisticated, but getting a single neuron
 model 99% right does not still tell you how to... It would be the analog of getting a transistor
 model right and now trying to build a microprocessor. And if you did not understand
 how a microprocessor works, but you say, oh, I now can model one transistor well, and now I will just
 try to interconnect the transistors according to whatever I could guess from the experiments
 and try to simulate it, then it is very unlikely that you will produce a functioning microprocessor.
 When you want to produce a functioning microprocessor, you want to understand
 Boolean logic, how do the gates work, all those things, and then understand how do those gates
 get implemented using transistors. Yeah. There is actually... This reminds me, this is a paper,
 maybe you are familiar with it, that I remember going through in a reading group that approaches
 a microprocessor from a perspective of a neuroscientist. I think it basically uses all the tools that we have of neuroscience to try to understand,
 like as if we just aliens showed up to study computers and to see if those tools could be used to get any kind of sense of how the microprocessor works.
 I think the final takeaway from at least this initial exploration is that we are screwed.
 There is no way that the tools of neuroscience would be able to get us to anything, like not even Boolean logic.
 I mean, it is just any aspect of the architecture of the function of the processes involved,
 the clocks, the timing, all that, you can't figure that out from the tools of neuroscience.
 Yes, I am very familiar with this particular paper. I think it was called,
 Can a Neuroscientist Understand a Microprocessor? Something like that.
 Following the methodology in that paper, even an electrical engineer would not understand microprocessors.
 I don't think it is that bad in the sense of saying, neuroscientists do find valuable things by observing the brain.
 They do find good insights, but those insights cannot be put together just as a simulation.
 You have to investigate what are the computational underpinnings of those findings.
 How do all of them fit together from an information processing perspective?
 Somebody has to painstakingly put those things together and build hypotheses.
 So, I don't want to have all of these neuroscientists saying, oh, they are not finding anything.
 No, that paper almost went to that level of neuroscientists will never understand.
 That's not true. I think they do find lots of useful things, but it has to be put together in a computational framework.
 The AI systems will be listening to this podcast a hundred years from now.
 There is some non-zero probability they will find your words laughable.
 I remember humans thought they understood something about the brain that are totally clueless.
 There is a sense about neuroscience that we may be in the very early days of understanding the brain.
 But, I mean, that's one perspective. In your perspective, how far are we into understanding any aspect of the brain?
 So, the dynamics of the individual neuron communication to how in a collective sense,
 how they are able to store information, transfer information, how the intelligence then emerges, all that kind of stuff.
 Where are we on that timeline?
 Yeah. So, timelines are very, very hard to predict. And you can, of course, be wrong.
 And you can be wrong on either side. We know that when we look back, the first flight was in 1903.
 In 1900, there was a New York Times article on flying machines that do not fly.
 And humans might not fly for another hundred years. That was what that article stated.
 But no, they flew three years after that. So, it's very hard to...
 Well, and on that point, one of the Wright brothers, I think two years before, said that,
 like he said, some number, like 50 years, he has become convinced that it's impossible.
 Even during their experimentation. Yeah.
 I mean, that's a tribute to when... That's like the entrepreneurial battle of depression,
 of going through just thinking, this is impossible. But there, yeah, there's something...
 Even the person that's in it is not able to see, estimate correctly.
 Exactly. But I can tell from the point of objectively what are the things that we
 know about the brain and how that can be used to build AI models, which can then go back and
 inform how the brain works. So, my way of understanding the brain would be to basically say,
 look at the insights neuroscientists have found, understand that from a computational angle,
 information processing angle, build models using that. And then building that model,
 which is a functional model, which is doing the task that we want the model to do. It is not just
 trying to model a phenomena in the brain. It is trying to do what the brain is trying to do on
 the whole functional level. And building that model will help you fill in the missing pieces
 that biology just gives you the hints. And building the model fills in the rest of the
 pieces of the puzzle. And then you can go and connect that back to biology and say, okay, now
 it makes sense that this part of the brain is doing this, or this layer in the cortical circuit
 is doing this. And then continue this iteratively, because now that will inform new experiments in
 neuroscience. And of course, building the model and verifying that in the real world will also
 tell you more about does the model actually work? And you can refine the model, find better ways of
 putting these neuroscience insights together. So, I would say it is... Neuroscientists alone,
 just from experimentation, will not be able to build a functional model of the brain.
 So, there's lots of efforts, which are very impressive efforts in collecting more and more
 connectivity data from the brain. How are the microcircuits of the brain connected with each
 other? Those are beautiful, by the way. Those are beautiful. And at the same time, those do not
 itself, by themselves, convey the story of how does it work? And somebody has to understand,
 okay, why are they connected like that? And what are those things doing? And we do that by building
 models in AI, using hints from neuroscience, and repeat the cycle. So, what aspect of the brain
 are useful in this whole endeavor? Which, by the way, I should say, you're both a neuroscientist
 and an AI person. I guess the dream is to both understand the brain and to build AGI systems.
 So, it's like an engineer's perspective of trying to understand the brain. So, what aspects of the
 brain functionally speaking, like you said, do you find interesting? Yeah, quite a lot of things.
 All right. So, one is, if you look at the visual cortex, and the visual cortex is a large part of
 the brain, I forget the exact fraction, but a huge part of our brain area is occupied by just vision.
 So, visual cortex is not just a feed-forward cascade of neurons. There are a lot more feedback
 connections in the brain compared to the feed-forward connections. And it is surprising
 to the level of detail neuroscientists have actually studied this. If you go into neuroscience
 literature and poke around and ask, have they studied what will be the effect of poking a neuron
 in level IT in level V1? And have they studied that? And you will say, yes, they have studied that.
 So, every possible combination. I mean, it's not a random exploration at all. It's very
 hypothesis-driven. Experimental neuroscientists are very, very systematic in how they probe the
 brain because experiments are very costly to conduct. They take a lot of preparation. They
 need a lot of control. So, they are very hypothesis-driven in how they probe the brain.
 And often what I find is that when we have a question in AI about, has anybody probed how
 lateral connections in the brain works? And when you go and read the literature, yes, people have
 probed it and people have probed it very systematically. And they have hypotheses about
 how those lateral connections are supposedly contributing to visual processing. But of course,
 they haven't built very, very functional detailed models of it. By the way, how do the studies,
 sorry to interrupt, do they stimulate like a neuron in one particular area of the visual cortex
 and then see how the signal travels kind of thing? Very, very fascinating experiments.
 So, I can give you one example I was impressed with. So, before going to that, let me give you
 overview of how the layers in the cortex are organized. Visual cortex is organized into
 roughly four hierarchical levels. So, V1, V2, V4, IT. And in V1- What happened to V3?
 Well, yeah, there's another pathway. Okay. So, I'm talking about just object recognition pathway.
 All right, cool. And then in V1 itself,
 there is a very detailed micro circuit in V1 itself. There is organization within a level
 itself. The cortical sheet is organized into multiple layers and there are columnar structure.
 And this layer-wise and columnar structure is repeated in V1, V2, V4, IT, all of them.
 And the connections between these layers within a level, in V1 itself, there are six layers roughly,
 and the connections between them, there is a particular structure to them.
 And now, so one example of an experiment people did is when you present a stimulus, which is,
 let's say, requires separating the foreground from the background of an object. So, it's a
 textured triangle on a textured background. And you can check, does the surface settle first,
 or does the contour settle first? Settle?
 Settle in the sense that when you finally form the percept of the triangle, you understand where the
 contours of the triangle are, and you also know where the inside of the triangle is, right? That's
 when you form the final percept. Now, you can ask, what is the dynamics of forming that final percept?
 Do the neurons first find the edges and converge on where the edges are, and then they find the
 inner surfaces, or does it go the other way around? The other way around. So, what's the answer?
 In this case, it turns out that it first settles on the edges, it converges on the edge hypothesis
 first, and then the surfaces are filled in from the edges to the inside. That's fantastic.
 And the detail to which you can study this, it's amazing that you can actually not only find
 the temporal dynamics of when this happens, and then you can also find which layer in V1,
 which layer is encoding the edges, which layer is encoding the surfaces, and which layer is encoding
 the feedback, which layer is encoding the feedforward, and what's the combination of
 them that produces the final person. And these kinds of experiments stand out when you try to
 explain illusions. One example of a favorite illusion of mine is the Kinesa triangle. I don't
 know that you are familiar with this one. So, this is an example where it's a triangle, but
 only the corners of the triangle are shown in the stimulus. So, they look like kind of Pac-Man.
 Oh, the black Pac-Man. Yeah. And then you start to see.
 Your visual system hallucinates the edges. And when you look at it, you will see a faint edge.
 And you can go inside the brain and look, do actually neurons signal the presence of this edge?
 And if they signal, how do they do it? Because they are not receiving anything from the input.
 The input is black for those neurons. So, how do they signal it? When does the signaling happen?
 So, if a real contour is present in the input, then the neurons immediately signal,
 oh, okay, there is an edge here. When it is an illusory edge, it is clearly not in the input.
 It is coming from the context. So, those neurons fire later. And you can say that, okay,
 it's the feedback connections that is causing them to fire. And they happen later. And you can
 find the dynamics of them. So, these studies are pretty impressive and very detailed.
 So, by the way, just to step back, you said that there may be more feedback connections
 than feedforward connections. Yeah.
 First of all, just for like a machine learning folks, I mean, that's crazy that there's all
 these feedback connections. We often think about, thanks to deep learning, you start to think about
 the human brain as a kind of feedforward mechanism. So, what the heck are these
 feedback connections? Yeah. What's the dynamics? What are we supposed to think about them?
 Yeah. So, this fits into a very beautiful picture about how the brain works. So,
 the beautiful picture of how the brain works is that our brain is building a model of the world.
 So, our visual system is building a model of how objects behave in the world. And we are
 constantly projecting that model back onto the world. So, what we are seeing is not just a
 feedforward thing that just gets interpreted in a feedforward part. We are constantly projecting
 our expectations onto the world. And what the final percept is a combination of what we project
 onto the world combined with what the actual sensory input is.
 Almost like trying to calculate the difference and then trying to interpret the difference.
 Yeah. I wouldn't put it as calculating the difference. It's more like what is the best
 explanation for the input stimulus based on the model of the world I have?
 Got it. Got it. And that's where all the illusions come in. But that's an incredibly efficient
 process. So, the feedback mechanism, it just helps you constantly, yeah. So, hallucinate how
 the world should be based on your world model and then just looking at if there's novelty,
 like trying to explain it. Yeah.
 Hence, that's why movement, we detect movement really well. There's all these kinds of things.
 And this is like at all different levels of the cortex you're saying.
 Every. This happens at the lowest level,
 at the highest level. Exactly. Yes. Yeah. In fact,
 feedback connections are more prevalent everywhere in the cortex. And so, one way to think about it,
 and there's a lot of evidence for this, is inference. So, basically, if you have a model
 of the world and when some evidence comes in, what you are doing is inference. You are trying to now
 explain this evidence using your model of the world. And this inference includes projecting
 your model onto the evidence and taking the evidence back into the model and doing an
 iterative procedure. And this iterative procedure is what happens using the feed-forward feedback
 propagation. And feedback affects what you see in the world and it also affects feed-forward
 propagation. And examples are everywhere. We see these kinds of things everywhere.
 The idea that there can be multiple competing hypotheses in our model trying to explain the
 same evidence. And then you have to kind of make them compete. And one hypothesis will explain away
 the other hypothesis through this competition process.
 So, you have competing models of the world that try to explain. What do you mean by explain away?
 So, this is a classic example in graphical models, probabilistic models.
 What are those? I think it's useful to mention it because we'll talk about them more.
 Yeah. So, neural networks are one class of machine learning models. You have a distributed set of
 nodes which are called the neurons. Each one is doing a dot product and you can approximate
 any function using this multi-level network of neurons. So, that's a class of models which are
 useful for function approximation. There is another class of models in machine learning
 called probabilistic graphical models. And you can think of them as each node in that
 model is variable, which is talking about something. It can be a variable representing
 is an edge present in the input or not. And at the top of the network, a node can be
 representing is there an object present in the world or not. And then, so it is another way of
 encoding knowledge. And then, once you encode the knowledge, you can do inference in the right way.
 What is the best way to explain some set of evidence using this model that you encoded?
 So, when you encode the model, you are encoding the relationship between these different variables.
 How is the edge connected to the model of the object? How is the surface connected to the model
 of the object? And then, of course, this is a very distributed complicated model. And inference is,
 how do you explain a piece of evidence when a set of stimulus comes in? If somebody tells me
 there is a 50% probability that there is an edge here in this part of the model, how does that
 affect my belief on whether I should think that there should be a square present in the image?
 So, this is the process of inference. So, one example of inference is having this
 experience of effect between multiple causes. So, graphical models can be used to represent
 causality in the world. So, let's say your alarm at home can be triggered by a burglar getting into
 your house, or it can be triggered by an earthquake. Both can be causes of the alarm going off.
 So, now you are in your office, you heard burglar alarm going off, you are heading home,
 thinking that there's a burglar. But while driving home, if you hear on the radio that there was an
 earthquake in the vicinity, now your strength of evidence for a burglar getting into their house
 is diminished. Because now that piece of evidence is explained by the earthquake being present.
 So, if you think about these two causes explaining at lower level variable, which is alarm, now what
 we are seeing is that increasing the evidence for some cause, there is evidence coming in from below
 for alarm being present. And initially it was flowing to a burglar being present. But now,
 since there is side evidence for this other cause, it explains away this evidence and evidence will
 now flow to the other cause. These are two competing causal things trying to explain the same evidence.
 And the brain has a similar kind of mechanism for doing so. That's kind of interesting.
 How is that all encoded in the brain? Where is the storage of information? Are we talking
 just maybe to get it a little bit more specific? Is it in the hardware of the actual connections?
 Is it in chemical communication? Is it electrical communication? Do we know?
 So, this is a paper that we are bringing out soon.
 Which one is this?
 This is the cortical microcircuits paper that I sent you a draft of. Of course, a lot of it is
 still hypothesis. One hypothesis is that you can think of a cortical column as encoding a concept.
 Think of it as an example of a concept. Is an edge present or not? Or is an object present or not?
 So, you can think of it as a binary variable, a binary random variable. The presence of an edge or
 not or the presence of an object or not. So, each cortical column can be thought of as representing
 that one concept, one variable. And then the connections between these cortical columns
 are basically encoding the relationship between these random variables. And then there are
 connections within the cortical column. Each cortical column is implemented using multiple
 layers of neurons with very, very rich structure there. There are thousands of neurons in a cortical
 column. But that structure is similar across the different cortical columns.
 Correct. And also these cortical columns connect to a substructure called thalamus in the
 structure. So, all cortical columns pass through this substructure. So, our hypothesis is that
 the connections between the cortical columns implement this. That's where the knowledge is
 stored about how these different concepts connect to each other. And then the neurons inside this
 cortical column and in thalamus in combination implement these actual computations needed for
 inference, which includes explaining away and competing between the different hypothesis.
 So, what is amazing is that neuroscientists have actually done experiments to the tune of showing
 these things. They might not be putting it in the overall inference framework, but they will show
 things like if I poke this higher level neuron, it will inhibit through this complicated loop
 through the thalamus, it will inhibit this other column. So, they will do such experiments.
 Do they use terminology of concepts, for example? Is it something where it's easy to anthropomorphize
 and think about concepts like you start moving into logic-based reasoning systems? So,
 are we just thinking of concepts in that kind of way? Or is it a lot messier, a lot more gray area,
 even more gray, even more messy than the artificial neural network kinds, kinds of abstractions?
 It's the easiest way to think of it as a variable. It's a binary variable,
 which is showing the presence or absence of something.
 But I guess what I'm asking is, is that something that we're supposed to think of something that's
 human interpretable, that something? It doesn't need to be. It doesn't need to be human
 interpretable. There's no need for it to be human interpretable. But it's almost like
 you will be able to find some interpretation of it because it is connected to the other things
 that you know about. And the point is it's useful somehow. It's useful as an entity in the graph,
 in connecting to the other entities that are, let's call them concepts. By the way, are these
 the cortical microcircuits? Correct. These are the cortical microcircuits. That's what neuroscientists
 use to talk about the circuits within a level of the cortex. So you can think of, let's think of
 a neural network, artificial neural network terms. People talk about the architecture of the,
 how many layers they build, what is the fan in, fan out, etc. That is the macro architecture.
 And then within a layer of the neural network, the cortical neural network is much more structured
 within a level. There's a lot more intricate structure there. But even within an artificial
 neural network, you can think of feature detection plus pooling as one level. And so that is kind of
 a microcircuit. It's much more complex in the real brain. And so within a level, whatever is that
 circuitry within a column of the cortex and between the layers of the cortex, that's the
 microcircuitry. I love that terminology. Machine learning people don't use the circuit terminology,
 but they should. That's nice. So okay. So that's the
 cortical microcircuit. So what's interesting about, what can we say, what is the paper that
 you're working on, propose about the ideas around these cortical microcircuits?
 So this is a fully functional model for the microcircuits of the visual cortex.
 So the paper focuses on your idea and our discussion now is focusing on
 vision, the visual cortex. Okay. This is a model. This is a full model. This is how vision works.
 But this is a model of- A hypothesis.
 Okay. So let me step back a bit. So we looked at neuroscience for insights on how to build a vision
 model. And we synthesized all those insights into a computational model. This is called the recursive
 cortical network model that we used for breaking captures. And we are using the same model for
 robotic picking and tracking of objects. And that again is a vision system.
 That's a vision system. Computer vision system.
 That's a computer vision system. Takes in images and outputs what?
 On one side, it outputs the class of the image and also segments the image. And you can also
 ask it further queries. Where is the edge of the object? Where is the interior of the object?
 So it's a model that you build to answer multiple questions. So you're not trying to build a model
 for just classification or just segmentation, et cetera. It's a joint model that can do
 multiple things. So that's the model that we built using insights from neuroscience. And some of
 those insights are what is the role of feedback connections? What is the role of lateral connections?
 So all those things went into the model. The model actually uses feedback connections.
 All these ideas from your science. So what the heck is a recursive cortical network?
 Like what are the architecture approaches, interesting aspects here,
 which is essentially a brain-inspired approach to a computer vision?
 Yeah. So there are multiple layers to this question. I can go from the very top and then zoom in.
 Okay. So one important thing, constraint that went into the model is that you should not think
 of vision as something in isolation. We should not think perception as something as a pre-processor
 for cognition. Perception and cognition are interconnected. And so you should not think of
 one problem in separation from the other problem. And so that means if you finally want to have a
 system that understands concepts about the world and can learn in a very conceptual model of the
 world and can reason and connect to language, all of those things, you need to think all the way
 through and make sure that your perception system is compatible with your cognition system and
 language system and all of them. And one aspect of that is top-down controllability.
 What does that mean? So that means...
 In this context. So think of... You can close your eyes
 and think about the details of one object. I can zoom in further and further. So think of the
 bottle in front of me. And now you can think about, okay, what the cap of that bottle looks.
 We can think about what's the texture on that bottle of the cap. You can think about what
 will happen if something hits that. So you can manipulate your visual knowledge in
 cognition-driven ways. And so this top-down controllability and being able to simulate
 scenarios in the world. So you're not just a passive player in this
 perception game. You can control it. You have imagination.
 Correct. So basically having a generative network, which is a model, and it is not just
 some arbitrary generated network. It has to be built in a way that it is controllable top-down.
 It is not just trying to generate a whole picture at once. It's not trying to generate photorealistic
 things of the world. You don't have good photorealistic models of the world. Human
 brains do not have. If I, for example, ask you the question, what is the color of the letter E in the
 Google logo? You have no idea. Although you have seen it millions of times, hundreds of times.
 Our model is not photorealistic, but it has other properties that we can manipulate it.
 You can think about filling in a different color in that logo. You can think about expanding the
 letter E. You can imagine the consequence of actions that you have never performed.
 These are the kind of characteristics the generative model needs to have.
 This is one constraint that went into our model. When you read just the perception side of the
 paper, it is not obvious that this was a constraint that went into the model, this top-down
 controllability of the generative model. What does top-down controllability in a model look like?
 It's a really interesting concept, fascinating concept. Is that recursive?
 Quite a few things. It's like what does the model factorize? What is the model representing
 as different pieces in the puzzle? In the RCN network, it thinks of the world. For example,
 the background of an image is modeled separately from the foreground of the image.
 The objects are separate from the background. They are different entities.
 There's a kind of segmentation that's built in fundamentally.
 Then even that object is composed of parts. Another one is the shape of the object is
 differently modeled from the texture of the object.
 Got it. You know who Francois Chalet is?
 Yes. He developed this IQ test type of thing for Arc Challenge. It's kind of cool that there's
 these concepts, priors that he defines that you bring to the table in order to be able to
 reason about basic shapes and things in an IQ test. Here you're making it quite explicit that
 these are distinct things that you should be able to model in this.
 Yes. Keep in mind that you can derive these from much more general principles.
 You don't need to explicitly put it as objects versus foreground versus background,
 the surface versus texture. These are derivable from more fundamental principles of
 what's the property of continuity of natural signals.
 What's the property of continuity of natural signals? By the way, that sounds very poetic.
 You're saying there's some low-level properties from which emerges the idea that shapes should
 be different. There should be parts of an object. There should be objects. There's objectness.
 There's all these things that it's kind of crazy that we humans evolved to have because it's useful.
 Correct. It derives mostly from the properties of natural signals.
 Yeah. Natural signals are the kind of things we'll perceive in the natural world.
 Correct. I don't know why that sounds so beautiful. Natural signals, yeah.
 As opposed to a QR code, which is an artificial signal that we created,
 humans are not very good at classifying QR codes. We are very good at saying something is a cat or
 a dog, but not very good at classifying. Computers are very good at classifying QR codes. Our visual
 system is tuned for natural signals. There are fundamental assumptions in the architecture
 that are derived from natural signals properties. I wonder when you take hallucinogenic drugs,
 does that go into natural or is that closer to the QR code? It's still natural.
 It's still natural. Yeah, because it is still operating using our brains.
 By the way, on that topic, I haven't been following. I think they're becoming legalized.
 I can't wait until they become legalized to a degree that vision science researchers could study
 it. Just through medical chemical ways, modify it. There could be ethical concerns, but that's
 another way to study. The brain is to be able to chemically modify it. There's probably very long
 a way to figure out how to do it ethically. Yeah, but I think there are studies on that
 already. I think so, because it's not unethical to give it to rats. Oh, that's true. That's true.
 There's a lot of drugged up rats out there. Okay, cool. Sorry. Sorry. It's okay. There's
 these low level things from natural signals that... From which these properties will emerge.
 Yes. But it is still a very hard problem on how to encode that. You mentioned the priors
 Franchot wanted to encode in the abstract reasoning challenge, but it is not straightforward how to
 encode those priors. Some of those challenges, the object completion challenges are things that we
 purely use our visual system to do. It looks like abstract reasoning, but it is purely an output of
 the vision system. For example, completing the corners of that Kansar triangle, completing the
 lines of that Kansar triangle. It's a purely visual system property. There is no abstract
 reasoning involved. It uses all these priors, but it is stored in our visual system in a particular
 way that is amenable to inference. That is one of the things that we tackled. It's basically saying,
 these are the prior knowledge which will be derived from the world. But then how is that
 prior knowledge represented in the model such that inference, when some piece of evidence comes in,
 can be done very efficiently and in a very distributed way? There are so many ways of
 representing knowledge which is not amenable to very quick inference, quick lookups. That's one
 core part of what we tackled in the RCN model. How do you encode visual knowledge to do very
 quick inference? Can you maybe comment on folks listening to this and in general may be familiar
 with different kinds of architectures of neural networks. What are we talking about with RCN?
 What does the architecture look like? What are the different components? Is it close to neural
 networks? Is it far away from neural networks? What does it look like? You can think of the
 delta between the model and a convolutional neural network if people are familiar with
 convolutional neural networks. Convolutional neural networks have this feed-forward processing
 cascade which is called feature detectors and pooling. That is repeated in a multi-level
 system. If you want an intuitive idea of what is happening, feature detectors are detecting
 interesting co-occurrences in the input. It can be a line, a corner, an eye, or a piece of texture
 etc. The pooling neurons are doing some local transformation of that and making it invariant
 to local transformations. This is what the structure of convolutional neural network is.
 Recursive cortical network has a similar structure when you look at just the feed-forward pathway.
 But in addition to that, it is also structured in a way that it is generative
 so that it can run it backward and combine the forward with the backward.
 Another aspect that it has is it has lateral connections.
 These lateral connections, so if you have an edge here and an edge here,
 it has connections between these edges. It is not just feed-forward connections.
 It is something between these edges which is the nodes are presenting these edges
 which is to enforce compatibility between them. Otherwise, what will happen is that-
 Like constraints? It is a constraint. Basically, if you do just feature detection followed by pooling,
 then your transformations in different parts of the visual field are not coordinated.
 So, when you generate from the model, you will create jagged things and
 uncoordinated transformations. So, these lateral connections are enforcing the transformations.
 Is the whole thing still differentiable? No.
 It is not trained using backprop. Okay, that is really important.
 So, there is this feed-forward, there is feedback mechanisms,
 there are some interesting connectivity things. It is still layered?
 Yes, there are multiple layers. Okay, very interesting.
 The interconnection between adjacent connections across service constraints
 that keep the thing stable. Correct.
 Okay, so what else? Then there is this idea of doing inference.
 A neural network does not do inference on the fly. An example of why this inference is important is
 one of the first applications that we showed in the paper was to crack text-based captures.
 What are captures, by the way? By the way, one of the most awesome-
 People do not use this term anymore. It is human computation, I think.
 I love this term. The guy who created captures, I think, came up with this term.
 Yeah. I love it. Anyway, what are captures?
 So, captures are those strings that you fill in. If you are opening a new account in Google,
 they show you a picture. It used to be a set of garbled letters that you have to figure out what
 is that string of characters and type it. The reason captures exist is because Google or Twitter
 do not want automatic creation of accounts. You can use a computer to create millions of accounts
 and use that for nefarious purposes. So, you want to make sure that, to the extent possible,
 the interaction that their system is having is with a human. So, it's called a human interaction
 proof. A capture is a human interaction proof. So, this captures are, by design, things that are
 easy for humans to solve, but hard for computers. Hard for robots, yeah.
 Yeah. Text-based captures was the one which is prevalent around 2014, because at that time,
 text-based captures were hard for computers to crack. Even now, they are actually, in the sense
 of an arbitrary text-based capture will be unsolvable even now. But with the techniques
 that we have developed, you can quickly develop a mechanism that solves the capture.
 They've probably gotten a lot harder too. People have been getting clever and clever
 at generating these text captures. Yeah. Correct.
 Okay. So, that was one of the things you've tested on is these kinds of captures in 2014,
 2015, that kind of stuff. So, by the way, why captures?
 Yeah. Even now, I would say capture is a very, very good challenge problem if you want to
 understand how human perception works and if you want to build systems that work like the human
 brain. And I wouldn't say capture is a solved problem. We have cracked the fundamental
 defense of captures, but it is not solved in the way that humans solve it. So, I can give an example.
 I can take a five-year-old child who has just learned characters and show them any new capture
 that we create. They will be able to solve it. I can show you pretty much any new capture from
 any new website. You'll be able to solve it without getting any training examples from that particular
 style of capture. You're assuming I'm human, yeah.
 Yes. Yeah. That's right. So, if you are human, otherwise I will be able to figure that out
 using this one. But... This whole podcast is just a touring test.
 That's right. A long touring test. Anyway, I'm sorry. So, yeah. So,
 humans can figure it out with very few examples. Or no training examples. No training examples from
 that particular style of capture. Even now, this is unreachable for the current deep learning system.
 So, basically, I don't think a system exists where you can basically say, train on whatever you want.
 And then now say, hey, I will show you a new capture, which I did not show you in the training
 setup. Will the system be able to solve it? It still doesn't exist. So, that is the magic of human
 perception. And Doug Huffstater put this very beautifully in one of his talks. The central
 problem in AI is what is the letter A. If you can build a system that reliably can detect all the
 variations of the letter A, you don't even need to go to the... The B and the C.
 Yeah. You don't even need to go to the B and the C or the strings of characters.
 And so, that is the spirit with which we tackle that problem.
 What does it mean by that? I mean, is it like without training examples, try to figure out
 the fundamental elements that make up the letter A in all of its forms?
 In all of its forms. A can be made with two humans standing leaning against each other,
 holding the hands. And it can be made of leaves. It can be...
 Yeah. You might have to understand everything about this world in order to understand the
 letter A. Yeah. Exactly.
 So, it's common sense reasoning, essentially. Yeah.
 Right. So, to finally, to really solve, finally to say that you have solved capture,
 you have to solve the whole problem. Yeah. Okay. So, how does this kind of
 the RCN architecture help us to get... Do a better job of that kind of thing?
 So, as I mentioned, one of the important things was being able to do inference,
 being able to dynamically do inference. Can you clarify what you mean?
 Because you said neural networks don't do inference.
 Yeah. So, what do you mean by inference in this context?
 So, okay. So, in captures, what they do to confuse people is to make these characters
 crowd together. Yes.
 Okay. And when you make the characters crowd together, what happens is that you will now
 start seeing combinations of characters as some other new character or an existing character.
 So, you would put an R and N together, it will start looking like an M.
 And so, locally, there is very strong evidence for it being some incorrect character.
 But globally, the only explanation that fits together is something that is different from
 what you can find locally. So, this is inference. You are basically taking local evidence and
 putting it in the global context and often coming to a conclusion locally, which is
 conflicting with the local information. So, you mean inference in the way it's used
 when you talk about reasoning, for example, as opposed to inference with artificial neural
 networks, which is a single pass through the network. Okay. So, you're basically doing some
 basic forms of reasoning, like integration of how local things fit into the global picture.
 Right. And things like explaining away coming into this one, because you are explaining that
 piece of evidence as something else, because globally, that's the only thing that makes sense.
 So, now, you can amortize this inference in a neural network. If you want to do this,
 you can brute force it. You can just show it all combinations of things that you want your
 reasoning to work over. And you can just train the help out of that neural network. And it will
 look like it is doing inference on the fly, but it is really just doing amortized inference.
 It is because you have shown it a lot of these combinations during training time.
 So, what you want to do is be able to do dynamic inference rather than just being able to show all
 those combinations in the training time. And that's something we emphasized in the model.
 What does it mean dynamic inference? Does that have to do with the feedback thing?
 Yes.
 I'm trying to visualize what dynamic inference would be in this case. What is it doing with the
 input? It's showing the input the first time. What's changing over temporally? What's the
 dynamics of this inference process? So, you can think of it as you have,
 at the top of the model, the characters that you are trained on. They are the causes. You're trying
 to explain the pixels using the characters as the causes. The characters are the things that cause
 the pixels. Yeah. So, there's this causality thing. So, the reason you mentioned causality,
 I guess, is because there's a temporal aspect to this whole thing.
 In this particular case, the temporal aspect is not important. It is more like
 when if I turn the character on, the pixels will turn on. Yeah. It will be after there's a little
 bit, but yeah. So, it's causality in the sense of like a logic causality, like hence inference.
 Okay. The dynamics is that even though locally it will look like, okay, this is an A.
 And locally, when I look at just that patch of the image, it looks like an A. But when I look at it
 in the context of all the other causes, it might not, A is not something that makes sense. So,
 that is something you have to kind of recursively figure out. Yeah. So, okay. And this thing
 performed pretty well on the captures. Correct. And I mean, is there some kind of interesting
 intuition you can provide? Why it did well? What did it look like? Is there visualizations that
 could be human interpretable to us humans? Yes. Yeah. So, the good thing about the model is that
 it is extremely... So, it is not just doing a classification. It is providing a full explanation
 for the scene. So, when it operates on a scene, it is coming back and saying, look, this is the part
 is the A, and these are the pixels that turned on. These are the pixels in the input that makes me
 think that it is an A. And also, these are the portions I hallucinated. It provides a complete
 explanation of that form. And then, these are the contours. This is the interior, and this is in
 front of this other object. So, that's the kind of explanation the inference network provides.
 So, that is useful and interpretable. And then, the kind of errors it makes are also...
 I don't want to read too much into it, but the kind of errors the network makes are very similar
 to the kinds of errors humans would make in a similar situation. So, there's something about
 the structure that feels reminiscent of the way humans visual system works. Well, I mean,
 how hard-coded is this to the capture problem, this idea? Not really hard-coded because it's the
 assumptions, as I mentioned, are general, right? And those themselves can be applied in many
 situations which are natural signals. So, it's the foreground versus background factorization,
 and the factorization of the surfaces versus the contours. So, these are all generally applicable
 assumptions. In all vision. So, why attack the capture problem, which is quite interesting.
 Why attack the capture problem, which is quite unique in the computer vision context versus
 the traditional benchmarks of ImageNet and all those kinds of image classification, or even
 segmentation tasks and all that kind of stuff. Do you feel like that's... I mean, what's your
 thinking about those kinds of benchmarks in this context? I mean, those benchmarks are useful for
 algorithms where you... So, the settings that deep learning works in are, here is my huge training
 set, and here is my test set. So, the training set is almost 100x, 1000x bigger than the test set in
 many cases. What we wanted to do was invert that. The training set is way smaller than the test set.
 Yes. And capture is a problem that is by definition hard for computers, and it has these
 good properties of strong generalization, strong out of training distribution generalization. If
 you are interested in studying that and having your model have that property, then it's a good
 data set to tackle. So, have you attempted to, which I think I believe there's quite a growing
 body of work on looking at MNIST and ImageNet without training? So, taking... The basic challenge
 is what tiny fraction of the training set can we take in order to do a reasonable job of the
 classification task? Have you explored that angle in these classic benchmarks?
 Yes. So, we did do MNIST. So, it's not just capture. So, there was also
 multiple versions of MNIST, including the standard version, where we inverted the problem,
 which is basically saying rather than train on 60,000 training data, how quickly can you get
 to high level accuracy with very little training data?
 Is there some performance that you remember? How well did it do? How many examples did it need?
 Yeah. I remember that it was on the order of
 tens or hundreds of examples to get into 95% accuracy, and it was definitely better than
 the other systems out there at that time. At that time. Yeah. Yeah. They're really
 pushing it. I think that's a really interesting space, actually. I think there's an actual name
 for MNIST that... There's different names to the different sizes of training sets. I mean,
 people are attacking this problem. I think it's super interesting. It's funny how the MNIST will
 probably be with us all the way to AGI as a data set that just sticks by. It's a clean, simple
 data set to study the fundamentals of learning with. Just like CAPTCHAs, it's interesting.
 Correct. Not enough people... I don't know, maybe you can correct me, but I feel like CAPTCHAs don't
 show up as often in papers as they probably should. That's correct. Yeah. Because usually these
 things have a momentum. Once something gets established as a standard benchmark, there is
 a dynamics of how graduate students operate and how the academic system works that pushes people
 to track that benchmark. Nobody wants to think outside the box. Okay. Good performance on the
 CAPTCHAs. What else is there interesting on the RCN side before we talk about the cortical
 microswitch? Yeah. The same model. The important part of the model was that it trains very quickly
 with very little training data. It's quite robust to out of distribution perturbations.
 We are using that very fruitfully in... Advocatious in many of the robotic tasks we are solving.
 Let me ask you this touchy question. I have spoken with your friend, colleague, Jeff Hawkins,
 too. I have to ask, whenever you have brain-inspired stuff and you make big claims,
 big sexy claims, there's critics, machine learning subreddit. Don't give me that name.
 Machine learning subreddit. Don't give me started on those people. Criticism is good, but they're
 a bit over the top. There is quite a bit of skepticism and criticism. Is this work
 really as good as it promises to be? Do you have thoughts on that kind of skepticism?
 Do you have comments on the kind of criticism we might have received about, is this approach
 legit? Is this a promising approach? Or at least as promising as it seems to be advertised as?
 Yeah, I can comment on it. Our Arsene paper is published in Science, which I would argue is a
 very high quality journal, very hard to publish in. Usually it is indicative of the quality of the
 work. I am very, very certain that the ideas that we brought together in that paper, in terms of
 the importance of feedback connections, recursive inference, lateral connections,
 coming to best explanation of the scene as the problem to solve, trying to solve recognition,
 segmentation, all jointly in a way that is compatible with higher level cognition, top-down
 attention, all those ideas that we brought together into something coherent and workable
 in the world and tackling a challenging problem. I think that will stay and that contribution I
 stand by. Now, I can tell you a story which is funny in the context of this. If you read the
 abstract of the paper and the argument we are putting in, look, current deep learning systems
 take a lot of training data. They don't use these insights. Here is our new model, which is not a
 deep neural network. It's a graphical model. It does inference. This is how the paper is.
 Once the paper was accepted and everything, it went to the press department in science,
 AAAS, science office. We didn't do any press release when it was published. It went to the
 press department. What was the press release that they wrote up? A new deep learning model.
 Solves captures. Solves captures. You can see what was being hyped in that thing.
 There is a dynamic in the community. That especially happens when there are lots of new
 people coming into the field and they get attracted to one thing. Some people are trying to think
 different compared to that. I think skepticism is important in science and it is very much
 required. But it's also not skepticism. Usually, it's mostly bandwagon effect that is happening.
 But that's not even that. I'll tell you what they react to, which I'm sensitive to as well.
 If you look at just companies, OpenAI, DeepMind, Vicarious, there's a little bit of a race
 to the top and hype. It doesn't pay off to be humble.
 And the press is just irresponsible often. Don't get me started on the state of journalism today.
 It seems like the people who write articles about these things, they literally have not even spent
 an hour on the Wikipedia article about what is neural networks. They haven't invested just even
 the language to laziness. It's like robots beat humans. They write this kind of stuff.
 The researchers are quite sensitive to that because it gets a lot of attention. They're like,
 why did this work get so much attention? That's over the top and people get really sensitive.
 The same kind of criticism with OpenAI did work with Rubik's Cube, with the robot that people
 criticized. Same with GPT-2 and 3. They criticized the same thing with DeepMinds, with AlphaZero.
 I'm sensitive to it. And of course, with your work, you mentioned deep learning, but there's
 something super sexy to the public about brain inspired. That immediately grabs people's
 imagination. Not even neural networks, but really brain inspired, brain like neural networks.
 That seems really compelling to people and to me as well, to the world as a narrative.
 People hook onto that and sometimes the skepticism engine turns on in the research community and
 they're skeptical. But I think putting aside the ideas of the actual performance on CAPTCHAs or
 performance on any data set, to me, all these data sets are useless anyway. It's nice to have them,
 but in the grand scheme of things, they're silly toy examples. The point is, is there intuition
 about the ideas, just like you mentioned, bringing the ideas together in a unique way?
 Is there something there? Is there some value there? And is it going to stand the test of time?
 My confidence there is very high. I don't treat brain inspired as a marketing term.
 I am looking into the details of biology and puzzling over those things and I am grappling
 with those things. It is not a marketing term at all. You can use it as a marketing term and
 people often use it and you can get combined with them. When people don't understand how we're
 approaching the problem, it is easy to be misunderstood and think of it as purely marketing.
 But that's not the way we are. As a scientist, you believe that if we just stick to really
 understanding the brain, you should constantly meditate on how does the brain do this?
 Because that's going to be really helpful for engineering intelligent systems.
 I think it's one input and it is helpful, but you should know when to deviate from it too.
 An example is convolutional neural networks. Convolution is not an operation brain
 implements. The visual cortex is not convolutional. The visual cortex has local receptive fields,
 local connectivity, but there is no translation invariance in the network weights in the visual
 cortex. That is a computational trick, which is a very good engineering trick that we use
 for sharing the training between the different nodes. That trick will be with us for some time.
 It will go away when we have robots with eyes and heads that move. Then that trick will go away.
 It will not be useful at that time. The brain doesn't have translational
 invariance. It has a focal point. It has a thing it focuses on. It has a fovea. Because of the fovea,
 the receptive fields are not like the copying of the weights. The weights in the center are
 very different from the weights in the periphery. I did this. I actually wrote a paper and just
 got an chance to really study peripheral vision, which is a fascinating thing. Very
 under-understood thing of what the brain does at every level with the periphery. It does some funky
 stuff. It's another kind of trick than convolutional. Convolution in neural networks is a trick for
 efficiency. The brain does a whole other kind of thing, I guess.
 You need to understand the principles or processing so that you can still apply engineering tricks
 where you want to do. You don't want to be slavishly mimicking all the things of the brain.
 It should be one input. I think it is extremely helpful. It should be the point of really
 understanding so that you know when to deviate from it.
 Okay. That's really cool. That's work from a few years ago. You did work in Umenta with Jeff
 Hawkins with hierarchical temporal memory. If you could give a brief history, how has your view
 of the way the models of the brain changed over the past few years leading up to now?
 Is there some interesting aspects where there was an adjustment to your understanding of the brain,
 or is it all just building on top of each other? In terms of the higher level ideas,
 especially the ones Jeff wrote about in the book, if you blur out the details and if you just zoom
 out and at the higher level idea, things are, I would say, consistent with what he wrote about.
 But many things will be consistent with that because deep learning systems are also multi-level,
 hierarchical, all of those things. But in terms of the detail, a lot of things are different,
 and those details matter a lot. One point of difference I had with Jeff
 was how much of biological plausibility and realism do you want in the learning algorithms?
 So when I was there, this was almost 10 years ago now.
 That flies when you're having fun.
 I don't know what Jeff thinks now, but 10 years ago, the difference was that I did not want to
 be so constrained on saying my learning algorithms need to be biologically plausible
 based on some filter of biological plausibility available at that time. To me, that is a
 dangerous cut to make because we are discovering more and more things about the brain all the time.
 New biophysical mechanisms, new channels are being discovered all the time. So I don't want to
 up front kill off a learning algorithm just because we don't really understand the full
 biophysics of how the brain learns.
 Exactly.
 Let me ask Sergeant Chaurabh, what's your sense? What's our best understanding of how the brain
 learns?
 Things like back propagation, credit assignment. Many of these learning algorithms have things in
 common. Back propagation is one way of credit assignment. There is another algorithm called
 expectation maximization, which is another weight adjustment algorithm.
 But is your sense the brain does something like this?
 Has to. There is no way around it in the sense of saying that you do have to adjust the connections.
 So you're saying credit assignment, you have to reward the connections that were useful in
 making a correct prediction. I guess it doesn't have to be differentiable.
 Yeah, it doesn't have to be differentiable. But you have a model that you start with,
 you have data comes in, and you have to have a way of adjusting the model such that it better
 fits the data. So that is all of learning. And some of them can be using backprop to do that.
 Some of it can be using very local graph changes to do that. Many of these learning algorithms have
 similar update properties locally in terms of what the neurons need to do locally.
 I wonder if small differences in learning algorithms can have huge differences in the
 actual effect. So the dynamics of sort of the reverse spiking, if credit assignment is like
 a lightning versus a rainstorm or something, whether there's a looping local type of situation
 with the credit assignment, whether there is regularization, how it injects robustness into
 the whole thing, whether it's chemical or electrical or mechanical, all those kinds of things.
 I feel like those differences could be essential, right?
 It could be. It's just that you don't know enough to, on the learning side,
 you don't know enough to say that is definitely not the way the brain does it.
 Got it. So you don't want to be stuck to it. So you've been open-minded on that side of things.
 Correct. On the inference side, on the recognition side, I am much more amenable to being constrained
 because it's much easier to do experiments because it's like, okay, here's the stimulus.
 How many steps did it get to take the answer? I can trace it back. I can understand the speed
 of that computation, et cetera, much more readily on the inference side.
 Got it. And then you can't do good experiments on the learning side.
 Correct.
 So let's go right into the cortical microcircuits right back. So what are these ideas beyond
 recursive cortical network that you're looking at now?
 So we have made a pass through multiple of the steps that, as I mentioned earlier,
 we were looking at perception from the angle of cognition. It was not just perception for
 perception sake. How do you connect it to cognition? How do you learn concepts? And
 how do you learn abstract reasoning similar to some of the things Francois talked about?
 So we have taken one pass through it, basically saying, what is the basic cognitive architecture
 that you need to have, which has a perceptual system, which has a system that learns dynamics
 of the world, and then has something like a routine program learning system on top of it
 to learn concepts. So we have built the version point one of that system,
 this was another science robotics paper. The title of that paper was something like cognitive
 programs. How do you build cognitive programs? And the application there was on manipulation,
 robotic manipulation?
 It was, so think of it like this. Suppose you wanted to tell a new person that you met,
 you don't know the language that person uses, you want to communicate to that person to achieve
 some task. So I want to say, hey, you need to pick up all the red cups from the kitchen counter
 and put it here. How do you communicate that? You can show pictures, you can basically say,
 look, this is the starting state, the things are here, this is the ending state, how do you
 communicate that? And what does the person need to understand from that? The person needs to
 understand what conceptually happened in those pictures from the input to the output. So we are
 looking at pre-verbal conceptual understanding. Without language, how do you have a set of
 concepts that you can manipulate in your head? And from a set of images of input and output,
 can you infer what is happening in those images?
 Got it. With concepts that are pre-language, okay. So what does it mean for a concept to be pre-language?
 Yeah.
 Why is language so important here?
 So I want to make a distinction between concepts that are just learned from text by just feeding
 brute force text. You can start extracting things like, okay, cow is likely to be on grass.
 So those kinds of things you can extract purely from text. But that's kind of a simple association
 thing rather than a concept as an abstraction of something that happens in the real world
 in a grounded way that I can simulate it in my mind and then I can use it to make
 it in my mind and connect it back to the real world.
 And you think kind of the visual world, concepts in the visual world are somehow lower level
 than just the language?
 The lower level kind of makes it feel like, okay, that's like an unimportant, like it's
 more like I would say the concepts in the visual and the motor system and the concept
 learning system, which if you cut off the language part, just what we learn by interacting
 with the world and abstractions from that, that is a prerequisite for any real language
 understanding.
 So you disagree with Chomsky because he says language is at the bottom of everything.
 No, I disagree with Chomsky completely on how many levels from universal grammar to
 yeah.
 So that was a paper in science beyond the recursive cortical network. What other interesting
 problems are there, the open problems and brain inspired approaches that you're thinking
 about?
 I mean, everything is open, right? Like no problem is solved, solved. I think of perception
 as kind of the first thing that you have to build, but the last thing that you will be
 actually solved. Because if you do not build perception system in the right way, you cannot
 build concept system in the right way. So you have to build a perception system, however
 wrong that might be, you have to still build that and learn concepts from there and then
 keep iterating. And finally, perception will get solved fully when perception, cognition,
 language, all those things work together finally.
 So great, we've talked a lot about perception, but then maybe on the concept side and like
 common sense or just general reasoning side, is there some intuition you can draw from
 the brain about how we could do that?
 So I have this classic example I give. So suppose I give you a few sentences and then
 ask you a question following that sentence. This is a natural language processing problem,
 right? So here it goes. I'm telling you, Sally pounded a nail on the ceiling. Okay, that's
 a sentence. Now I'm asking you a question, was the nail horizontal or vertical?
 Vertical.
 Okay, how did you answer that?
 Well, I imagined Sally, it was kind of hard to imagine what the hell she was doing, but
 I imagined a visual of the whole situation.
 Exactly, exactly. So here, I post a question in natural language. The answer to that question
 was you got the answer from actually simulating the scene. Now I can go more and more detail
 about, okay, was Sally standing on something while doing this? Could she have been standing
 on a light bulb to do this? I could ask more and more questions about this and I can ask,
 make you simulate the scene in more and more detail, right? Where is all that knowledge
 that you're accessing stored? It is not in your language system. It was not just by reading
 text you got that knowledge. It is stored from the everyday experiences that you have
 had. By the age of five, you have pretty much all of this, right? It is stored in your visual
 system, motor system, in a way such that it can be accessed through language.
 Got it. I mean, right. So the language is just almost sort of the query into the whole
 visual cortex and that does the whole feedback thing. But I mean, it is all reasoning kind
 of connected to the perception system in some way.
 You can do a lot of it. You can still do a lot of it by quick associations without having
 to go into the depth. And most of the time you will be right, right? You can just do
 quick associations, but I can easily create tricky situations for you where that quick
 association is wrong and you have to actually run the simulation.
 So figuring out how these concepts connect, do you have a good idea of how to do that?
 That's exactly what one of the problems that we are working on. And the way we are approaching
 that is basically saying, okay, you need to... So the takeaway is that language is simulation
 control and your perceptual plus motor system is building a simulation of the world. And
 so that's basically the way we are approaching it. And the first thing that we built was
 a controllable perceptual system and we built a schema networks, which was a controllable
 dynamic system. Then we built a concept learning system that puts all these things together
 into programs or subtractions that you can run and simulate. And now we are taking the
 step of connecting it to language. And it will be very simple examples initially. It
 will not be the GPT-3 like examples, but it will be grounded simulation-based language.
 And for like the querying would be like question answering kind of thing?
 Correct. And it will be in some simple world initially, but it will be about, okay, can
 the system connect the language and grounded in the right way and run the right simulations
 to come up with the answer.
 And the goal is to try to do things that, for example, GPT-3 couldn't do.
 Correct.
 Speaking of which, if we could talk about GPT-3 a little bit, I think it's an interesting
 thought-provoking set of ideas that OpenAI is pushing forward. I think it's good for
 us to talk about the limits and the possibilities in neural networks. So in general, what are
 your thoughts about this recently released very large 175 billion parameter language
 model?
 So I haven't directly evaluated it yet. From what I have seen on Twitter and other people
 evaluating it, it looks very intriguing. I am very intrigued by some of the properties
 it is displaying. And of course, the text generation part of that was already evident
 in GPT-2, that it can generate coherent text over long distances. But of course, the weaknesses
 are also pretty visible in saying that, okay, it is not really carrying a world state around.
 And sometimes you get sentences like, I went up the hill to reach the valley or something
 like that, some completely incompatible statements. Or when you're traveling from one place to
 the other, it doesn't take into account the time of travel, things like that. So those
 things I think will happen less in GPT-3 because it is trained on even more data and it can
 do even more longer distance coherence. But it will still have the fundamental limitations
 that it doesn't have a world model and it can't run simulations in its head to find
 whether something is true in the world or not.
 Do you think within, so it's taking a huge amount of text from the internet and forming
 a compressed representation. Do you think in that could emerge something that's an approximation
 of a world model, which essentially could be used for reasoning? I'm not talking about
 GPT-3, I'm talking about GPT-4, 5 and GPT-10.
 Yeah, I mean, they will look more impressive than GPT-3. So if you take that to the extreme,
 then a Markov chain of just first order, if you go to, I'm taking it the other extreme,
 if you read Shannon's book, he has a model of English text, which is based on first-order
 Markov chains, second-order Markov chains, third-order Markov chains and think that,
 okay, third-order Markov chains look better than first-order Markov chains. So does that
 mean a first-order Markov chain has a model of the world? Yes, it does. So yes, in that
 level, when you go higher-order models or more sophisticated structure in the model
 like the transformer networks have, yes, they have a model of the text world. But that is
 not a model of the world. It's a model of the text world and it will have interesting
 properties and it will be useful, but just scaling it up is not going to give us AGI
 or natural language understanding or meaning. The question is whether being forced to compress
 a very large amount of text forces you to construct things that are very much like...
 Because the ideas of concepts and meaning is a spectrum. So in order to form that kind
 of compression, maybe it will be forced to figure out abstractions, which look awfully
 a lot like the kind of things that we think about as concepts, as world models, as common
 sense. Is that possible? No, I don't think it is possible because the information is
 not there. The information is there behind the text, right? No, unless somebody has written
 down all the details about how everything works in the world to the absurd amounts like,
 okay, it is easier to walk forward than backward, that you have to open the door to go out of
 the thing, doctors wear underwear, unless all these things somebody has written down
 somewhere or somehow the program found it to be useful for compression from some other
 text, the information is not there. So that's an argument that text is a lot lower fidelity
 than the experience of our physical world. Right, right. It's worth a thousand words.
 Well, in this case, pictures aren't really so the richest aspect of the physical world
 isn't even just pictures, it's the interactivity with the world. Exactly. Yeah, it's being
 able to interact. It's almost like if you could interact. So I disagree. Well, maybe
 I agree with you that pictures were a thousand words, but a thousand is still. Yeah, you
 could say you could capture it with the GPT X. So I wonder if there's some interactive
 element where a system could live in text world where it could be part of the chat,
 be part of talking to people. It's interesting. I mean, fundamentally, so you're making a
 statement about the limitation of text. Okay. So let's say we have a text corpus that includes
 basically every experience we could possibly have. I mean, just a very large corpus of
 text and also interactive components. I guess the question is whether the neural network
 architecture, these very simple transformers, but if they had like hundreds of trillions
 or whatever comes after a trillion parameters, whether that could store the information needed
 that's architecturally. Do you have thoughts about the limitation on that side of things
 with neural networks? I mean, so transformer is still a feed forward neural network. It
 has a very interesting architecture, which is good for text modeling and probably some
 aspects of video modeling, but it is still a feed forward architecture.
 You believe in the feedback mechanism, the recursion.
 Oh, and also causality, being able to do counterfactual reasoning, being able to do interventions,
 which is actions in the world. So all those things require different kinds of models to
 be built. I don't think transformers captures that family. It is very good at statistical
 modeling of text and it will become better and better with more data, bigger models,
 but that is only going to get so far. So I had this joke on Twitter saying that, hey,
 this is a model that has read all of quantum mechanics and theory of relativity and we
 are asking you to do text completion or we are asking you to solve simple puzzles. When
 you have AGI, that's not what you ask a system to do. We'll ask the system to do experiments
 and come up with hypothesis and revise the hypothesis based on evidence from experiments,
 all those things. Those are the things that we want the system to do when we have AGI,
 not solve the simple puzzles.
 Next impressive demo, somebody generating a red button in HTML.
 Right. Which are all useful. There is no dissing the usefulness of it.
 By the way, I'm playing a little bit of a devil's advocate, so calm down, Internet.
 So I'm curious almost in which ways will a dumb but large neural network will surprise
 us. So like, um, it's kind of your, I completely agree with your intuition is just that I don't
 want to dogmatically like a hundred percent put all the chips there. We've been surprised
 so much. Even the current GPT two and three are so surprising. Yeah. The self play mechanisms
 of alpha zero are really surprising. And I, the reinforcement, the fact that reinforcement
 learning works at all to me is really surprising. The fact that neural networks work at all
 is quite surprising given how nonlinear the spaces, the fact that it's able to find local
 minima that are at all reasonable. It's very surprising. So it, uh, I wonder sometimes
 whether us humans just want it to not, for AGI, not to be such a dumb thing. So I just,
 cause exactly what you're saying is like, um, the ideas of concepts and be able to reasonable
 those concepts and, and connect those concepts in a, like hierarchical ways. And then to
 be able to have a world models. I mean, just everything we're describing in human language
 in this poetic way, uh, seems to make sense that that is what intelligence, the reasoning
 are like. I wonder if at the core of it, it could be much dumber. Uh, well finally it
 is still connections and messages passing over them. Right. Right. So in that way it
 is done. So I guess the recursion, the, the feedback mechanism that does seem to be a
 fundamental kind of thing. Um, yeah, yeah. The idea of concepts, also memory. Correct.
 Yeah. Like having an episodic memory. Yeah. Yeah. That seems to be an important thing.
 So how do we get memory? So yeah, we have another piece of work that, which came on
 recently on how do you form episodic memories and, uh, and form abstractions from them.
 Uh, and we haven't figured out a, you know, all the connections of that to the overall
 cognitive architecture, but, um, yeah, what are your ideas about how you could have episodic
 memory? So at least it's very clear that there, you need to have two kinds of memory, right?
 That's very, very clear, right? We chose, there are things that happen, uh, as statistical
 patterns in the world. Uh, but then there is the, the, the one timeline of things that
 happen only once in your life, right? Uh, uh, and this day is not going to happen ever
 again. And, and so, and that needs to be stored as a, as a, you know, just a stream of, uh,
 string, right? This is, this is my experience. And then, then the question is about how do
 you take that experience and connect it to the statistical part of it? How do you now
 say that, okay, I experienced this thing. Now I want to be careful about similar situations.
 Uh, and so, so you need to be able to index that similarity using your other giants that
 is, you know, the, the model of the world that you have learned, although the situation
 came from the episode, you need to be able to index the other one. So, uh, the episodic
 memory being implemented as an indexing over the other, uh, model that you're building.
 So the memories remain and they, uh, they, they're an index into this, like the statistical
 thing that you formed. Yeah. Statistical causal structural model that you built over, over
 time. So, so it's basically the idea is that, uh, the hippocampus is, uh, just storing or
 sequencing, uh, a, in a set of pointers that happens over time. And then whenever you want
 to reconstitute that memory and evaluate the different, uh, uh, aspects of it, whether
 it was good, bad, do I need to encounter the situation again? You need the cortex, uh,
 to reinstantiate, to replay that memory. So how do you find that memory? Like which direction
 is the important direction? Both directions are, you know, it's again bidirectional. So
 I guess how do you retrieve the memory? So this is again hypothesis, right? We're making
 this up. So when you, uh, when you come to a, uh, new situation, right? Uh, your, your
 cortex is doing inference, uh, over in the new situation. And then, uh, of course hippocampus
 is connected to different parts of the cortex. Um, and, and you have this deja vu situation,
 right? Okay. I have seen this thing before. And, uh, and then in the hippocampus, you
 can have an index of, okay, this is when it happened as a, as a, as a timeline. Uh, and,
 and, and then then you can use the hippocampus to drive the similar timelines to say, now
 I am, I am rather than being driven by my current input stimuli, I am going back in
 time and rewinding my experience from there, putting back into the cortex and then putting
 it back into the cortex, of course affects what you're going to see next in your current
 situation. Got it. Yeah. So that's, that's the whole thing. Having a world model and
 then the, yeah. Uh, connecting to the perception. Yeah. It does seem to be that that's what's
 happening and be, uh, on the neural network side, it's, um, it's interesting to think
 of how we actually do that. Yeah. Yeah. So I have a knowledge base. Yes. It is possible
 that you can put many of these structures into, uh, neural networks and we will find
 ways of combining properties of neural networks and graphical models. So, uh, I mean, it's
 already started happening. Uh, graph neural networks are kind of a merge between them.
 Uh, and there will be more of that thing. So, but to me it is the direction is pretty
 clearly, you know, looking at biology and the history of, uh, uh, evolutionary history
 of intelligence, it is pretty clear that, okay, what we need is more structure in the
 models and modeling of the world and supporting dynamic inference.
 Well, let me ask you, uh, there's a guy named Elon Musk. There's a company called Neuralink
 and there's a general field called brain computer interfaces. Yeah. Um, it's kind of, uh, interface
 between your two loves the brain and the intelligence. Uh, so there's like very direct applications
 of brain computer interfaces for, um, people with different conditions, more in the short
 term, but there's also these sci-fi futuristic kinds of ideas of, uh, AI systems being able
 to, uh, communicate in a high bandwidth way with the brain, bi-directional. Yeah. Uh,
 what are your thoughts about, uh, Neuralink and BCI in general as a, as a possibility?
 So I think BCI is a cool research area. And in fact, um, when I got interested in brains
 initially, when, you know, so I was enrolled at Stanford and when I got interested in brains,
 it was, it was through a brain, uh, computer interface talk that Krishna Shainoy gave.
 That's when I even started thinking about the problem. So, uh, so it is definitely a
 fascinating research idea and it is, the applications are enormous, right? Um, so, you know, there's
 a science fiction scenario of, you know, brains directly communicating. Let's, you know, let's
 keep that aside for the time being. Uh, uh, even just the intermediate milestones that
 we're pursuing, which are very reasonable as far as I can see, uh, being able to control
 an external limb using, uh, uh, in a direct connections from the brain and being able
 to write things into the brain. Uh, so, so those are all, uh, good steps to take and
 they have enormous applications, you know, people losing limbs, being able to control
 prosthetics, uh, quadriplegics, being able to control something, uh, so, and therapeutics.
 And you know, I also know about another company working in this space, uh, called Paradromics.
 Uh, they're doing, you know, it's based on a different, uh, electrode array, but trying
 to attack some of the same problems. So I think it's a very-
 Also surgery.
 Correct. Surgically implanted electrodes. Yeah. Um, so, uh, yeah, I think of it as
 a very, very promising field, especially when it is helping people overcome some limitations.
 You know, at some point, of course it will advance the level of being able to communicate.
 Uh,
 How hard is that problem do you think? Like, so, so, okay, let's say we magically solve
 what I think is a really hard problem of doing all of this safely. Yeah. So, so like being
 able to connect electrodes and not just thousands, but like millions to the brain.
 I think it's very, very hard because you also do not know what the, what will happen to
 the brain with that, right? In the sense of how does the brain adapt to something like
 that?
 And it's, you know, as we were learning, it's the brain is quite, uh, in terms of neuroplasticity
 is pretty malleable. So it's going to adjust. So the machine learning side, the computer
 side is going to adjust and then the brain is going to adjust. Exactly. And then what
 soup does this land us into is the kind of hallucinations you might get from it.
 From this. That might be pretty intense. Yeah. Yeah. So, uh, just connecting to all of Wikipedia.
 It's interesting whether we need to be able to figure out the basic protocol of the brain's
 communication schemes in order to get them to the machine and the brain to talk. Cause
 another possibility is the brain actually just adjust to whatever the heck the computer
 is doing.
 Exactly. That's the way I think that I find that to be a more promising way. It's basically
 saying, you know, okay, attach electrodes to some part of the cortex. Okay. And make
 sure maybe if it is done from birth, the brain will adapt. It says that, you know, that part
 is not damaged. It was not used for anything. These electrodes are attached there. Right.
 And now you, you train that part of the brain to do this high bandwidth communication between
 something else. Right. And, and, uh, if you do it like that, then it is brain adapting
 to, and of course your external system is designed such that it is adaptable. You know,
 just like we, you know, design computers or mouse keyboard, all of them to be, uh, uh,
 interacting with humans. So, uh, of course that feedback system is designed to be, uh,
 human compatible, but, um, now it is not trying to record from the, all of the brain and,
 uh, uh, you know, now, you know, two system trying to adapt to each other. It's a brain
 adapting into one way.
 That's fascinating. The brain is connected to like the internet. It's connected. Just
 imagine just connecting it to Twitter and just, just, just taking that stream of information.
 Um, yeah. But again, if we take a step back, I don't know what your intuition is. I feel
 like that is not as hard of a problem as the doing it safely. There's, there's a huge barrier
 to surgery that cause it, cause the biological system, it's a mush of like weird stuff.
 Correct. So that the surgery part of it, biology part of it, the long-term repercussions part
 of it, again, I don't know what else will, uh, you know, we, we often find, uh, after
 a long time, uh, in biology that, okay, that idea was wrong, right. You know, so people
 used to cut off this, uh, uh, the gland called the thymus or something. Uh, and, uh, then
 they found that, oh no, that actually causes cancer.
 And then there's a subtle, like millions of variables involved, but this whole process,
 the nice thing, just like again, with the Elon, just like colonizing Mars seems like
 a ridiculously difficult idea, but in the process of doing it, we might learn a lot
 about the biology of the neurobiology of the brain, the neuroscience side of things. It's
 like, if you want to learn something, do the most difficult version of it and see what
 you learn.
 The intermediate steps that they are taking sounded already reasonable to me.
 Yeah, it's great. Well, but like everything with Elon is the timeline seems insanely fast.
 So that's, that's the only awful question. Uh, well, one, we've been talking about cognition
 a little bit, so like reasoning, we haven't mentioned the other C word, which is consciousness.
 Uh, do you ever think about that one? Do you, is that useful at all? Uh, in this whole context
 of what it takes to create an intelligent reasoning being, or is that completely outside
 of, uh, your, uh, like the engineering perspective of intelligence?
 It is not outside the realm, but it doesn't on a day to day basis inform what we do, but
 it's more so in, in many ways, the company name is connected to this idea of consciousness.
 What's the company name?
 Vicarious, you know, so Vicarious is the company name. And, uh, and so what does Vicarious
 mean, right? It's, um, uh, at the first level, it is about modeling the world and, uh, and
 it is internalizing the external actions. So, so you interact with the world and learn a
 lot about the world. And now after having learned a lot about the world, you can run
 those things in your mind without actually having to act in the world. So you can run,
 uh, things Vicariously just in your, in your, in your brain. And similarly, you can experience
 another person's thoughts by, you know, having a model of how that person works and, uh,
 and running there, you know, putting yourself in some other person's shoes. So that is being
 Vicarious.
 Now, it's the same modeling apparatus that you're using to model the external world or
 some other person's thoughts. You can turn it to yourself. You can up, you know, if that
 same modeling thing is applied to your own modeling apparatus, uh, then that is what
 gives rise to consciousness, I think.
 Well, that's more like self-awareness. There's the hard problem of consciousness, which is
 like, when the model becomes, when, when the model feels like something, when this whole
 process is like, it act, it's like, you really are in it. It, you feel like an entity in
 this world, not just, you know, that you're an entity, but it feels like something to
 be that entity. It, um, it, you know, and thereby we attribute this, you know, then
 this starts to be where something that has consciousness can suffer. You start to have
 these kinds of things that we can reason about that is much, uh, much heavier. It seems like
 there's much greater cost of your, your decisions and like mortality is tied up into that. Like
 the fact that these things end, right? First of all, I end at some point and then other
 things end. And you know that, that somehow seems to be at least for us humans, a deep
 motivator. Yes. And that, you know, that, that idea of motivation in general, we talk
 about goals and AI, but the goals aren't quite the same thing as like the, our mortality.
 It feels like, it feels like, first of all, humans don't have a goal and they just kind
 of create goals at different levels. They like make up goals because we're terrified
 by the mystery of the thing that gets us all. So we make these goals up. So we're like a
 goal generation machine as opposed to a machine which optimizes the trajectory towards a singular
 goal. So it feels like that's an important part of, uh, cognition, that whole mortality
 thing. Well, it is, it is a part of human, uh, cognition, uh, but there is no, uh, reason
 for, uh, that mortality to come to the question for a, uh, artificial system because we can,
 uh, copy the artificial system. The, the, the problem with humans is that we can't,
 I can't clone you. I can't, I, I, you know, I can, I can clone, even if I clone you as
 a, uh, you know, the, the hardware, your experience, uh, that was stored in your brain, uh, your,
 uh, episodic memory, all those will not be captured in the, in the new clone. Um, so,
 um, but that's not the same with an AI system, right? So, but it's also possible that the,
 the thing that you mentioned with us humans is actually fundamental, of fundamental importance
 for intelligence. So like the fact that you can copy an AI system means that that AI system
 is not yet an, um, AGI. So like, so if you look at existence proof, if we reason based
 on existence proof, you could say that it doesn't feel like death is a fundamental property
 of an intelligent system, but we don't yet give me an example of an immortal intelligent
 being. Uh, we don't have those. It could, it's very possible that, you know, that's,
 it is a fundamental property of intelligence is a thing that has a deadline for itself.
 You can think of it like this. So suppose you invent a way to freeze people, uh, for
 a long time, it's not dying, right? Uh, so, so you can be frozen and woken up, uh, thousands
 of years from now. Uh, so it's no fear of death.
 So no, the, you're still, it's not, it's not about time. It's about the knowledge that
 it's temporary. Uh, the, that aspect of it, the finiteness of it, I think, um, creates
 a kind of urgency. Correct. For us, for humans. Yeah. For humans. Yes. Uh, and that, that
 is part of our drives. Uh, but, um, and that's why I'm not too worried about AI, uh, you
 know, uh, having motivations to kill all humans and those kinds of things. Why just wait?
 You know, so why do you need to do that? Yeah. I've never heard that before. That's a good,
 that's a good point. Yeah. Just murder seems like a lot of work. We'll just wait, wait
 it out. That'll probably hurt themselves. Let me ask you, um, people often kind of wonder
 world-class researchers such as yourself, what kind of books, technical fiction, philosophical
 were, um, uh, had an impact on you in your life and, uh, maybe ones you could probably
 possibly recommend that others read. Maybe if you have three books that pop into mind.
 Yeah. So I definitely liked, uh, Judy Apple's book, uh, probabilistic reasoning and intelligence
 systems. It's, um, it's a very deep technical book, but what I liked is that in, so there
 are many places where you can learn about probabilistic graphical models from, but throughout
 this book, Judy Apple kind of sprinkles his philosophical observations and, and he thinks
 about, connects us to how the brain thinks and attentions and resources, all those things.
 So, so that whole thing makes it more interesting to read. Uh, he emphasizes the importance
 of causality. So that was in his later book. So this was the first book probabilistic reasoning
 and intelligence systems. He mentions causality, but he hadn't really sunk his teeth into like,
 you know, how do you actually formalize that? Yeah. And, uh, the second book causality is
 the 2000, uh, the one in 2000, that one is really hard. So I wouldn't recommend that.
 Oh yeah. So that looks at the, like the mathematical, like his model of, uh, do calculus. Yeah.
 It was pretty dense mathematical. Right. Right. Right. Uh, the book of why is definitely more
 enjoyable. Oh, for sure. Yeah. Um, so yeah, so I would, I would recommend probabilistic
 reasoning in intelligence systems. Another book I liked, uh, was, uh, one from Doug Huffstater.
 Uh, this was a long time ago. He has a book, he had a book, I think called it was called
 the mind's eye. It was, um, uh, probably Huffstater and Daniel Dennett together. Uh, yeah. So,
 and it actually was, uh, I bought that book. It's in my show. I haven't read it yet, but
 I, uh, I couldn't get an electronic version of it, which is annoying because I'm, you
 read everything on Kindle. Uh, you had to actually purchase the physical. It's like
 one of the only physical books I have. Cause anyway, there's a lot of people who recommended
 it highly. So yeah. And the third one, uh, I would definitely recommend reading is, um,
 uh, this is not a technical book. It is history. It's called it's the name of the book. I think
 it's Bishop's boys. It's about pride brothers and, uh, and their, their, their path and
 how it was, uh, it's, there are multiple books on this topic and all of them are great. It's,
 uh, fascinating how, uh, flight was, uh, you know, treated as an unsolvable problem. And,
 and, and also, you know, what aspects did people emphasize? Uh, you know, people thought,
 oh, it is all about, uh, just powerful engines, you know, just need to have powerful lightweight
 engines. Uh, and, um, so, you know, some people thought of it as how far can we just throw
 the thing, you know, throw it. So, uh, so it's a, it's a very fascinating. And even
 after they, uh, made the invention, like, you know, people not believing it and, uh,
 uh, the social aspect of it. It's a different, you know, very passionate. Do you, do you
 draw any parallels between, you know, birds fly? So there's the natural approach to, uh,
 to flight and then there's the engineered approach. Do you, uh, do you see the same
 kind of thing with the brain and our trying to engineer intelligence? Yeah, it's, it's
 a good analogy to have. Uh, of course, all analogies have their, you know, so, so, so
 people in, uh, you know, AI often, uh, use airplanes as an example of, Hey, we didn't
 learn anything from birds. Look, but the funny thing is that, uh, and, and the, the saying
 is, uh, airplanes don't flap wings. Right. This is what they say. The funny thing and
 ironic thing is that, that you don't need to flap to fly is something Wright brothers
 found by observing birds. So they have in their notebook, you know, in some of these
 books, they show their, their notebook drawings, right? They, they make detailed notes about
 buzzards, uh, just soaring over thermals. And they basically say, look, flapping is
 not the important proportion is not the important problem to solve here. We want to solve control.
 Uh, and, uh, once you solve control, propulsion will fall into place. All of, all of these
 are people, uh, you know, they realize this by observing birds.
 Beautiful report. That's actually brilliant. Uh, cause people do use that analogy. I'm
 gonna have to remember that one. Do you have advice for people interested in artificial
 intelligence, like young folks today? I talked to undergraduate students all the time, uh,
 interested in neuroscience, interested in, uh, understanding how the brain works. Is
 there advice you would give them about their career, maybe about their life?
 Uh, sure. I think every, you know, every piece of advice should be taken with a pinch of
 salt because each person is different. Their motivations are different. Uh, but I can,
 I can definitely say if your goal is to understand the brain from the angle of wanting to build
 one, you know, uh, then, uh, being an experimental neuroscientist might not be the way to go about
 it. Um, uh, it might be a better way to pursue it might be through, um, computer science,
 electrical engineering, machine learning and AI. And of course you have to study up the
 neuroscience, but that you can do on your own. Um, if you are more, uh, attracted by
 finding something intriguing about the discovering something intriguing about the brain, uh,
 then of course it is better to be an experimentalist. Uh, so find that motivation. What are you
 intrigued by? And of course, find your strengths too. Some people are very good experimentalists,
 uh, and, and they enjoy doing that. Uh, and it's interesting to see which department,
 if you're, if you're picking, uh, in terms of like your education path, whether to, um,
 uh, to, to go with like an MIT it's brain and computer, uh, no, uh, BCS brain and cognitive
 sciences. Yeah. Uh, or, or the CS side of things. And actually, uh, the brain folks,
 the neuroscience folks are more and more now embracing of, uh, you know, learning TensorFlow
 and PyTorch, right there. They, they see the power of, uh, trying to engineer ideas, uh,
 that, uh, that they get from the brain into, and then explore how those could be used to,
 uh, to create intelligent systems. So that might be the right department actually. Yeah.
 To, uh, so this was a question in, uh, uh, you know, one of the Redwood neuroscience
 Institute workshops, uh, that Jeff Hawkins organized, uh, uh, almost 10 years ago. This
 question was put to a panel, right? What, what should be the undergrad major, uh, you should
 take if you want to understand the brain. And, uh, and the majority of opinion in that one was,
 uh, electrical engineering. Interesting. Uh, because I mean, I'm a doubly undergrad,
 so I got lucky in that way. Uh, um, but it, I think it does have some of the right ingredients
 because you learn about circuits, you, you learn about how you can construct circuits to, uh, you
 know, approach, you know, do functions, uh, you learn about microprocessors, um, you learn
 information theory, you learn signal processing, uh, you learn continuous math. So, um, so in that
 way, it's, it's a good step to, if you want to go to computer science or neuroscience, you can,
 it's, it's a good step. The downside, you're more likely to be forced to use MATLAB.
 So, so, so one of the, one of the interesting things about, I mean, this is changing, the world
 is changing, but, uh, like certain departments lagged on the programming side of things on
 developing good, uh, good habits and software engineering. But I think that's more and more
 changing and, and students can take the answer of their own hands, like learn to program. I feel
 like everybody should learn to program because it, uh, it's like everyone in the sciences,
 because it empowers, it puts the data at your fingertips. So you can organize it. You can find
 all kinds of things in the data. And then you can also for the appropriate sciences, build systems
 that like based on that. So like then engineer intelligence systems, uh, we already talked about
 mortality, so we hit a ridiculous point, but let me ask you the, uh, you know,
 one of the things about intelligence is it's goal-driven and, uh, you study the brain.
 So the question is like, what's the goal that the brain is operating under? What's,
 what's the meaning of it all for us humans in your view? What's the meaning of learning?
 What's the meaning of it all for us humans in your view? What's the meaning of life?
 The meaning of life is what you construct out of it. It's completely open.
 It's open. So there's no, there's nothing, uh, uh, like you mentioned, you like constraints. So
 there's what's, uh, it's, it's, it's wide open. Is there, is there some
 useful aspect that you think about in terms of like the openness of it and just the basic
 mechanisms of generating goals, uh, in studying cognition in the brain that you think about,
 or is it just about, cause everything we've talked about, kind of the perception system is to
 understand the environment. That's like to be able to like not die, like not fall over and like be
 able to, uh, you don't think we need to, um, think about anything bigger than that.
 Yeah, I think so because it's, it's basically being able to understand the machinery of the world,
 uh, such that you can pursue whatever goals you want. Right. So the machinery of the world is,
 is really ultimately what we should be striving to understand. The rest is just,
 the rest is just whatever the heck you want to do or whatever, whatever.
 I think that's, uh, I, that's beautifully put. I don't think there's a better way to end it.
 I'm so honored that you, you would show up here and waste your time with me. It's been
 an awesome conversation. Thanks so much for talking today.
 Oh, thank you so much. This was, this was so much more fun than I expected.
 Thank you.
 Thanks for listening to this conversation with Dilip George.
 And thank you to our sponsors, Babbel, Raycon Earbuds and Masterclass. Please consider
 supporting this podcast by going to babbel.com and use code Lex, going to buy Raycon.com slash Lex
 and signing up at masterclass.com slash Lex. Click the links, get the discount. It really is the best
 way to support this podcast. If you enjoy this thing, subscribe to our channel, and we'll see you
 next time.
 Thank you for listening and hope to see you next time.