Luís and João Batalha: Fermat's Library and the Art of Studying Papers | Lex Fridman Podcast #209
agi ai ai podcast artificial intelligence artificial intelligence podcast lex ai lex fridman lex jre lex mit lex podcast luís and joão batalha mit ai
Luis and Joao Batalha are co-founders of Fermat's Library. Please support this podcast by checking out our sponsors:
- Skiff: https://skiff.org/lex to get early access
- SimpliSafe: https://simplisafe.com/lex and use code LEX to get a free security camera
- Indeed: https://indeed.com/lex to get $75 credit
- NetSuite: http://netsuite.com/lex to get free product tour
- Four Sigmatic: https://foursigmatic.com/lex and use code LexPod to get up to 60% off

EPISODE LINKS:
Fermat's Library Twitter: https://twitter.com/fermatslibrary
Luis's Twitter: https://twitter.com/luismbat
Joao's Twitter: https://twitter.com/joao_batalha
Fermat's Library Website: https://fermatslibrary.com

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4
Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41

OUTLINE:
0:00 - Introduction
2:22 - Backstories to research papers
17:13 - Fermat's Library
37:14 - Scientific publishing
1:00:54 - How to read a paper
1:06:48 - Taking good notes
1:15:27 - Favorite papers on Fermat's Library
1:56:18 - Fermat's Library on Twitter
2:05:50 - What it takes to build a successful startup
2:14:46 - Game of Thrones
2:17:34 - Realism in science fiction movies
2:23:33 - Greatest soccer player of all time
2:46:22 - Advice for young people

SOCIAL:
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
- Medium: https://medium.com/@lexfridman
- Reddit: https://reddit.com/r/lexfridman
- Support on Patreon: https://www.patreon.com/lexfridman
 The following is a conversation with Luis and Joao Batalla, brothers and co-founders of Fermat's Library, which is an incredible platform for annotating papers. As they write on the Fermat's Library website, Justice Pierre de Fermat scribbled his famous last theorem in the margins, professional scientists, academics, and citizen scientists can annotate equations, figures, ideas, and write in the margins. Fermat's Library is also a really good Twitter account to follow. I highly recommend it. They post little visual factoids and explorations that reveal the beauty of mathematics. I love it. Quick mention of our sponsors. Skiff, Simplisafe, Indeed, Nutsweet, and FourSigmatic. Check them out in the description to support this podcast. As a side note, let me say a few words about the dissemination of scientific ideas. I believe that all scientific articles should be freely accessible to the public. They currently are not. In one analysis I saw, more than 70% of published research articles are behind a paywall. In case you don't know, the funders of the research, whether that's government or industry, aren't the ones putting up the paywall. The journals are the ones putting up the paywall while using unpaid labor from researchers for the peer review process. Where is all that money from the paywall going? In this digital age, the costs here should be minimal. This cost can easily be covered through donation, advertisement, or public funding of science. The benefit versus the cost of all papers being free to read is obvious. And the fact that they're not free goes against everything science should stand for, which is the free dissemination of ideas that educate and inspire. Science cannot be a gated institution. The more people can freely learn and collaborate on ideas, the more problems we can solve in the world together. And the faster we can drive old ideas out and bring new, better ideas in. Science is beautiful and powerful. And its dissemination in this digital age should be free. This is the Lex Friedman Podcast, and here's my conversation with Luis and Joao Batalla. Luis, you suggested an interesting idea. Imagine if most papers had a backstory section, the same way that they have an abstract. So knowing more about how the authors ended up working on a paper can be extremely insightful. And then you went on to give a backstory for the Feynman QED paper. This is all in a tweet, by the way. We're doing tweet analysis today. How much of the human backstory do you think is important in understanding the idea itself that's presented in the paper or in general? I think this gives way more context to the work of scientists. I think a lot of people have this almost kind of romantic misconception that the way a lot of scientists work is almost as the sum of Eureka moments where all of a sudden they sit down and start writing two papers in a row and the papers are usually isolated. And when you actually look at it, the papers are chapters of a way more complex story. And the Feynman QED paper is a good example. So Feynman was actually going through a pretty dark phase before writing that paper. He lost enthusiasm with physics and doing physics problems. And there was one time when he was in the cafeteria of Cornell and he saw a guy that was throwing plates in the air and he noticed that when the plate was in the air, there were two movements there. The plate was wobbling, but he also noticed that the Cornell symbol was rotating. And he was able to figure out the equations of motions of those plates. And that led him to kind of think a little bit about electron orbits in relativity, which led to the paper about quantum electrodynamics. So that kind of reignited his interest in physics and ended up publishing the paper that led to his Nobel Prize, basically. And I think there are a lot of really interesting backstories about papers that readers never get to know. For instance, we did a couple of months ago an AMA around a paper, a pretty famous paper, the Gans paper with Ian Goodfellow. And so we did an AMA where everyone could ask questions about the paper and Ian was responding to those questions. He was also telling the story of how he got the idea for that paper in a bar. So there was also an interesting backstory. I also read a book by Cedric Villani. Cedric Villani is this mathematical field medalist. And in his book, he tries to explain how he got from a PhD student to the field's medal. And he tries to be as descriptive as possible about every single step how he got to the field's medal. And it's interesting also to see just the amount of random interactions and discussions with other researchers, sometimes over coffee, and how it led to fundamental breakthroughs in some of his most important papers. So I think it's super interesting to have that context of the backstory. Well, the Ian Goodfellow story is kind of interesting and perhaps that's true for Feynman as well. I don't know if it's romanticizing the thing, but it seems like just a few little insights and a little bit of work does most of the leap required. Do you have a sense that for a lot of the stuff you've looked at, just looking back through history, it wasn't necessarily the grind of like Andrew Wiles or the Fermat's Last Theorem, for example. It was more like a brilliant moment of insight. In fact, Ian Goodfellow has a kind of sadness to him almost in that at that time in machine learning, like at that time, especially in for GANs, you could code something up really quickly in a single machine and almost do the invention, go from idea to experimental validation and like a single person could do it. And now there's kind of a sadness that a lot of the breakthroughs you might have in machine learning kind of require large scale experiments. So it was almost like the early days. So I wonder how many low hanging fruit there are in science and mathematics and even engineering where it's like, you could do that little experiment quickly. Like you have an insight in a bar. Why is it always a bar? But you have an insight at a bar and then just implement and the world changes. It's a good point. I think it also depends a lot on the maturity of the field. When you look at a field like mathematics, like it's a pretty mature field, a field like machine learning. It's growing pretty fast. And it's actually pretty interesting. I looked up like the number of new papers on archive with the keyword machine learning and like 50% of those papers have been published in the last 12 months. So you can see just the sense. Five zero, 50%. So you can see the magnitude of growth in that field. And so I think like as fields mature, like those types of moments, I think naturally are less frequent. It's just a consequence of that. The other point that is interesting about the backstory is that it can really make it more memorable in a way. And by making it more memorable, it kind of sediments the knowledge more. In your mind, I remember also reading the sort of the backstory to Dijkstra's shortest path algorithm, right? Where he came up with it essentially while he was sitting down at a coffee shop in Amsterdam. And he came up with that algorithm over 20 minutes. And one interesting aspect is that he didn't have any pen or paper at the time. And so he had to do it all in his mind. And so there's only so much complexity that he can handle if you're just thinking about it in your mind. And that like, when you think about the simplicity of Dijkstra's shortest path finding algorithm, it's knowing that backstory helps sediment that algorithm in your mind so that you don't forget about it as easily. It might be from you that I saw a meme about Dijkstra. It's like he's trying to solve it and he comes up with some kind of random path. And then it's like, my parents aren't home. And then he figures out the algorithm for the shortest path. I tried through words to convey memes, but that's hilarious. I don't know if it's in post that we construct stories that romanticize it. Apparently with Newton, there was no Apple, especially when you're working on problems that have a physical manifestation or a visual manifestation. It feels like the world could be an inspiration to you. So it doesn't have to be completely on paper. Like you could be sitting at a bar and all of a sudden see something in a pattern will spark another pattern and you can visualize it and rethink a problem in a particular way. Of course, you can also load the math that you have on paper and always carry that with you. So when you show up to the bar, some little inspiration could be the thing that changes it. Is there any other people almost on the human side, whether it's physics with Feynman, Dirac, Einstein, or computer science, touring, anybody else? Any backstories that you remember that jump out? Because I'm also referring to, not necessarily these stories where something magical happens, but these are personalities. They have big egos. Some of them are super friendly. Some of them are like self-obsessed. Some of them have anger issues. Some of them, how do I describe Feynman? But he appears to have appreciation of the beautiful in all its forms. It has a wit and a cleverness and a humor about him. Does that come into play in terms of the construction of the science? I think you brought up Newton. Newton is a good example also to think about his backstory because there's a certain backstory of Newton that people always talk about, but then there's a whole another aspect of him that is also a big part of the person that he was. But he was really into alchemy and that he spent a lot of time thinking about that and writing about it. He took it very seriously. He was really into Bible interpretation and trying to predict things based on the Bible. And so there's also a whole backstory then. And of course, you need to look at it in the context and the time that when Newton lived, but it adds to his personality and it's important to also understand those aspects that maybe people are not as proud to teach to little kids, but it's important as part of who he was and maybe without those, who knows what he would have done otherwise. Well, the cool thing about alchemy, I don't know how it was viewed at the time, but it almost like to me symbolizes dreaming of the impossible. Like most of the breakthrough ideas kind of seem impossible until they're actually done. It's like achieving human flight. It's not completely obvious to me that alchemy is impossible or like putting myself in the mindset of the time. And perhaps even still, everything that some of the most incredible breakthroughs are would seem impossible. And I wonder the value of believing almost like focusing and dreaming of the impossible such that it is actually is possible in your mind and that in itself manifests, whether the accomplishing that goal or making progress in some unexpected direction. So alchemy almost symbolizes that for me. I distinctly remember having the same thought of thinking, you know, when I learned about atoms and that they have protons and electrons, I was like, okay, to make gold, you just take whatever has an atomic weight below it and then shove another proton in there and then you have a bunch of gold. So like, why don't people do that? It seemed like conceptually is like, you know, this sounds feasible. You might be able to do it. And you can actually is just very, very expensive. Yeah, yeah, exactly. Exactly. So in a sense, we do have alchemy and maybe even back then, it wasn't as crazy that he was so into it. But people just don't like to talk about that as much. Yeah, but Newton in general was a very interesting fellow. Anybody else come to mind in terms of people that inspire you, in terms of people that you just are happy that they have once or still exist on this earth? I think I mean, Freeman Dyson for me. Yeah, Freeman Dyson was, was I've had a chance to actually exchange a couple of emails with him. It was probably one of the most humble scientists that I've ever met. And that had a big impact on me. We were trying, we're actually trying to convince him to annotate a paper on Fermat's library. And I sent him an email asking him if you could annotate a paper. And his response was something like, I have very limited knowledge. I just know a couple of things about certain fields. I'm not sure if I'm qualified to do that. That was his first response. And this was someone that should have won a Nobel Prize and worked on a bunch of different fields, did some really, really great work. And then just the interactions that I had with him, every time I asked him a couple of questions about his papers, and he always responded saying, I'm not here to answer your questions. I just want to open more questions. And so that had a big impact on me. It was like just an example of an extremely humble yet accomplished scientist. And Feynman was also a big, big inspiration in the sense that he was able to be, you know, again, extremely talented and, and a scientist, but at the same time, socially, he was able to, to, he was also really smart from a social perspective. And he was able to interact with people. It was also a really good teacher. And it was also to did awesome work in terms of explaining physics to the masses and motivating and getting people interested in physics. And that for me was, was also a big inspiration. Yeah. I like the childlike curiosity of some of those folks that you mentioned, Freeman. I've Daniel Kahneman, I got a chance to meet and interact with some of these truly special scientists. What makes them special is that even in older age, there are still like, there's still that fire of childlike curiosity that burns. And some of that is like not taking yourself so seriously that you think you figured it all out, but almost like thinking that you don't know much of it. And that's like step one in having a great conversation or collaboration or exploring a scientific question. And it's cool how the very thing that probably earned people the Nobel prize or, or work that's seminal in some way, it is the very thing that still burns even after they've won the prize. It's cool to see. And they're rare humans, it seems. And to that point, I remember like the last email that I sent to Freeman Dyson was like in his last birthday. He was really into number theory and primes. So what I did is I took like a photo of him, picture, and then I turned that into like a giant prime number. So I converted the picture into a bunch of one and eights, and then I moved some numbers around until it was a prime. And then I sent him that. Also the visual, like it still looked like the picture, it was made up of a prime. That's tricky to do. It's hard to do. It looks harder to do. It's harder than it actually is. So the way you do it is like you convert the darker regions into eights and the lighter regions in ones. And then there's- And then just keep flipping numbers until- But there's like some primality tests that are cheaper from a computational standpoint. But what it tells you is it excludes numbers that are not prime. Then you end up with a set of numbers that you don't know if they are prime or not. And then you run the full primality test on that. So you just have to keep iterating on that. And it was, it's funny because when you got the picture, it was like, how did you do that? It was super curious to, and then we got into the details. And again, this was, it was already 90, I think 92 or something. And that curiosity was still there. So you can really see that in some of these scientists. So could we talk about Fermat's library? Yeah, absolutely. What is it? What's the main goal? What's the dream? It is a platform for annotating papers in its essence. And so academic papers can be one of the densest forms of content out there and generally pretty hard to understand at times. And the idea is that you can make them more accessible and easier to understand by adding these rich annotations to the site. And so we can just imagine a PDF view on your browser, and then you have annotations on each side. And then when you click on them, a sidebar expands, and then you have annotations that support LaTeX and Markdown. And so the idea is that you can say, explain a tougher part of a paper where there's a step that is not completely obvious, or you can add more context to it. And then over time, papers can become easier and easier to understand and can evolve in a way. But it really came from myself, Luis, and two other friends. We've had this long running habit of kind of running a journal club amongst us. We come from different backgrounds. I studied CS, Luis studied physics, and so we'd read papers and present them to each other. And then we tried to bring some of that online. And that's when we decided to build Fermat's library. Then over time, it kind of grew into something with a broader goal. And really what we're trying to do is trying to help move science in the right direction. That's really the ultimate goal and where we want to take it now. So there's a lot to be said. So first of all, for people who haven't seen it, the interface is exceptionally well done. Execution is really important here. Absolutely. The other thing is just to mention for a large number of people, apparently, which is new to me, don't know what LaTeX is. So it's spelled like LaTeX, so be careful Googling it if you haven't before. It's a... Sorry, I don't even know the correct terminology. Type setting language? It's a type setting language where you're basically writing a program that then generates something that looks, from a typography perspective, beautiful. So a lot of academics use it to write papers. I think there's a bunch of communities that use it to write papers. I would say it's mathematics, physics, computer science. That's the main thing. Because I'm collaborating currently on a paper with two neuroscientists from Stanford. And they don't know that. So I'm using Microsoft Word and Mendeley and all of those kinds of things. And I'm being very Zen-like about the whole process, but it's fascinating. It's a little heartbreaking, actually, because it's funny to say, and we'll talk about open science, actually the bigger mission behind it for Mars Library is really opening up the world of science to everybody. It's these silly two facts of one community uses LaTeX and another uses Word is actually a barrier between them. It's boring and practical, in a sense, but it makes it very difficult to collaborate. Just on that, I think there are some people that should have received a Nobel Prize, but will never get it. And I think one of those is Donald Knuth, because of tech and LaTeX. Because it had a huge impact in terms of just making it easier for researchers to put their content out there, making it uniform as much as possible. Oh, you mean like a Nobel Peace Prize? Maybe a Nobel Peace Prize. Yeah, I think so. I mean, he, at a very young age, got the Turing Award for his work in algorithms and so on. I think it's in, it might be even the 60s, but I think it's the 70s. So when he was really young, and then he went on to do incredible work with his book and with tech that people don't know. And going back just on the reason why we ended up, because I think this is interesting, the reason why we ended up using the name Fermat's Library, this was because of Fermat's Last Theorem. And Fermat's Last Theorem is actually a funny story. So Pierre de Fermat, he was like a lawyer, and he wrote like on a book that he had a solution to Fermat's Last Theorem, but that didn't fit the margin of that book. And so Fermat's Last Theorem basically states that there's no solution. If you have integers a, b, and c, there's no solution to a to the power of n plus b to the power of n equals to c to the power of n if n is bigger than two. So there's no solutions. And he said that, and that problem remained open for almost 300 years, I believe. And a lot of the most famous mathematicians tried to tackle that problem. No one was able to figure that out until Andrew Wiles, I think was in the 90s, was able to publish the solution, which was, I believe, almost 300 pages long. And so it's kind of an anecdote that there's a lot of knowledge and insights that can be trapped in the margins, and there's a lot of potential energy that you can release if you can release, if you actually spend some time trying to digest that. And that was the origin story for the name. You actually can share the contents of the margins with the world. Exactly. That could inspire a solution or a communication that then leads to a solution. But if you think about papers, like, papers are, as Joao was saying, probably one of the densest pieces of text that any human can read. And you have these researchers, like some of the brightest minds in these fields, working on new discoveries and publishing these work on journals that are imposing them restrictions in terms of the number of pages that they can have to explain a new scientific breakthrough. And so at the end of the day, papers are not optimized for clarity and for a proper explanation of that content, because there are so many restrictions. And so there's, as I mentioned, there's a lot of potential energy that can be freed if you actually try to digest a lot of the contents of papers. Can you explain some of the other things? So margins, librarian, journal club. So journal club is what a lot of people know us for, where we every week we release an annotated paper and in all sorts of different fields, but physics, CS, math, margins is kind of the same software that we use to run the journal club and to host the annotations. But we've made that available for free to anybody that wants to use it. And so folks use it at universities and for running journal clubs. And so we've just made that freely available. And then librarian is a browser extension that we developed that is sort of an overlay on top of archive. So it's about bringing some of the same functionality around comments, plus adding some extra niceties to archive, like being able to very easily extract the references of a paper that you're looking at or being able to extract the bibtech in order to cite that paper yourself. So it's an overlay on top of archive. Yeah, the idea is that you can have that commenting interface without having to leave archive. It's kind of incredible. I didn't know about it. And once I learned of it, it's like, holy shit. Why isn't it more popular given how popular archive is? Like everybody should be using it. Archive sucks in terms of its interface. Or let me rephrase that, it's limited. Archive is a pretty incredible project. And it is, in a way, the growth has been completely linear over time. If you look at like number of papers published on archive, it's pretty much a straight line for the past 20 years. Especially if you're coming from a startup background and then you're trying to do archive, you'd probably try all sorts of growth acts and try to then maybe have paid features and things like that. And that would maybe ruin it. And so there's a subtle balance there. And I don't know what aspects you can change about it. For some tools in science, it just takes time for them to grow. Archive has just turned 30, I believe. And for people that don't know, archive is this kind of online repository where people put pre-prints, which are versions of the papers before they actually make it to journals. ARXIV for people who don't know. And it's actually a really vibrant place to publish your papers in the aforementioned communities of mathematics, physics, and computer science. It started with mathematics and physics, and then over the last 30 years it evolved and now, actually, computer science now, it's a more popular category than physics and math on archive. And there's also, which I don't know very much about, like a biology medical version of that. Bioarchive, yeah, bioarchive. It's interesting because if you look at these platforms for pre-prints, they actually play a super important role. Because if you look at a category like math, for some papers in math, it might take close to three years after you click upload paper on the journal website and the paper gets published on the website of the journal. So this is literally the longest upload period on the internet. And during those three years, like, it's, you know, their content is just, you know, locked. And so that's why it's so important for people to have websites like archive so that you can share that before it goes to the journal with the rest of the world. There was actually an archive that Perelman published, the three papers that led to the proof of the Poincare conjecture. And then you have other fields like machine learning, for instance, where the field is evolving at such a high rate that people don't even wait before the papers go to journals before they start working on top of those papers. So they publish papers, so they publish them on archive, then other people see them, they start working on that. And archive did a really good job at like building that core platform to host papers. But I think there's a really, really big opportunity in building more features on top of that platform, apart from just hosting papers. So collaboration, annotations, and like having other things apart from from papers like code and other things. Because, for instance, in the field like machine learning, there's a really big, you know, as I mentioned, people start working on on top of pre prints, and they are assuming that that that pre print is correct. But you really need a way, for instance, to maybe it's not peer review, but distinguish what is good work from bad work on archive. How do you do that? So like a commenting interface like library, and it's useful for that so that you can distinguish that at at in a field that is growing so fast as machine learning. And, and then you have platforms that focus for instance on on just biology, bio archive is a good example. bio archive is also super interesting, because there, there's actually an interesting experiment that was run in the 60s. So in the 60s, the NIH supported this, this, this experiment called the information exchange group, which at the time was a way for researchers to share biology pre prints via mail, or using libraries. And that project in the 1960s got canceled six years after it started. And it was due to intense pressure from the journals to kill that project, because they were fearing competition from from the pre prints for the journal industry. Creek was also one of the famous scientists that opposed to the information exchange group. And it's interesting because right now, if you analyze the number of biology papers that appear first as pre prints, it's only 2% of the papers. And it this was almost 50 almost 50 years after that first experiment. So you can see like that pressure from the journals to cancel that initial version of a pre print repo had a tremendous impact on on on the number of papers that are showing up in biology as pre prints. So it delayed a lot that that revolution and but now platforms like bio archive are doing that work. But there's still a lot of room for growth there. And I think it's super important because those are the papers that are open that everyone can read. Okay, so but if we just look at the entire process of science as a big system, can we just talk about how it can be revolutionized? So you have an idea, depending on the field, you want to make that idea concrete, you want to run a few experiments in computer science, there might be some code, there'd be a data set for you know, some of the more sort of biology, psychology, you might be collecting the data set that's called, you know, a study, right? So that's part of that, that's part of the methodology. And so you are putting all that into a paper form. And then you have some results. And then you you submit that to a place for review through the peer review process. And there's a process where how would you summarize the peer review process, but it's really just like a handful of people look over your paper and comment and based on that decide whether your paper is good or not. So there's a whole broken nature to it. At the same time, I love the peer review process when I buy stuff on Amazon, like for like the commenting system, whatever that is. So okay, so there's a bunch of possibilities for revolutions there. And then there's the other side, which is the collaborative aspect of the science, which is people annotating people commenting, sort of the low effort collaboration, which is a comment. Sometimes, as you've talked about a comment can change everything, but you know, or a higher effort collaboration, like more like maybe annotations or even like contributing to the paper, you can think of like collaborative updating of the paper over time. So there's all these possibilities for doing things better than they've been done. Can we talk about some ideas in this space, some ideas that you're working on some ideas that you're not yet working on, but should be revolutionized? Because it does seem that archive and like open review, for example, are like the Craigslist of science. Like, like, yeah, okay, I'm very grateful that we have it, but it just feels like it's like 10 to 20 years, like it doesn't feel like that's a feature. The simplicity of it is a feature. It feels like it's a it's a it's a bug. But then again, the pushback there is Wikipedia has the same kind of simplicity to it. And it seems to work exceptionally well in the crowdsourcing aspect of it. So sorry, this that's a bunch of stuff going on the table. Let's just pick random things that we can talk about. Wikipedia, you know, for me, it's the cosmological constant of the internet. It's, I think we are lucky to live in the parallel universe where Wikipedia exists. Yes. Because if someone had pitched me Wikipedia, like a publicly edited, I seek an encyclopedia, like a couple of years ago, like it would be, I don't know how many people would have said that that would have survived. I mean, it makes almost no sense. It's like having a Google Doc that everybody on the internet can edit and like, they'll be like the most reliable source for knowledge. And I don't know how many but hundreds of thousands of topics. Yeah. It's insane. It's insane. And like you have, and then you have users, like there's one, a single user that edited one third of the articles on Wikipedia. So we have these really, really big power users, there are a substantial part of like, what makes Wikipedia successful. And so, like, no one would have ever imagined that that could happen. And so that's one thing I completely agree with what you just said. I also- Sorry to interrupt briefly. Maybe let's inject that into the discussion of everything else. I also believe, I've seen that with Stack Overflow, that one individual or a small collection of individuals contribute or revolutionize most of the community. Like, if you create a really powerful system for archive or like open review and made it really easy and compelling and exciting for one person who is in like a 10X contributor to do their thing, that's going to change everything. It seems like that was the mechanism that changed everything for Wikipedia. And that's the mechanism that changed everything for Stack Overflow, is gamifying or making it exciting or just making it fun or pleasant or fulfilling in some way for those people who are insane enough to like answer thousands of questions or write thousands of factoids and like research them and check them all those kinds of things or read thousands of papers. Yeah. No, Stack Overflow is another great example of that. And it's just, and those are both two incredibly productive communities that generate a ton of value and capture almost none of it. And in a way, it's very counterintuitive that these communities would exist and thrive. And it's really hard to, there aren't that many communities like that. So how do we do that for science? Do you have ideas there? Like what are the biggest problems that you see? You're working on some of them. Like just on that, there are a couple of really interesting experiments that people are running. An example would be like the polymath projects. So this is a kind of a social experiment that was created by Tim Gowers, a Fields Medalist. And his idea was to try to prove that is it possible to do mathematics in a massively collaborative way on the internet. So he decided to pick a couple of problems and test that. And they found out that it actually is possible for specific types of problems, namely problems that you're able to break down in little pieces and go step by step. You might need, as with open source, you might need people that are just kind of reorganizing the house every once in a while. And then people throw a bunch of ideas and then you make some progress, then you reorganize, you reframe the problem, you go step by step. But they were actually able to prove that it is possible to collaborate online and do progress in terms of mathematics. And so I'm confident that there are other avenues that could be explored here. Can we talk about peer review, for example? Absolutely. I think like in terms of the peer review, I think it's important to look at the bigger picture here of like, of what the scientific publishing ecosystem looks like. Because for me, there are a lot of things that are wrong about that entire process. So if you look at, for instance, what publishing means in like a traditional journal, you have journals that pay authors for their articles, and then they might pay like reviewers to review those articles, and finally they pay people to, or distributors to distribute the content. In the scientific publishing world, you have scientists that are usually backed by government grants, they are giving away their work for free in the form of papers. And then you have other scientists that are reviewing their work. This process is known as the peer review process, again, for free. And then finally, we have government-backed universities and libraries that are buying back all that work so that other scientists can read. So this is, for me, it's bizarre. You have the government that is funding the research, it's paying the salaries of the scientists, it's paying the salaries of the reviewers, and it's buying back all that product of their work again. And I think the problem with this system, and it's why it's so difficult to break this suboptimal equilibrium, is because of the way academia works right now and the way you can't progress in your academic life. And so in a lot of fields, the competition in academia is really insane. So you have hundreds of PhD students, they are trying to get to a professor position, and it's hyper competitive. And the only way for you to get there is if you publish papers, ideally in journals, with a high impact factor. In computer science, it's often conferences are also very prestigious, or actually more prestigious than journals now. Okay, interesting. So that's the one discipline that you have to work with. I mean, that has to do with the thing we've discussed in terms of how quickly the field turns around. But like, NeurIPS, CVPR, those conferences are more prestigious, or at the very least as prestigious as the journals. But yeah, but doesn't matter, the process is what it is. So for people that don't know, the impact factor of a journal is basically the average number of citations that a paper would get if it gets published on that journal. But so you can really think that the problem with the impact factor is that it's a way to turn papers into accounting units. And let me unpack this, because the impact factor is almost like a nobility title. So because papers are born with impact, even before anyone reads them. So the researchers, they don't have the incentive to care about if this paper is going to have a long term impact on the world. What they care, their goal, their end goal is the paper to get published. Yes. So that they get that value upfront. So for me, that is one of the problems of that. And that really creates a tyranny of metrics. Because at the end of the day, if you are a dean, what you want to hire is like people, researchers that publish papers on journals with high impact factors, because that will increase the ranking of your university and will allow you to charge more for tuition, so on and so forth. And especially when you are in super competitive areas, that people will try to gamify that system and misconduct starts showing up. There's a really interesting book on this topic called Gaming the Matrix. It's a book by a researcher called Mario Biagioli. It goes a lot into how the impact factor and metrics affect science negatively. And it's interesting to think, especially in terms of citations, if you look at the early work of looking at citations, there was a lot of work that was done by a guy called Eugene Garfield. And this guy, the early work in terms of citation, they wanted to use citations as from a descriptive point of view. So what they wanted to create was a map. And that map would create a visual representation of influence. So citations would be links between papers. And ideally, what they would show, they would represent is that you read someone else's paper and it had an impact on your research. They weren't supposed to be counted. I think this inspired Larry and Sergey's work for Google. Exactly. I think they even mentioned that. But what happens is as you start counting citations, you create a market. And the same way, the work of Eugene Garfield was a big inspiration for Larry and Sergey and for the PageRank algorithm that led to the creation of Google. And they even recognized that. And if you think about it, it's like the same way there's a gigantic market for search engine optimization, SEO, where people try to optimize the PageRank and how high the web page will rank on Google, the same will happen for papers. People will try to optimize like the impact factors and the citations that they get. And that creates a really big problem. And it's super interesting to actually analyze them. If you look at the distribution of the high impact, the impact factors of journals, you have like nature with nature, I believe it's like in the low 40s. And then you have, I believe science is high 30s. And then you have a really good set of good journals that will fall between 10 and 30. And then you have a gigantic tale of journals that have impact factor below two. And you can really see two economies here. You see the universities that are maybe less prestigious, less known, where the faculty are pressured to just publish papers, regardless of the journal. What I want to do is increase the ranking of my university. And so they end up publishing as many papers as they can in like journals with low impact factor. And unfortunately, this represents a lot of the global south. And then you have the luxury good economy. So for instance, for, and there are also problems here in the luxury good economy. So if you look at the journal like nature, so with impact factor of like in the low 40s, there's no way that you're going to be able to sustain that level of impact factor by just grabbing the attention of scientists. What I mean by that is like, for the journals, the articles that get published in nature, they need to be New York times great. So they need to make it to the, you know, to the, to the big media, they need to be captured by the big media, big media. And because that's the only way for you to capture enough attention to sustain that level of citations. And that of course creates problems because people then will try to again, gamify the system and have like titles or abstracts or that are bigger claim, make the claims that are bigger than what is actually can be sustained by the data or the content of the paper. And you'll have clickbait titles or clickbait abstracts. And again, this is all a consequence of metrics and science of metrics. And, and this is a very dangerous cycle that I think it's very hard to break, but it's happening in academia in a lot of fields right now. Is it fundamentally the existence of metrics or the metrics just need to be significantly improved? Because like I said, the metrics used for Amazon for purchasing, I don't know, computer parts is pretty damn good in terms of selecting which are the good ones, which are not in that same way. If we had Amazon type of review system in the space of ideas, in the space of science, it feels like that those metrics would be a little bit better. Sort of when it's, when it's significantly more open to the crowdsource nature of the internet, of the, of the scientific internet, meaning as opposed to like my biggest problem with peer review has always been that it's like five, six, seven people, usually even less. And it's often, nobody's incentivized to do a good job in the whole process. Meaning it's anonymous in a way that doesn't incentivize, like doesn't gamify or incentivize great work. And also it doesn't necessarily have to be anonymous. Like there has to be, the entire system is, doesn't encourage actual sort of rigorous review. For example, like open review does kind of incentivize that kind of process of collaborative review, but it's also imperfect. It just feels like the thing that Amazon has, which is like thousands of people contributing their reviews to a product. It feels like that could be applied to science. Where the same kind of thing you're doing with Formal's library, but doing at a scale that's much larger. It feels like that should be possible given the number of grad students, given the number of general public that's getting, for example, I personally, as a person who got an education in mathematics and computer science, like, and I'm not going to say that I'm not a person who got an education in mathematics and computer science. Like I can be a quote unquote like reviewer on a lot bigger set of things than is my exact expertise. If I'm one of thousands of reviewers, if I'm the only reviewer, one of five, then I better be like an expert in the thing. But if I, and I've learned this with COVID, which is like, you can just use your basic skills as a data analyst and to contribute to the review process and a particular little aspect of a paper and be able to comment, be able to sort of draw in some references that challenge the ideas presented or to enrich the ideas that are presented. It just feels like crowdsourcing the review process would be able to allow you to have metrics in terms of how good a paper is that are much better representative of its actual impact in the world, of its actual value to the world, as opposed to some kind of arbitrary gamified version of its impact. I agree with that. I think we, there's definitely the possibility, at least for more resilient, a more resilient system than what we have today. And that's, I think that's kind of what you're describing, Alex. And I mean, to an extent, we kind of have like a little bit of a Heisenberg uncertainty principle. When you pick a metric, as soon as you do it, then maybe it works as a good heuristic for a short amount of time, but soon enough people will start gamifying. But then you can definitely have metrics that are more resilient to gamification and they'll work as a better heuristic to try to push you in the in the best direction. But I guess the underlying problem you're saying is there's a shortage of positions in academia. That's a big problem for me. Yeah. And that, and so they're going to be constantly gamifying the metrics. It's a bit of a zero sum game. It's a very competitive, it's what, it's a very competitive field. And that's what usually happens in very competitive fields. Yeah. But I think some of like the peer review problems, like scale helps, I think. And it's interesting to look at like what you're mentioning, breaking it down, maybe in like smaller parts and having more people jumping in. But this is definitely a problem. And the peer review problem, as I mentioned, is correlated with the problem of like academic career progression and it's all intertwined. And it's, that's why I think it's so hard to break it. There are like a couple of really interesting things that are being done right now. There are a couple of, for instance, journals that are overlay journals on top of platforms like archive and bio archive that want to remove like the more traditional journals from the equation. So essentially a journal is just a collection of links to papers. And what they're trying to do is like removing that middleman and trying to make the review process a little bit more transparent and not charging universities. Like there's a couple of, there are a couple of more famous ones. There's one discrete analysis in mathematics. There's one called the quantum journal, which we are actually working with them. We have a partnership with them for the papers that get published in quantum journal. They also get the annotations on formats and they're doing pretty well. They've been able to grow substantially. The problem there is getting to critical mass. So it's again, convincing the researchers and especially the young researchers that need that impact factor, need those publications to have citations to not publish on the traditional journal and go on an open journal and publish their work there. I think there are a couple of really high profile scientists, people like Tim Gowers, they are trying to incentivize like famous scientists that already have tenure and that don't need that to publish that to increase the reputation of those journals so that other, maybe younger scientists can start publishing on those as well. So they can try to break that vicious cycle of the more traditional journals. I mean, another possible way to break this cycle is to like raise public awareness and just by force, like ban paid journals. Like what exactly are they contributing to the world? Like basically making it illegal to, forget the fact that it's mostly federally funded. So that's a super ugly picture too, but like why should knowledge be so expensive? Like where everyone is working for the public good and then there's these gatekeepers that, you know, most people can't read most papers without having to pay money and that doesn't make any sense. Like that should be illegal. I mean, that's what you're saying is exactly right. I mean, for instance, right, I went to school here in the US, we studied in Europe and you would sit like, you'd ask me all the time to download papers and send it to him because he just couldn't get it and like papers that he needed for his research. And so, but he's a student, like he's a grad student. He was a grad student, but that, you know, I'm even referring to just regular people. Oh yeah. Okay. That too. Yeah. And I think during 2020 because of COVID, a lot of journals put down the walls for certain kind of coronavirus related papers, but like that just gave me an indication that like this should be done for everything. It's absurd. Like people should be outraged that there is these gates because, so the moment you dissolve the journals, then there'll be an opportunity for startups to build stuff on top of archive. It'd be an opportunity for like, for Ma's library to step up, to scale up to something much even larger. I mean, that was the original dream of Google, which I've always admired, which is make the world's information accessible. Actually, it's interesting that Google hasn't, maybe you guys can correct me, but they have put together Google scholar, which is incredible, but they, and they've the scanning of books, but they haven't really tried to make science accessible in the following way. Like besides doing Google scholar, they haven't like delved into the papers, right? Which is especially curious given what we was saying, right? That it's kind of in their genesis. There's this, you know, research that was very connected with our papers reference each other and like building a network out of that. Interesting enough, like Google, I think there was a, there was not intended. Google plus was like the Google social network that got canceled was used by a lot of researchers with who I think was just a, you know, side kind of a side effect, but then a lot of people ended up migrating to Twitter, but it was not on purpose. But yeah, I agree with you. Like they haven't gone past Google scholar and, you know, why that said, Google scholars incredible for people who are not familiar. It's one of the best aggregation of all the scientific work that's out there. And especially the network that connects to all of them, what sites, what, and also trying to aggregate all of the versions of the papers that are available there and trying to merge them in a way that one particular work, even though it's available in a bunch of places, conscious, you know, like a central hub of what that work is across the multiple versions. But that almost seems like a fun pet project of a couple of engineers within within Google, as opposed to a serious effort to make the world science accessible. But going back to just the journals, when you're talking about that Lex, I believe that in that front, I think we might be past the event horizon. So I think the, the, the model, the business model for the journals, you know, doesn't make sense. They're a middle layer that is not adding a lot of value. And you see a lot of motions, whereas like in Europe, a lot of the papers that are get, that are funded by the European Union, they will have to be open to the public. And I think there's a lot of Bill Gates to like the, the, what the Gates Foundation funds, like the demand that it, then it will, that it's accessible to everybody. So I think it's, it's the question of time before that, that wall kind of falls. And that is going to help in a lot of possibilities. Because, you know, imagine if, if you had like the layer of that gigantic layer of papers all available online, you know, that unlocks a lot of potential as a platform for people to build things on top of that. But I think it's what you're saying. It is weird. Like you can literally go and listen to any song that was ever made on your phone, right? You open Spotify and you might not even pay for it. You might be on the free version and you can listen to any song that was ever made pretty much. But there's like, you, you don't have access to a huge percentage of academic papers, which is just like this fundamental knowledge that we're all funding, but you as an individual don't have access to it. And somehow, you know, like the problem for music got solved. But for papers, it's still like, it's just not yet. It could be ad supported, all those kinds of things. And then hopefully that would change the way we do science. This is the most exciting thing for me is, especially once I started like making videos and this silly podcast thing, I started to realize like that if you want to do science, one of the most effective ways is to do like couple the paper with a set of YouTube videos, like explaining it. Like that also seems like there's a lot of room for disruption there. What is the paper 2.0 going to look like? I think like LaTeX and the PDF seems like if you, it's interesting, if you look at the first paper that got published in nature, and if you look at the paper that got published in nature today, look at the two side by side, they are fundamentally the same. And even though like the paper that gets published today, you know, you get even, even code, like right now people put like code like on, on a PDF. Like, and there are so many things that are related to papers today, you know, you use, you have data, you have code, you might need videos to better explain the concepts. So it's, it, I think for me, it's natural that there's going to be also an evolution there, that papers are not going to be just the static PDFs or LaTeX. There's going to be a next, a next interface. So in academia, a lot of things that are judged, you're judged by is often quantity, not quality. I wonder if there's an opportunity to have like, I tend to judge people by the best work they've ever done as opposed to, I wonder if there's a possibility for that to encourage sort of focusing on the quality and not necessarily in paper form, but maybe a subset of a paper, subset of idea, almost even a blog post or an experiment. Like why does it have to be published in a journal to be legitimate? And it's interesting that you mentioned that. I also think like, yeah, it's why, why, why is that the only format? Why can't a blog post or we were even experimenting, experimenting with these a few months ago, or can you actually like publish something or like a new scientific breakthrough or or something that you've discovered in the form of like a set of tweets, a Twitter thread? Why can't that be possible? And we were experimenting with that idea. We even, yeah, we ran a couple of, like some people submitted a couple of those, like I think the limit was three or four tweets. Maybe it's a new way to look at a, you know, a proof or something, but I think it serves to show that there should be other ways to publish like scientific discoveries that don't fit the paper format. Well, but so even with the Twitter thread, it would be, it'd be nice to have some mechanism of formalizing it and making it the stat, making it into an NFT. Okay. Maybe like a concrete thing that you can reference with a link that's unique because I mean, everything we've been saying, all of that while being true, it's also true that the constraints and the formalism of a paper works well. It like forces you constraints forces you to narrow down your thing and literally put it on paper. But, you know, I agree, make concrete. And that's why, I mean, it's, it's a, it's a I mean, it's not broken. It's just could be better. And that's the main idea. I think there's something about writing, whether it's a blog post or Twitter thread or a paper, that's really nice to, to concretize a particular little idea that they can then be referenced by other ideas, then it can be built on top of with other ideas. So let me ask you've read quite a few papers. You've annotated quite a few papers. Let's talk about the process itself. How do you advise people read papers or maybe you want to broaden it beyond just papers, but just read concrete pieces of information to understand the insights that lay within. I would say for paper specifically, I would bring back kind of what Louise was talking about it is that it's important to keep in mind that papers are not optimized for ease of understanding. And so, right, there's all sorts of restrictions and size and format and language that they can use. And so it's important to keep that in mind. And so that if you're struggling to read a paper doesn't that might not mean that the underlying material is actually that hard. And so that's definitely something that especially for us that we read papers and most of the times it'll be papers that are completely outside of our comfort zone, I guess. And so it'd be completely new areas to us. So I always try to keep that in mind. So there's usually a certain kind of structure like abstract introductions, methodology, depending on the community and so on. Is there something about the process of like, how to read it, whether you want to skim it to try to find the parts that are easy to understand or not, reading it multiple times? Is there any kind of hacks that you can comment on? I remember like Feynman had this kind of hack when he was reading papers where you would basically, I think I believe he would read the conclusion of the paper and we would try to just see if he would be able to figure out how to get to the conclusion in like a couple of minutes by himself. And he would read a lot of papers that way. And I think Fermi also did that almost. And Fermi was known for doing a lot of back of the envelope calculations. So he was a master at doing that. And in terms of like, especially when reading a paper, I think a lot of times people might feel discouraged about the first time you read it, you know, it's very hard to grasp or you don't understand a huge fraction of the paper. And I think it's having read a lot of papers in my life, I think I've in peace with like the fact that you might spend hours where you're just reading a paper and jumping from paper to paper, reading something and then you're like, paper to paper, reading citations. And like your level of understanding of sometimes of the paper is very close to zero percent. And all of a sudden, you know, everything kind of makes sense in your mind. And then, you know, you have this quantum jump where all of a sudden you understand the big picture of the paper. And this is an exercise that I have to when reading papers, and especially like more complex papers, like, okay, you don't understand because you're just going through the process and just keep going. And like, and it might feel super chaotic, especially if you're jumping from reference to reference, you know, you might end up with like 20 tabs open, and you're reading a ton of other papers, but it's just trusting that process that at the end, like you'll find light. And I think for me, that's a good framework. When reading a paper, it's hard, because you know, you might end up spending a lot of time and you looks like you're lost. But, but that's the process to actually understand what they're talking about in a paper. Yeah, I think that process, I enjoy, I've found a lot of value in the process, especially for things outside my field, reading a lot of related work sections, and kind of going down that path of getting a big context of the field, because what's, especially when they're well written, there's opinions injected into the related work, like what work is important, what is not. And if you read multiple related work sections that cite or don't cite each other, like the papers, you get a sense of where the field, where the tensions of the field are, where the, where the field is striving. And that helps you put into context, like whether the work is radical, whether it's overselling itself, whether it's underselling itself, all those things. And on, added on top of that, I find that often the related work section is the most kind of accessible and readable part of a paper, because it's kind of, it's brief to the point, it's trying, like summarizing, it's almost like a Wikipedia style article. The introduction is supposed to be a compelling story or whatever, but it's often like overselling, there's like an agenda introduction. The related work usually has the least amount of agenda, except for the few, like elements where you're trying to talk shit about previous work, where you're trying to sell that you're doing much better. But other than that, when you're just painting where the, where the field came from, or where the field stands, that's really valuable. And also again, just to agree with what to agree with, fine. And the conclusion, but it's like, I get a lot of value from the breadth first search, kind of read the conclusion, then read the related work, and then go through the references in the related work, read the conclusion, read the related work, and just go down the tree until you like hit dead ends or run out of coffee. And then through that process, you go back up the tree and now you can see the results in their proper, in their proper context, unless of course the paper's truly revolutionary, which even that process will help you understand that is in fact truly revolutionary. You've also, you talked about just following your Twitter thread in depth first search. You talked about that you read the book on Grisha Perlman, Gurgurur Perlman, and then you had a really nice Twitter thread on it, and you were taking notes throughout. So at a high level, is there suggestions you can give on how to take good notes? Whether it's we're talking about annotations or just for yourself to try to put on paper ideas as you progress through the work in order to then like understand the work better. For me, I always try not to underestimate how much you can forget within six months after you've read some. Oh, I thought you were going to say five minutes, but yeah, six months is good. Yeah, or even shorter. And so that's something that I always try to keep in mind. And it's often, I mean, every once in a while I'll read back a paper that I annotated on Fermat and it's, and I'll read through my own annotations and it's, I've completely forgotten what I had written. But it also, it's interesting because in a way after you just understood something, you're kind of the best possible teacher that can teach your future self. You know, after you've forgotten it, you can, you're kind of your own best possible teacher at that moment. And so it's, it can be great to try to capture that. It's brilliant. It just made me kind of realize it's really nice to put yourself in the position of teaching an older version of yourself that returns to this paper, almost like thinking it literally. That's under explored, but it's, it's super powerful because you were the person that you can, like, if you, if you look at the scale from like one, not knowing anything about the topic and 10, like you are the one that progressed from one to 10 and you know which steps you struggled with. So you are the really the best person to help yourself make that transition from one to 10. And a lot of the times, like, and we don't, I really believe that the framework that we have to like expose ourselves to like be talking to like us when we were an expert, when we were taking that class and we knew everything about quantum mechanics. And then six months later, you don't remember half of the things. How could we make it easier for like to have those conversations between you and your past self, past expert self? I think there might be, you know, it's an under explored idea. I think notes on paper are probably not the best way. I'm not sure if it's a combination of like video, audio, where it's like you have a guided framework that you follow to extract information from yourself so that you can later kind of revisit to make it easier to remember. But that's, I think it's an interesting idea worth exploring that not, I haven't seen a lot of people kind of trying to distill that problem. Yeah, I'm creating the kind of tools I find if I record, it sounds weird, but I'll take notes. But if I record audio, like little clips of thoughts, like rants, that's really effective at capturing something that notes can't. Because when I replay them, for some reason, it loads my brain back into where I was when I was reading that in a way that notes don't. Like when I read notes, I'll often be like, whoa, what? What was I thinking there? But when I listen to the audio, it brings you right back to that place. So there might, and maybe with video, with visual, that might be even more powerful. I think so. Yeah. And I think just the process of verbalizing it, that alone kind of makes you have to structure your thought and put it in a way that somebody else could come and understand it. And just the process of that is useful to organize your thoughts. And yeah, just that alone. Does the Fermat's Library Journal Club have a video component or no? Not natively. Sometimes we'll include videos, but it's always embedded. Do people like build videos on top of it to explain the paper? Because you're doing all the hard work of understanding deeply the paper. Not, we haven't seen that happening too much, but we were, we were actually playing around with the idea of creating some sort of podcast version where we try to distill the paper on an audio format that not maybe you could have access. Might be tricky. Might be trickier, but there are definitely people that could be interested in the paper and that topic, but are not willing to read it. But they might listen to a 30 minute episode on that paper. Yes. You could reach more people and you might even bring the authors to the conversation, but it's tricky. And especially for like more technical papers. We've thought about doing that, but we haven't like converged. Sure, if you have any tips. Well, I'm going to take that as a small project to take one of the Fermat's almost like half advertisement and half as a challenge for myself to take, take one of the annotated papers and like use it as a basis for creating a quick video. I, I've seen like, hopefully I'm saying the name correctly, but machine learning street talk. I think that's the name of the show. The, that I recommend highly, that's the right name, but they, they do exactly that, which is multiple hour breakdown of a paper with video component. Sometimes with authors, people love it. It's very effective. There's, there's also I've seen, I haven't seen the entire in its entirety, but I've seen like the, the founder of comma.ai George. Yeah, George. I've seen him like just taking a paper and then, you know, distilling the paper and, and coding it, coding it sometimes during 10 hours. Yeah. And he was able to, you know, get a lot of people interested in, in that and viewing him. So I'm a huge fan of that. Like, uh, it's George is a personality. I think a lot of people like listen to this podcast for the same reason. It's not necessarily the content is they, they, they, they like to listen to like a, a silly Russian who's has a childlike brain and mumbles and all those like struggle with ideas. Right. And George is a mad man who people just enjoy. Like, how is he going to struggle in implementing this particular paper? How's he going to struggle with this idea? It's fun to watch. And that actually pulls you in the personality is important there. True. But there's, there's, you know, I agree with you, but there also it's visible. Like it's, there's an extraordinary ability that is there, like is talented and you need to have, there's a craft and this guy definitely has talent and he's doing something that is not easy. And I think that also draws the attention of people. Oh yeah. And, and, and it's like the, the other day we were actually, we ran into this YouTube channel of this guy that was restoring art. Right. And, and, and it was basically just a video of him. Like the production is really, like really well done. And he's just him taking really old pieces of, of art, like, and then paintings and then restoring them. But he's really good at that. And he describes that process and that draws attention, draws the attention of people, regardless of your craft beat, like annotating a paper, like restoring it. Yeah. Like George is incredibly good at programming like quick, like, you know, those, uh, competitive programmers, like top coder and all those kinds of stuff. He has the same kind of element where the brain just jumps around really quickly. And that's, uh, yeah, just like, it's also motivation. It's yeah, it's motivating, but, but you're right. In, in watching people who are good at what they do, it's motivating. Even if the thing you're trying to do is not what they're doing, it's just like contagious when they're really good at it. And the same kind of analysis with the paper, I think, so not just like the final result, but the process of struggling with it. That's really interesting. Yeah. I think, I mean, I think Twitch proved that like, you know, that there's really a market for, for that, for watching people do things that they're really good at. And, and you'll just watch it. You will enjoy that, that might even spike your interest in that specific topic. And yeah, and people, people will enjoy watching sometimes hours on ends of great craftsmen. Do you mind if we talk about some of the papers, do any papers come to mind that have been annotated on Formaz library? The papers that we annotated can be about completely random topics, but that's part of what we enjoy as well. It forces you to explore these topics that otherwise maybe you'd never run into. And so, so the ones that come to mind are, to me are fairly random, but one that I really enjoyed learning more about is a paper written by mathematician, actually, Tom Apostol, and about a tunnel in a Greek island off the coast of Turkey. So it's very random. So this, okay, so what's interesting about this tunnel? So this tunnel was built in the sixth century BC. And, and it was built in this, in the island of Samos, which is, as I said, off the coast of Turkey. And right there, the city on one side, then the other mountain, and then they had a bunch of springs on the other side, and they wanted to bring water into the city. Building an aqueduct would be pretty hard because of the way the mountain was shaped. And it would also, you know, if they, if they were under a siege, like they could just easily destroy that aqueduct, and then the water wouldn't have any water supply, the city wouldn't have any water supply. And so they decided to build a tunnel. And they decided to try to do it quickly. And so the, they started digging from both ends at the same time, through the mountain, right? And so, like, when you start thinking about this, it's a fairly difficult problem. And this is like sixth century BC. So you had very limited access to, you know, the mathematical tools that you had at the time were very limited. And so what this paper is about is about the story of how they built it, and about the fact that for about 2000 years, kind of the accepted, the accepted explanation of how they built it was actually wrong. And so this tunnel has been famous for a while. There are a number of historians that talked about it since ancient Egypt. And the method that they described for building it is, was just wrong. And so these researchers went there and were able to figure that out. And so basically, kind of the way that they thought they had built it was basically, if you can imagine looking at the mountain from the top, and you have the mountain, then you have both entrances. And so what they thought, and this is what the ancient historians described, is that they effectively tried to draw a right angle, a right angle triangle with the two entrances at each end of the hypotenuse. And the way they did is like they would go around the mountain and kind of walking in a grid fashion. And then you can, you can figure out the two sides of the triangle. And then after you have that triangle, you can effectively draw two smaller triangles at each entrance that are proportional to that big triangle. And then you kind of have arrows pointing in each way. And then you can, you know, at least that these, that you have a line going through the mountain that connects both entrances. The issue with that is like, once you, once you go to this mountain and you start thinking of doing this, you realize that, especially given that the tools that they had at the time, that your error margin would be too small. You wouldn't be able to do it. You, you, you, the, just the fact of, of trying to, to build this triangle in that fashion, the error would accumulate and you would end up missing. You'd start building these tunnels and they would miss each other. So the task ultimately is to figure out like really perfectly as close, as close as possible to the direction you should be digging. First of all, that it's possible to have a straight line through and then what that the direction would be. And then you are trying to infer that by constructing a right triangle by doing, I, I'm not exactly sure about how to do that rigorously, like by tracing the mountain, by walking along the mountain, how to, you said grids. Yeah. You, you kind of walk as if you were in a, in a grid. And so you just walk in right angles. So, right. So you have to walk really precisely then you have to you have to use tools to measure this and the terrain is probably, yeah, very, so this makes more sense in 2d and 3d gets even weirder. So, okay. Gotcha. But so this method was described by like an ancient Egyptian historian, I think hero of Alexandria. And, and then for about like, yeah, for about 2000 years, that's, that's how, like, that's how we thought that they had built this, this tunnel. And then, and yeah, and then these researchers went there and found out that actually, they must have had to use other methods. And then in this paper, they describe these, these other methods. And of course, they can't know for sure. But there's, they presented a bunch of plausible alternatives. The one that for me was is the most plausible is that what they probably must have done is to use something that is similar to an iron sight on a rifle, the way you can line up your rifle with a target off in the distance by by having an iron sight. And, and they they must have done something similar to that effectively with three with three sticks. And that way they were able to line up sticks along the side of the mountain that were all on the same height. And so that then you could get to the other side and you could cut and then you could draw that line. So this for me is the most plausible way that they might have done that. And they, but then they, they, they described this in detail and other possible approaches in this paper. So this is a mathematician doing this? Yeah, this is a mathematician that did this. Which I suppose is the right mindset instead of skills required to solve an ancient problem. Right? Yeah. Yeah, there's mathematicians and engineers, a lot of things. Because they didn't have computers or drones or LIDAR back then or whatever technology you would use modern day for civil engineering. Yeah. And another fascinating thing is that like, you know, after effectively after the the downfall of the Roman civilization, people didn't build tunnels for about a thousand years. We go a thousand years without tunnels. And then like only in like in late middle ages that we start doing them again. But here is the tunnel, like sixth century BC, like incredibly limited mathematics. And they build it in this way. And it was a mystery for a long time exactly how they did it. And then these mathematicians went there and basically with no archaeology kind of background were able to figure it out. How do annotations for this paper look like? What is it? What's a successful annotation for a paper like this? Yes. So sometimes you're for this paper, sometimes adding some more context on a specific part. Like sometimes they mentioned, for instance, these instruments that were common in ancient Greece and ancient Rome for building things. And so in some of those annotations, I described these instruments in more detail and how they worked because sometimes it can be hard to visualize these. Then this paper, I forget exactly when this was published, I believe maybe maybe the 70s. But then there was further research into this tunnel and more interesting other interesting aspects about it. I add those to that paper as well. There's historical context that I also go into there. For instance, the fact that as I said, that effectively after the downfall of the Roman Empire, no tunnels were built. Like that's something that I added to the paper as well. Yeah. So when other people look at the paper, how do they usually consume the annotations? So is there a commenting feature? I mean, this is a really enriching experience the way you read a paper. What aspects do people usually talk about that they value from this? So anybody can just go on there and either add a new annotation or add a comment to an existing annotation. And so you can start a thread within an existing annotation. And that's something that happens relative frequency. And then because I was the original author of the initial annotation, I get pinged. And so oftentimes I'll go back and add on to that thread. How'd you pick the paper? I mean, first of all, this whole process is really exciting. I'm going to, especially after this conversation, I'm going to make sure I participate much more actively on papers that I know a lot about and on paper I know nothing about. I should get this when I say the paper. When I say the paper. I would love to. I also, I mean, I realized that there's a, like it's an opportunity for people like me to publicly annotate a paper. Like, or do an AMA around the paper. Yeah, exactly. But yeah, but like be, be in the conversation about a paper. It's like a place to have a conversation about an idea. The other way to do it that's much more ad hoc is on Twitter. Right. But this is more like formal and you could actually probably integrate the two. They have a conversation about the conversation. So the Twitter is the conversation about a conversation. And the main conversation is in the space of annotations. There's an interesting effect that we see sometimes with the annotations on our papers is that a lot of people, especially if we, the annotations are really well done. People sometimes are afraid of adding more annotations because they see that as a kind of a finished work. And so they don't want to pollute that or, and especially if it's like a silly question. This is, I don't think that's good. I think, you know, we should as much as possible try to lower the barrier for someone to jump in and ask questions. I think it only, like most of the times it adds value, but it's some feedback that we got from users and readers. I'm not exactly sure how to to kind of fight that, but. Well, I think I, I think if I serve as an inspiration in any way is by asking a lot of dumb questions and saying a bunch of dumb shit all the time. And hopefully that inspires the rest of the other folks to do the same because that's the only way to knowledge, I think, is to be willing to ask the dumb questions. And there are papers that are like, you know, we have a lot of papers on Formaz where it's just one page or really short papers. And you, we have like the shortest paper ever published in a math journal, like with like just a couple of words. One of my favorite papers on the platform is actually a paper written by Enrico Fermi. And the title of the paper is my, I think is my observations at Trinity. So basically Fermi was part of the Manhattan project. So he was in New Mexico when they exploded the first atomic bomb. And so it was a couple of miles away from the explosion. And he was probably one of the first persons to calculate the energy of the explosion. And so the way he did that was he took a piece of paper and he tore down a piece of paper in little pieces. And when the bomb exploded, the Trinity bomb was the name of the bomb, like he waited for the blast to arrive at where he was. And then he threw those pieces of paper in the air and he calculated the energy based on the displacement of the paper, the pieces of paper. And then he wrote a report, which was classified until like a couple of years ago, one page report, like calculating the energy of the explosion. So bad ass. And we actually went there and kind of unpacked and I think he just mentions basically the energy and we actually went and one of the annotations is like explaining how he did that. I wonder how accurate he was. It was maybe, I think, maybe like 20 or 25% off. Then there was another person that actually calculated the energy based on images after the explosion and the rate at which the mushroom of the explosion expanded. And it's more accurate to calculate the energy based on that. And I think it was like 20, 20% off. But it's really interesting because Fermi was known for all these, being a master at these back of the envelope calculations, always like the Fermi problems are well known for that. And it's super interesting to see like that just one page report and it was also actually classified. And it's interesting because a couple of months ago when the Beirut explosion happened, there was a video circulating of this, a bride that was doing a photo shoot when the explosion in Beirut happened. And so you can see a video of her with the wedding dress and then the explosion happens and the blast arrives at where she was. She was a couple of miles away from the blast. And you can see like the displacement of the dress as well. And I actually looked and that video went viral on Twitter. And I actually looked at that video and based, I used the same techniques that Fermi used to calculate the energy of the explosion based on the displacement of the dress. And you could actually see where she was at the distance from the explosion because there was a store behind her and you could look the name of the store. And so I calculated that it was the distance and then you can... Based on the distance where she was from the explosion and also on the displacement of the dress. Because when the blast happens, you can see the dress going back and then going back to the original position. And by just looking at how much the dress moved, you can estimate the energy of the explosion. I assume you published this? On Twitter it was just a Twitter thread. But actually a lot of people share that and it was picked up by a couple of news outlets. I was hoping it would be like a formal title and it would be an archive. No, no, no. It may be submitted. Just the Twitter thread. But it was interesting because it was exactly the same method that Fermi used. Is there something else that jumps to mind? Is there something... I know like in terms of papers, I know the Bitcoin paper is super popular. Is there something interesting to be said about any of the white papers in the cryptocurrency space? Yeah, the Bitcoin paper was the first paper that we put on Fermat. Why that choice as the first paper? It was a while ago and it was one of the papers that I read and then kind of explained it to Louise and her two other friends that do this journal club with us. I did some research in cryptography as an undergrad and so it was a topic that I was interested in. But even for me, I had that background. But reading the Bitcoin paper, it took me a few reads to really kind of wrap my head around it. It uses very Spartan precise language in a way. It's like you feel like you can't take any word out of it without something falling apart. And it's all there. I think it's a beautiful paper and it's very well written, of course. But we wanted to try to make it accessible so that anybody that maybe is an undergrad in computer science could go on there and know that you have all the information in that page that you're going to need to understand the mechanics of Bitcoin. And so I explain the basic public key cryptography that you need to know in order to understand it. You can explain, okay, what are the properties of a hash function and how they are useful in this context. Explain what a Merkle tree is. So a bunch of those basic concepts that maybe if you're reading it for the first time and you're an undergrad and you don't know those terms, you're going to be discouraged because maybe, okay, now I have to go and Google around until I understand these before I can make progress in the paper. And this way it's all there. So there's a magic to also to the fact that over time more people went on there and added further annotations. So the idea that the paper gets easier and more accessible over time, but that's still you're still looking at the original content the way the author intended it to be. But there's just more context and the toughest bits have more in-depth explanations. I think there's just so many interesting papers there. I remember reading the paper that was written by Freeman Dyson on the first time that he explained, he came up with the concept of the Dyson sphere and he put that out. Again, it's a one-page paper and what he explained was that eventually if a civilization develops and grows, there's going to be a point where when the resources on the planet are not enough for the energy requirements of that civilization. So if you want to go, the next step is you need to go to the next star and extract energy from that star. And the way to do it is you need to build some sort of cap around the star that extracts the energy. So he theorized this idea of the Dyson sphere and he went on to kind of analyze how you would build that, the stability of that sphere, like if something happens, if there's like a small oscillation with that sphere collapsing to the star or no, what would happen. And even went on to kind of say that a good way for us to look for signs of intelligent life out there is to look for signals of these Dyson spheres. And because according to the second law of thermodynamics, there's going to be a lot of infrared radiation that is going to be emitted as a consequence of extracting energy from the star. And we should be able to see those signals of like infrared if we look at the sky. But all these like from the introduction of the concept, like how to build a Dyson sphere, the problems of like having a Dyson sphere, how to detect how that could be used as a signal for intelligence life. Wait, really? That's all in the paper? All in one page paper. And it's like, for me, it's beautiful. Where was this published? It's fascinating that papers like that could be, I mean, the guts it takes to put that all together in a paper form. You know, that kind of challenges our previous discussion of paper. I mean, papers can be beautiful. You can play with the format, right? But there's a lot to unpack there. That's like the starting point. But it's beautiful that you're able to put that in one page and then people can build on top of that. But the key ideas are there. Yeah, exactly. What about, have you looked at any of the big seminal papers throughout the history of science? Like you look at simple like Einstein papers. Have any of those been annotated? Yeah, yeah, no. We have some more seminal papers that people have heard about. You know, we have the DNA double elix paper on there. We have the Higgs boson paper. Yeah, there's papers that we know that they're not going to be finding out about them because of us. But it's papers that we think should be more widely read and that folks would benefit from having some annotations there. And so we also have a number of those. A lot of like discovery papers for fundamental particles and all that. We have a lot of those on Formas library. I would like to, we haven't annotated that one, but I'd like to on the Riemann hypothesis. That's a really interesting paper as well. But we haven't annotated that one, but there's a lot of like more historical landmark papers on the platform. Have you done Poincare conjecture with Perlman? That's too much. Too much for me. But it's interesting that, you know, and going back to our discussion, like the Poincare paper was like published on archive and it was not on a journal. Like the three papers. And what do you make of that? I mean, he's such a fascinating human being. Exactly. I mentioned to you offline that I'm going to Russia. He's somebody I'm really... You should try to interview him. Yeah. Well, so I definitely will interview him. And I believe I will. I believe I can. I just don't know how to... I know where he lives. So here, okay. My hope is, my conjecture is that if I just show up to the house and look desperate enough that, or threatening enough for some combination of both, that like the only way to get rid of me is to just get the thing done. That's the hope. It's actually interesting that you mentioned that because I, after I... So a couple of weeks ago, I was searching for like stuff about Perlman online and ended up on this Twitter account of this guy that claims to be Perlman's assistant. And he's like, he has been posting a bunch of pictures next to Perlman. You can see Perlman in a library and he's next to him taking a selfie, or Perlman walking on the street and like, maybe you could reach out to this assistant and I'll send you this Twitter account. So maybe you're onto something. No, but going back to Perlman, it's super interesting because the fact that he published the proofs on archive was also a way for him to... Because he really didn't like the scientific publishing industry and the fact that you had to pay to get access to articles. And that was a form of like protest. And that's why he published those papers there. I mean, I think Perlman is just a fascinating like character. And for me, it's this kind of ideal of platonic ideal of what a mathematician should be. It's someone that is, just cares about, deeply cares about mathematics. You know, he cares about fair attribution of disregards money. And the fact that he published on archive is a good example of that. What about the Fields Medal that he turned down the Fields Medal? What do you make of that? Yeah, I mean, if you look at the reasons why he rejected the Fields Medal. So Perlman did a post-doc in the US and when he came back to Russia... Do you know how good his English is? I think it's fairly good. I think it's pretty good, right? I think it's really good. Especially in the US. Because he's given lectures in the American University. But I haven't been able to... Listen to anything. Well, certainly not listen, but I haven't been able to get anybody... Because I know a lot of people have been to those lectures. I'm not able to get a sense of like, yeah, but how strong is the accent? What are we talking about here? Is this going to have to be in Russian? Is it going to have to be in English? It's fascinating. But he writes the papers in English. So... But there's such a fascinating character and there are a couple of examples like him. I think in 28 or 29, he proved like a really famous conjecture called the Soul Conjecture. I believe it was like in a very short four-page proof of that. It was a really big breakthrough. Then he went to Princeton to give a lecture on that. And after the lecture, the chair of the math department at Princeton, a guy called Peter Sarnak, went up to Perlman and was trying to recruit him, trying to offer him a position at Princeton. And at some point he asked for Perlman's resume. And Perlman responded saying, just gave a lecture on like these really tough problem. Why do you need my resume? I'm not going to send you. I just proved my value. But going back to the Fields Medal, when Perlman went back to Russia, he arrived at a time where the salary of post-docs was so much off in regards to inflation that they were not making any money. Like people didn't even bother to pick up the checks at the end of the month because they were just like ridiculous. But thankfully he had some money that he had gained while he was doing his post-docs. So he just concentrated on like the Poincare Conjecture problem, which when he took that, he took it after it was reframed by this mathematician called Richard Hamilton, which posed the problem in a way that it turned into this super like math Olympiad problem with like perfect boundaries, well-defined, and that was perfect for Perlman to attack. And so he spent like seven years working on that. And then in 2002, he started publishing those papers on archive. And people started jumping on that, reading those papers, and there was like a lot of excitement around that. A couple of years later, there were two researchers, I believe they were from Harvard, that took Perlman's work, they sent some of the edges, and they republished that, saying that based on Perlman's work, they were able to figure out the Poincare Conjecture. And then there was, at the time at the International Conference of Mathematics in 2006, I believe that's when they were going to give out the Fields Medal, there was a lot of debate of like, oh, who's, like, who should get the credit for solving this big problem? And for Perlman, it felt really sad that people were even considering that he was not the person that solved that. And the claims that those like researchers, when they published after Perlman, there were false claims that they were the ones, they just sent it a couple of edges, like Perlman did all the really hard work. And so just the fact that they doubted that Perlman had done that, like was enough for him to say, I'm not interested in this prize. And that was one of the reasons why he rejected the Fields Medal. He also rejected the Clay Prize. So the Poincare Conjecture was one of the Millennium Prizes. There was a lot of Millennium Prizes. There was a million dollar prize associated with that problem. And that had to do with the fact that for them to attribute that prize, I think it had to be published on a journal. Yes. The proof. And again, Perlman's principles of like interfered here. And he also just didn't care about the money. He was like, Clay, I think was a businessman and he's like, doesn't have to do anything with mathematics. I don't care about these. Like, that's one of the reasons why he rejected it. Yeah, it's hard to convert into words, but at MIT, I'm distinctly aware of the distinction between when I enter a room, there's a certain kind of music to the way people talk when we're talking about ideas versus what that music sounds like when we're talking, when it's like bickering in the space of like, whether it's politics or funding or egos, it's a different sound to it. And I'm distinctly aware of the two. And I kind of sort of, to me, personally, happiness was just like swimming around the one that like is the political stuff or the money stuff and all that, or egos. And I think that's probably what Perlman is as well. Like the moment he senses there's any, as with the Fields Medal, like the moment you start to have any kind of drama around credit assignment, all those kinds of things, it's almost not that it's important who gets the credit. It's like the drama in itself gets in the way of the exploration of the ideas or the fundamental thing that makes science so damn beautiful. And you can really see that this is also a product of that Russian school of like doing science. And you can see that people were, during the Cold War, a lot of mathematicians, they were not making any money. They were doing math for the sake of math, like for the intellectual pleasure of like solving a difficult problem. And even if it was a flawed system and there were a lot of problems with that, they were able to actually achieve these. And there were a lot of, and Perlman for me is the perfect product of that. He just cared about like working on tough problems. He didn't care about anything else. It was just math, you know, pure math. Yeah, there's like for the broader audience, I think another example of that is like professional sports versus Olympics. Especially in Russia, I've seen that clear distinction where, because the state manages so much of the Olympic process in Russia, as people know with the steroids, yes, yes, yes. But outside of the steroids thing is like the athlete can focus on the pure artistry of the sport. Like not worry about the money, not just in the way they talk about it, the way they think about it, the way they define excellence versus like in the, perhaps a bit of a capitalist system in the United States with American football, with baseball, with basketball. So much of the discussion is about money. Now, of course, at the end of the day, it's about excellence and artistry and all of that. But when the culture is so richly grounded in discussions of money and sort of this capitalistic like merch and businesses and all those kinds of things, it changes the nature of the activity. And it's in a way that's hard again to describe in words, but when it's purely about the activity itself, it's almost like you quiet down all the noise, enough to hear the signal, enough to hear the beauty. Like whenever you're talking about the money, that's when the marketing people come and the business people, the non-creatives come and they fill the room and they create drama and they know how to create the drama and the noise, as opposed to the people who are truly excellent at what they do, the person in their arena, right? Like when you remove all the money and you just let that thing shine, that's when the true excellence can come out. And that was of the few things that worked with the communist system in the Soviet Union, to me at least, as somebody who loves sport and loves mathematics and science, that worked well, removing the money from the picture. Not that I'm saying poverty is good for science. There's some level in which not worrying about money is good for science. It's a weird, I'm not exactly sure what to make of that because capitalism works really damn well, but it's tricky how to find that balance. One Fields Medalist that is interesting to look at, and I think you mentioned it earlier, but it's Cedric Villani, which is, might be the only Fields Medalist that is also a politician now. But so it's this brilliant French mathematician that won the Fields Medal. And after that, he decided that one of the ways that he could have the biggest leverage in pushing science in the direction that he thinks science should go, would be to try to go into politics. And so that's what he did. And he is ran, I'm not sure if he has won any election. I think he's running for mayor of Paris. For mayor of Paris or something like that. But it's this brilliant mathematician that before winning the Fields Medal, had only been just a brilliant mathematician. But after that, he decided to go into politics to try to have an impact and try to change some of the things that he would complain about before. So there's that component as well. Yeah. And I've always thought mathematics and science should be like, James Bond would in my eyes, I think be sexier if he did math. We should as a society put excellence in mathematics at the same level as being able to kill a man with your bare hands. Those are both useful features, that's admirable. It's like, oh, that makes the person interesting. Being extremely well-read about history or philosophy, being good at mathematics, being able to kill a man with bare hands, those are all the same in my book. So I think all are useful for action stars. And I think the society will benefit for giving more value to that. One of the things that bothers me about American culture is the, I don't know the right words to use, but the nerdiness associated with science. I don't think nerd is a good word in American culture because it's seen as weakness. There's images that come with that. And it's fine. You could be all kinds of shapes and colors, and personalities, but to me, having sophisticated knowledge in science, being good at math doesn't mean you're weak. In fact, it could be the very opposite. And so it's an interesting thing because it was very much differently viewed in the Soviet Union. So I know for sure as an existence proof that it doesn't have to be that way. I also feel like we lack a lot of role models. If you ask people to mention one mathematician that they know that is alive today, I think a lot of people would struggle to answer that question. And I also think, I love Neil deGrasse Tyson. But having more role models is good, different kinds of personalities. He has fun, and it's like Bill Nye, the science guy. I don't know if you guys know him. That spectrum. Yeah, but there's not, Feynman is no longer there. Those kinds of personalities. Carl Sagan, man. Even Carl Sagan, yeah. Like a seriousness that's not playful. Not apologetic. Yeah, exactly. Not apologetic about being knowledgeable. And in fact, the kind of energy where you feel self-conscious about not having thought about some of these questions. Just like when I see James Bond, I feel bad about that I don't have never killed a man. I need to make sure I fix that. That's the way I feel. So the same way I want to feel that way. With Carl Sagan talks, I feel like I need to have that same kind of seriousness about science. If I don't know something, I want to know it well. What about Terrence Tau? He's kind of a superstar. What are your thoughts about him? True. I think he's a superstar. What about Terrence Tau? He's kind of a superstar. What are your thoughts about him? True. He's probably one of the most famous mathematicians alive today. And regardless of like, he won a Fields Medal. He's a really smart and talented mathematician. It's also like a big inspiration for us, at least for some of the work that we do with Formaz Library. So Terrence Tau is known for having a big blog and he's pretty open about his research. And he also tries to make his work as public as possible through his blog posts. In fact, there's a really interesting problem that got solved a couple of years ago. So Tau was working on an Erdos problem, actually. So Paul Erdos was this mathematician from Hungary. And he was known for a lot of things, but one of the things that he was also known for was the Erdos problem. So he was always creating these problems and usually associating prizes with those problems. And a lot of those problems are still open. Some of them will be open for maybe a couple hundred years. And I think that's actually an interesting hack for him to collaborate with future mathematicians. His name will keep coming up for future generations. So Tau was working on one of these problems called the Erdos discrepancy and he published a blog post about that problem and he reached a dead end. And then all of a sudden there was this guy from Germany that wrote a comment on his blog post saying, okay, so this problem is a Sudoku flavor and some of the machinery that we're using to solve Sudoku could be used here. And that was actually the key to solve the Erdos discrepancy problem. So there was a comment on his blog and I think that for me is an example of how to do, again, going back to collaborative science online and the power that it has. But Tau is also pretty public about some of the struggles of being a mathematician. And even he wrote about some of the unintended consequences of having extraordinary ability in a field and used himself as an example. When he was growing up, he was extremely talented in mathematics from a young age. Like Tau was a person, he won a medal in like one of the IMOs at the age, I think was a gold medal at the age of 10 or something like that. And so he mentioned that when he was growing up, like, and especially in college, when he was in a class that he enjoyed, it just came very natural for him and he didn't have to work hard to just ace the class. And when he found that the class was boring, like it didn't work and he barely passed. I think in college he almost failed two classes. And he was talking about that and how he brought those studying habits or like in existence of studying habits when he went to Princeton for his PhD. And in Princeton, when he started kind of delving into more complex problems in classes, he struggled a lot because he didn't have those habits. Like he wasn't taking notes and he wasn't studying hard when he faced problems. And he almost failed out of his PhD, he almost failed his PhD exam. And it talks about like having this conversation with his advisor and the advisor pointing out like, you're not, this is not working, you might have to get out of the program. And like how that was a kind of a turning point for him. And like it was super important in his career. So I think Tao is also like this figure that apart from being just an exceptional mathematician, he's also pretty open about what it takes to be a mathematician and some of the struggles of these type of careers. And I think that's super important. In many ways, he's a contributor to open science and open humanity. He's being an open human. True. By communicating, Scott Aronson is another in computer science world who's very different style, very different style. But there's something about a blog that is authentic and real and just gives us a window into the mind and soul of these brilliant folks. So it's definitely a gift. Let me ask you about Formaz Library on Twitter, which I mean, I don't know how to describe it. People should definitely just follow Formaz Library on Twitter. I keep following and unfollowing Formaz Library because it gives, when I follow it leads me down rabbit holes often that are very fruitful. But anyway, so the posts you do on Twitter are just these beautiful, are things that reveal some beautiful aspects of mathematics. Is there something you could say about the approach there? And maybe broadly what you find beautiful about mathematics and then more specifically how you convert that into a rigorous process of revealing that in tweet form. That's a good point. I think there's something about math that a lot of the mathematical content and your papers are like little proofs, has in a way sort of an infinite of life. What I mean by that is that if you look at like Euclid's elements, it's as valid today as it was when he was created like 2000 years ago. And that's not true for a lot of other scientific fields. And so in regards to Twitter, I think there's also a very, it's a very underexplored platform from a learning perspective. I think if you look at content on Twitter, it's very easy to consume. It's very easy to read. And especially when you're trying to explain something, you know, we humans get a dopamine hit if we learn something new. And that's a very, very powerful feeling. And that's why people go to classes when you have a really good professor, it's looking for those dopamine hits. And that's something that we try to explore when we're producing content on Twitter. Imagine if we could, if you would on a line to a restaurant, you could go to your phone to learn something new instead of going to a social network. And I think it's very hard to sometimes to kind of provide that feeling because you need to sometimes digest content and put it in a way that it fits 280 characters. And it requires a lot of sometimes time to do that, even though it's easy to consume, it's hard to make. But once you are able to provide that Eureka moment to people, like that's very powerful to get that whole dopamine hit and like you create this feedback cycle and people come back for more. And in Twitter compared to like, you know, an online course for a book, you have a 0% dropout. So people will read the content. So that it's like, it's part of the creators, like the person that is creating the content, if you're able to actually read the content, if you're able to actually get that feedback cycle, it's super, super powerful. Yeah, but some of this stuff is like, like, how the heck do you find that? And I don't know why it's so appealing. Like, this is from a, what is it? A couple days ago, I'll just read out the number 23456789 is the largest prime number with consecutive increasing digits. I mean, that is so cool. That's like some weird like glimpse into some deep universal truth, even though it's just a number. I mean, that's like so arbitrary. Like, why, why is it so pleasant that that's a thing, but it is in some way, it's almost like it is a little glimpse at some much bigger, like, and I think like, especially if we're talking about science, there's something unique about you go and with a lot of the tweets, you go sometimes from a state of not knowing something to knowing something. And that is very particular to science, science, math, physics, and that again is extra extremely addictive. And that's, that's how I feel about that. And that's why I think people engage so much with with our tweets and go into rabbit holes. And then they, you know, we start with prime numbers and all of a sudden you are spending hours reading number theory things and you go into Wikipedia and you lose a lot of time there. But the variety is really interesting too. There's human things, there's, there's physics things, there's like numeric things, like I just mentioned, but there's also more rigorous mathematical things. There's stuff that's tied to the history of math and the proofs and there's visual, there's animations that are looping animations that are incredible. There are animations that are incredible that reveal something. There's Andrew Wiles on Being Smart. This is just me now like ignoring you guys and just going through. Oh yeah, we're a bit like math drug dealers. We're just trying to get you hooked. We're trying to give you that hit and trying to get you hooked. Yes, some people are brighter than others, but I really believe that most people can really get to quite a good level of mathematics if they're prepared to deal with these psychological issues of how to handle the situation of being stuck. Yeah, there's some truth to that. That's truth. I feel that's like really, it's some truth in terms of research and also about startups. You're stuck a lot of the time before you get to a breakthrough and it's difficult to endure that process of like being stuck because you're not trained to be in that position. I feel, yeah, that's yeah, most people are broken by the stuckness or like they're distracted. Like I've been very cognizant of the fact that more and more social media becomes a thing. Like distractions become a thing that that moment of being stuck is your mind wants to to go do stuff that's unrelated to being stuck and you should be stuck. I'm referring to small stucknesses. Like you're like trying to design something and it's a dead end, basically little dead ends. Dead ends of programming, dead ends in trying to think through something. And then your mind wants to like, this is the problem with this work life balance culture is like take a break. Like as if taking a break will solve everything. Sometimes it solves quite a bit, but like sometimes you need to sit in the stuckness and suffer a little bit and then take a break. But you definitely need to be this. And like most people quit from that psychological battle of being stuck. So success is people who persevere through that. Yeah, yeah. And in the creative process, that's also true. The other day I was I think was reading about this, what is his name, Ed Sheeran, like the musician, was talking a little bit about the creative process and he was using this analogy of a faucet like where when you turn on a faucet it's like dirty water coming out in the beginning. And you just have to keep trusting that at some point your clean, clear water will come out. But you have to endure that process. Like in the beginning it's going to be dirty water and just embrace that. Yeah, actually this the entirety of my YouTube channel and this podcast have been following that philosophy of dirty water. Like I've been, you know, I do believe that. Like you have to get all the crap out of your system first. And sometimes it's all crappy work. I tend to be very self-critical, but I do think that quantity leads to quality for some people. It does for the way my mind works. It's like just keep putting stuff out there, keep creating, and the quality will come as opposed to sitting there waiting, not doing anything until the thing seems perfect because the perfect may never come. But just on like on our Twitter profile, I really, and sometimes when you look on some of those tweets, they might seem like pretty kind of, you know, why is this interesting? It's like so raw, like it's just a number. But I really believe that especially with math or physics, it is possible to get everyone to love math or physics, even if you think you hate it. It's not a function of the student or the person that is on the other side. I think it's just purely a function of like how you explain hidden beauty that they hadn't realized before. It's not easy, but I think it's like a lot of the times is on like on the creator's side to be able to like show that beauty to the other person. I think some of that is native to humans. We just have that curiosity and you look at small toddlers and babies and like them trying to figure things out and there's just something that is born with us that we want for that understanding. We want to figure out the world around us. And so, yeah, it shouldn't be like whether or not people are going to enjoy it. I also really believe that everybody has that capacity to fall in love with math and physics. You mentioned startup. What do you think it takes to build a successful startup? Yeah, that it's what Louise was saying that you need to be able to endure being stuck. And I think the best way to put it is that startups don't have a linear reward function. You oftentimes don't get rewarded for effort. And in most of our lives, we go through these processes that do give you those small rewards for effort. In school, you study hard. Generally, you'll get a good grade and then you get like good grades or you get grades every semester. And so you're slowly getting rewarded and pushed in the right direction. For startups, and startups are not the only thing that is like this, but for startups, you can put in a ton of effort into something and then get no reward for it. It's like Sisyphus boulder where you're pushing that boulder up the mountain and you get to the top and then it just rolls all the way back down. And so that's something that I think a lot of people are not equipped to deal with and can be incredibly demoralizing, especially if that happens more than a few times. But I think it's absolutely essential to power through it because by the nature of startups, it's oftentimes you're dealing with non-obvious ideas and things that might be contrarian. And so you're going to run into that a lot. You're going to do things that are not going to work out and you need to be prepared to deal with that. But if we're not coming out of college, you're just not equipped. I'm not sure if there's a way to train people to deal with those non-linear reward functions, but it's definitely, I think, one of the most difficult things about doing a startup. And also happens in research sometimes. We're talking about the default state is being stuck. You try things, you get zero results, you close doors, you're constantly closing doors until you find something. And yeah, that is a big thing. What about at this point when you're stuck, there's a kind of decision whether if you have a vision to persist through with this direction that you've been going along or what a lot of startups do or businesses is pivot. How do you decide whether to give up on a particular flavor of the way you've imagined the design and to adjust it or completely alter it? I think that's a core question for startups that I've asked myself exactly. And I've never been able to come up with a great framework to make those decisions. I think that's really at the core of a lot of the toughest questions that people that started a company have to deal with. I think maybe the best framework that I was able to figure out is when you run out of ideas. You're exploring something, it's not working, you try it in a different angle, you try a different business model. When you run out of ideas, you don't have any more cards, just switch. It's not perfect. Because you have a lot of stories of startups where people kept pushing and then that paid off. And then you have philosophies like fail fast and pivot fast. So it's hard to balance these two worlds and understand what is the best framework. If you look at Formal's library, you're maybe you can correct me, but it feels like you're operating in a space where there's a lot of things that are broken or could be significantly improved. So it feels like there's a lot of possibilities for pivoting. Or like how do you revolutionize science? How do you revolutionize the aggregation, the animalistic, the animalistic the annotation, the commenting, the community around information of knowledge, structured knowledge. I mean, that's kind of what like Stack Overflow and Stack Exchange has struggled with to come up with a solution. And they've come up, I think, with an interesting set of solutions that are also, I think, flawed in some ways, but they're much, much better than the alternatives. But there's a lot of other possibilities. If we just look at papers, as we talked about, there's so many possible revolutions and there's a lot of money to be potentially made in those revolutions. Plus coupled with that, the benefit to humanity. And so like you're sitting there, like, I don't know how many people are legitimately from a business perspective playing with these ideas. It feels like there's a lot of ideas here. True, there is. Are you right now grinding in a particular direction? Like, is there a like a five year vision that you're thinking in your mind? For us, it's more like a 20 year vision, in the sense that we've consciously tried to make the decision of so we run from outs as it's a side project. And it's a separate in the sense like, it's not what we're working on full time. And, but our thesis there is that we actually think that it's that's a good thing, at least for this stage of from our library. And also because some of these projects, you just, if you're coming from a start from a startup framework, you probably try to try to fit every single idea into something that can change the world within three to five years. And there's just some problems that take longer than that. Right. And so, you know, we're talking about archive, and I'm very doubtful that you could grow, like archive into what it is today, like within two or three years, no matter how much money you throw at it, there's just some things that can take longer. But you need to be able to power through the time that it takes. But if you look at it as okay, this is a company, this is a startup, we have to grow fast, we have to raise money, then, then sometimes you might forgo those ideas because of that, because they don't very well fit into the typical startup framework. And so for us, for Mats, it's something that we're okay with growing, with having it grow slowly, and maybe taking many years. And that's why we think it's, it's not a bad thing that it is a side project, because it makes it much more acceptable in a way, and that to be able to be okay with that. That said, I think what happens is, if you keep pushing new little features, new little ideas, I feel like there's like, certain ideas will just become viral. Like, and then you just won't be able to help yourself, but it'll revolutionize things. It feels like there needs to be, not needs to be, but there's opportunity for viral ideas to change science. Absolutely. And maybe we don't know what those are yet. It might be a very small kind of thing. Maybe you don't even know if, should this be a for-profit company doing these? That's the Wikipedia question. Yeah. There are a lot of questions, like really fundamental questions about this space that we've talked about. I mean, you take Wikipedia and you try to run it as a startup, and by now, we'd have a paywall, you'd be paying 90, 99 a month to read more than 20 articles. Or, I mean, that's one view. Yeah. The other, the ad-driven model. So they rejected the ad-driven model. I don't know if we could, I mean, this is a difficult question. You know, if Archive was supported by ads, I don't know if that's bad for Archive, if Fermat's Library was supported by ads. I don't know, it's not trivial to me. Unlike, I think, a lot of people, I'm not against advertisements. I think ads, when done well, are really good. I think the problem with Facebook and all the social networks are the way, the lack of transparency around the way they use data and the lack of control the users have over their data, not the fact that data is being collected and used to sell advertisements. It's a lack of transparency, lack of control over their data. And I think that's a big problem. If you do a good job of that, I feel like it's a really nice way to make stuff free. Yeah, it's like Stack Overflow, right? I think they've done a good job with that, even though, as we said, they're capturing very little of the value that they're putting out there, right? But it makes it a sustainable company and they're providing a lot of, it's a fantastic and very productive community. Let me ask a ridiculous tangent of a question, Luis. You wrote a paper on Game of Thrones, Battle of Winterfell, just as a side note. I'm sorry, I'm sure you've done a lot of ridiculous stuff like this. I just noticed that particular one. By ridiculous, I mean ridiculously awesome. Can you describe the approach in this work, which I believe is a legitimate publication? So going back to the original, like when we were talking about the backstory of papers and the importance of that. So this was actually when the last season of the show was airing, this was during a company lunch. In the last season, there's a really big battle against the forces of evil and the forces of good, and it's called the Battle of Winterfell. And in this battle, there are these two armies and there's a very particular thing that they have to take into account is that in the army of dead, if someone dies in the army of the living, that person is going to be reborn as a soldier in the army of the dead. And so that was an important thing to take into account. And the initial conditions, as you specify, it's about a hundred thousand on each side. Exactly. So I was able to, based on some images on previous episodes, to figure out what was the size of the armies. And so what we were theorizing was like, how many soldiers does a soldier on the army of the living has to kill in order for them to be able to destroy the army of the dead without losing? Because every time one of the good soldiers dies, it's going to turn into the other side. And so we were theorizing that and I wrote a couple of differential equations and I was able to figure out that based on the size of the armies, I think, was the ratio had to be like 1.7. So it had to kill like 1.7 soldiers of the army of the dead in order for them to win the battle. Yeah, that's science. It's the most powerful. And this is also somehow a pitch for like a hiring pitch in a sense, like this is the kind of important science you do at lunch. Exactly. Well, it turned out to be, you know, for people that have watched these shows, it's like they know that every time you try to predict something that is going to happen, you're going to fail miserably. And that's what happened. So it was not at all important for the show. But we ended up like putting that out and there was a lot of people that shared that. I think it was some like elements of the cast of the show that actually retweeted that and shared that. That was fun. I would love if this kind of calculation happened during the making of the show. I love it, like in, for example, I now know Alex Garland, the director of Ex Machina. And I love it. He doesn't seem to be some, not many people seem to do this, but I love it when directors and people who wrote the story really think through the technical details. Like whether it's knowing like how things, even if it's science fiction, if you were to try to do this, how would you do this? Like Stephen Wolfram and his son were collaborating with the movie Arrival in designing the alien language of how you communicate with the aliens. Like how would you really have a math based language that could span the alien and being and the human being. So I love it when they have that kind of rigor. The Martian was also big on that. Like the book in the movie was all about like can we actually, is this plausible? Can this happen? It was all about that. And that can really bring you in. Like sometimes the small details, I mean the guy that wrote the Martian book is another book that is also filled with those like things that when you realize that, okay, these are grounded in science can just really bring you in. He has a book about a colony on the moon. A colony on the moon and he goes about like all the details that would be required about setting up a colony in the moon and like things that you wouldn't think about. Like the fact that it's hard to bring like air to the moon so they wouldn't like, how do you make that breathable, that environment breathable? You need to bring oxygen, but like you probably wouldn't bring nitrogen. So what you do is like instead of having an atmosphere that is 100% oxygen, you like decrease the pressure so that you have the same ratio of oxygen on earth, but like lowering the pressure here. And so like things like water boils at the lower temperature. So people would have coffee and the coffee would be colder. Like there was a problem in this environment in the moon. So like, and these are like small things in the book, but I studied physics. So like when I read these, that throws me into like tangents and I start researching that. And it's like, I really like to read books and watch movies when they go to that level of detail about science. Yeah. I think Interstellar was one where they also consulted heavily with a number of physicists. I think even resulted in a couple of papers about like the black hole visualizations and yeah, but there's even more examples of interesting science around like these fantasy. We were reading at some point, like these guys that were trying to figure out if the Tolkien's middle earth, if it was round, if it was like a sphere, if it was like a flat earth. Based on the map. Based on the map and some of the references in the books. And so, yeah, we actually, I think we tweeted about that. Yeah, we did. Based on the distance between the cities, you can actually prove that that could be like a map of a sphere or like a spheroid. And you can actually calculate the radius of that planet. That's fascinating. I mean, yeah, that's fascinating. But there's something about like calculating the number, like exactly the calculation you did for the battle of Winterfell is something fascinating about that because that's not like being, that's very mathematical versus like grounded in physics. And that's really interesting. I mean, that's like injecting mathematics into fantasy. There's something magical about that. And that for me, that's why I think it's also when you look at things like Fermat's Last Theorem, like problems that are very kind of self-contained and simple to state. I think like that's the same with that paper. It's very easy to understand the boundaries of the problem. And that for me, that's why those, that's why math is so appealing and those like problems are also so appealing to the general public. It's not that they look simple or that people think that they are easy to like solve, but I feel that a lot of the times they are almost intellectually democratic because everyone understands the starting point. You know, you look at Fermat's Last Theorem, everyone understands like, this is the universe of the problem. And the same maybe with that paper, everyone understands, okay, these are the starting conditions. And yeah, the fact that it becomes intellectually democratic, and I think that's a huge motivation for people. And that's why so many people gravitate towards these like Riemann hypotheses or Fermat's Last Theorem or that simple paper, which is like just one page, it was very simple. And I just talked to somebody, I don't know if you know who he is, Jaco Willink, who is this person who among many things loves military tactics. So he would probably either publish a follow-on paper, maybe you guys should collaborate, but he would see the fundamental, the basic assumptions that you started that paper with as flawed, because you know, there's like dragons too, right? There's like, like you have to integrate tactics, because it's not a homogeneous system. I don't take into account the dragons and like, and he would say tactics fundamentally change the dynamics of the system. And so like, that's what happened. So yeah, so at least from a scientific perspective, he was right, but he never published. So there you go. Let me ask the most important question. You guys are from Portugal, both in Portugal. So who is the greatest soccer player, footballer of all time? Yeah, I think we're a little bit biased on this topic, but I mean, I have a huge, I have a tremendous respect for what- Here we go. This is the political interest. You can convince it. I mean, I have tremendous respect for what Ronaldo has achieved in his career. And I think soccer is one of those sports where I think you can get to maybe be one of the best players in the world. If you just have like natural talent, and even if you don't put a lot of hard work and discipline into soccer, you can be one of the best players in the world. And I think Ronaldo is kind of like, of course he's naturally talented, but he also- Can Ronaldo say the football from Portugal? Exactly, from Portugal, and not the Brazilian in this case. And so, and Ronaldo came from nothing. He's known from being probably one of the hardest working athletes in the game. And I see that sometimes a lot of these discussions about the best player, a lot of people tend to gravitate towards like, this person is naturally talented and the other person has to work hard. And so, as if it was bad, if he had to work hard to be good at something. And I think that so many people fall into that trap. And the reason why so many people fall into that trap is because if you're saying that someone is good and achieved a lot of success by working hard, as opposed to achieving success because he has some sort of God-given natural talent, then you can't explain why the person was born with that. What does it tell you about you? It tells you that maybe if you work hard on a lot of fields, you could have, could accomplish a lot of great things. And I think that's hard to digest for a lot of people. And now Ronaldo is inspiring that- I think so. So you find hard work inspiring, but he's way too good looking. That's the- I don't like him probably. No, I like the part of the hard work and like of him being like one of the hardest working athletes in soccer. So he's to you the greatest of all time? Is he up to- is he- will be number- Okay. I agree. Do you agree with this thing? I agree. I agree. Well, I definitely disagree. I mean, I like him very much. He works hard. I admire, I admire, you know, like he's incredible goal scorer, right? But I- so first of all, Leo Messi, and there was some confusion because I've kept saying Mardona is my favorite player, but I think Leo has surpassed them. So it's Messi, then Mardona, then Pelé for me. But the reason is, there's certain aesthetic definitions of beauty that I admire, whether it came by hard work or through God-given talent or through anything. It doesn't really matter to me. There's certain aesthetic, like genius. When I see it to me, and especially it doesn't have to be consistent. It isn't in the case of Messi and in the case of Ronaldo, but just even moments of genius, which is where Mardona really shines. It- I- Even if that doesn't translate into like results and goals being scored. Right, right. And that's the challenge. I'm like, they did that. Because that's where people that tell me that Leo Messi's never, even on strong teams, have led his, the national team, people aspire to the world cup, right? That's really important. And to me, no, it's the moment, like winning to me was never important. What's more important is the moments of genius. And, but you're talking to the human story and yeah, Cristiano Ronaldo, definitely has a beautiful human story. Yeah. And I think you can't, for me, it's hard to decouple those two. I don't just look at the list of achievements, but I like how he got there and how he keeps pushing the boundaries at like almost 40. And how that sets up an example. Like maybe 10 years ago, I wouldn't have ever imagined that like one of the top players in the world could be a top player at like 37 or- But so, and there's an interesting, the human story is really important, but like, if you look at Ronaldo, he's like, he's somebody like kids could aspire to be. But at the same time, I also like Maradona, who like is a tragic figure in many ways. It's like the, you know, the drugs, the temper, all of those things. That's beautiful too. Like I don't necessarily think to me first, the flaws are beautiful too in athletes. I don't think you need to be perfect. I don't think you need to be perfect from a personality perspective. Those flaws are also beautiful. So, but yeah, there is something about hard work and there's also something about the being an underdog and being able to carry a team. That's an argument from Maradona. I don't know if you can make that argument for Messi and Ronaldo either, because they've all played on superstar teams for most of their lives. So I don't know how, you know, it's difficult to know how they would do when they had to work, like did what Maradona had to do to carry a team on his shoulders. True. And Pelé did as well, depending on the context. Maybe you could argue that with the Portuguese national team, but we have a good team. Yeah, but maybe what Maradona did with Napoli and a couple other teams, it's incredible. It speaks to the beauty of the game that we're talking about all these different players that have, or especially if you're comparing Messi and Ronaldo that have such different styles of play and also even their bodies are so different. But these two very different players can be at the top of the game. And there are not a lot of other sports where you have that, you know? You have kind of a mental image of a basketball player and the top basketball players kind of fit that mental image and they look a certain way. But for soccer, it's not so much like that. And I think that's beautiful. That really adds something to the sport. Well, do you play soccer yourself? Have you played that in your life? What do you find beautiful about the game? Yeah, I mean, it's one of the, I'd say it's the biggest sport in Portugal. And so growing up, we played a lot. Did you see the paper from DeepMind? I didn't look at it where they're like doing some analysis on soccer strategy. Yeah, interesting. I saved that paper. I haven't read it yet. It's actually, when I was in college, I actually did some research on applying machine learning and statistics in sports. In our case, we're doing it for basketball. But what they're effectively trying to do was, have you ever watched Moneyball? So they're trying to do something similar to, right? Taking that, in this case, basketball, taking a statistical approach to basketball. The interesting thing there is that baseball is much more about having these discrete events that happen kind of in similar conditions. And so it's easier to take a statistical approach to it. Whereas basketball, it's a much more dynamic game. It started to measure, it started to replicate these conditions. And so you have to think about it in a slightly different way. And so we were doing work on that and working with the Celtics to analyze the data that they had. They had these cameras in the arena. They were tracking the players. And so they had a ton of data, but they didn't really know what to do with it. And so we were doing work on that. And soccer is maybe even a step further. It's a game where you don't have as many... In basketball, you have a lot of field goals. And so you can measure success. Soccer, it's more of a process on the field. You have a process almost where it's like you have a goal or two in a game. In terms of metrics, I wonder if there's a way... And I've actually have thought about this in the past, never coming up with any good solution. If there's a way to definitively say whether it's messy or not, they're the greatest of all time. Like honestly, sort of measure, like convert the game of soccer into metrics, like you said, baseball. But like those moments of genius, like if it's just about goals or passes that led to goals, that feels like it doesn't capture the genius of the play. There'll be like... You have more metrics, for instance, in chess. And you can try to understand how hard of a move that was. There's like Bobby Fisher has this move that I think it's called the move of the century where you have to go so deep into the tree to understand that that was the right move and you can quantify how hard it was. So it'd be interesting to try to think of those type of metrics, but say for soccer. Computer vision unlocks some of that for us. That's one possibility. I have a cool idea, a computer vision product that you could build for soccer. Let's go. I'm taking notes. If you could detect the ball and imagine that... It seems like totally doable right now, but if you could detect when the ball enters one of the goals and just have a crowd cheering for you when you're playing soccer with your friends every time you score a goal, or you had the Champions League song going on and having that... You go play soccer with your friends, you just turn that on and there's a computer vision program analyzing... Detects the ball. Detects the ball every time there's a goal. If you miss, the fans are reacting to that. And then... It should be pretty simple by now. I think there's an opportunity there. Just throwing that. I'm going to go all out. But by the way, I did... I've never released... I was thinking of just putting it on GitHub, but I did write exactly that, which is the trackers for the players, for the bodies of the player. This is the hard part, actually. The detection of player bodies and the ball is not hard. What's hard is very robust tracking through time of each of those. So I wrote a track that's pretty damn good. Is that open source? You open source? No, I've never released it. Because I thought I need to... This is the perfection thing. Because I knew it was going to be like... It's going to pull me in and it wasn't really that done. And so I've never actually been part of a GitHub project where it's really active development. And I didn't want to make it... I knew there's a non-zero probability that it will become my life for like a half a year. Just how much I love soccer and all those kinds of things. And ultimately, it will be all for just the joy of analyzing the game, which I'm all for. I remember you also... In one of the episodes, you mentioned that you did also a lot of high tracking analysis on Joe Rogan's... That was the research side of my life. Interesting. And you have that library, right? You kind of downloaded all the episodes. Yep. Allegedly. And of course I didn't, if you're a lawyer and listening to this. No, I was listening to the episode where you mentioned that. And actually, there was something that I might ask you for access to that, to like allegedly that library. But I was doing some... Not regarding like eye tracking, but I was playing around with analyzing the distribution of silences on one of the Joe Rogan episodes. So like, I did that for the Elon conversation, where it's like, you just take all the silences, after Joe asked the question and Elon responded, and you plot that distribution and see how that looks like. Yeah. I think there's a huge opportunity, especially with long form podcasts, to do that kind of analysis bigger than Joe. Exactly. But... It has to be a fairly unedited podcast so that you don't get the silences. So one of the benefits I have doing this podcast is like, what we're recording today is there's individual audio being recorded. So like I have the raw information. When it's published, it's all combined together and individual video feed. So even when you're listening, which I usually do, I only show one video stream, I'll know, I can track your blinks and so on. Interesting. But ultimately, the hope is you don't need that raw data, because if you don't need the raw data for whatever analysis you're doing, you can then do a huge number of podcasts. Because it's quickly growing now, the number, especially comedians. There's quite a few comedians with long form podcasts, and they have a lot of facial expressions, they have a lot of fun and all those kinds of things. And it's prone for analysis. Yeah. There's so many interesting things that... That idea actually sparked because I was watching a Q&A by Steve Jobs, and I think it was at MIT. And then people did a talk there, and then the Q&A started and people started asking questions. I was working while listening to it, and someone asked a question, and he goes on a 20 second silence before answering the question. I had to check if the video hadn't paused or something. And I was thinking about if that is a feature of a person, how long on average you take to respond to a question, and if it's like... Oh, that's fascinating....has to do with how thoughtful you are, and if that changes over time. But it also could be... This is a really fascinating metric, because it also could be... It's certainly a feature of a person, but it's also a function of the question. If you normalize to the person, you can probably infer a bunch of stuff about the question. So it's a nice flag. It's a really strong signal, the length of that silence relative to the usual silence they have. So one, the silence is a measure of how thoughtful they are, and two, the particular silence is a measure of how thoughtful the question was. Thoughtful the question was. It's really interesting. I just analyzed Elon's episode, but I think there's room for exploration there. I feel like the average for comedians would be... The time would be so small, because you're trained to... I would think you're reacting to hecklers. You're reacting to all sorts of things. You have to be so quick. Maybe. Yeah. But some of the greatest comedians are very good at sitting in the silence. There's Louis C.K. They play with that, because you have a rhythm. Like Dave Chappelle, a comedian who did Joe's show recently, he has... Especially when he's just having a conversation, he does long pauses. It's kind of cool. It's one of the ways to have people hang in your word, is to play with the pauses, to play with the silences and the emphasis and mid-sentence. There's a bunch of different things that... It'd be interesting to really analyze, but still soccer, to me, is that one. I just want a conclusive, definitive statement about... Because there are so many soccer highlights of both Messi and Ronaldo. I just feel like the raw data is there. I'm just... And decide. Because you don't have that with Pelé and Maradona. True. Yeah, true. But here's a huge amount of high-dev data. The annoying, the difficult thing. This is really hard for tracking. This is actually where I kind of gave up. I didn't really give much effort, but I gave up... The way that highlights, or usually football match, are filmed, is they switch the camera. They'll do a different switch of perspective. It's a really interesting computer vision problem. When the perspective is switched, you still have a lot of overlap about the players, but the perspective is sufficiently different that you have to recompute everything. There are two ways to solve this. One is doing it the full way, where you're constantly doing the slam problem. You're doing a 3D reconstruction the whole time and projecting into that 3D world. But there could be some hacks. I wonder some trick where you can hop. When the perspective shifts, do a high-probability tracking hop from one object to another. You're seeing. I thought, especially in exciting moments when you're passing players, you're doing a single ball dribble across players, and you switch perspective, which is what they often do when you're making a run on goal. If you switch your perspective, it feels like that's going to be really tricky to get right automatically. But in that case, for instance, I feel like if somebody released that data set, or it's like you just have this data set, a master data set of all these games from, say, Ronaldo and Messi, and you just add that in whatever CSV format. And some publicly available data set like that, I feel like people would just... There would be so many cool things that you could do with it, and you just set it free, and then the world would do its thing, and then interesting things would come out of it. By the way, I have this data set. So the two things I've did, of this scale, is soccer. So it's body pose and ball tracking for soccer. And then it's pupil tracking and blink tracking for... It was Joe Rogan and a few other podcasts that I did. So those are the two data sets I have. Did you analyze any of your podcasts? No, I think I really started doing this podcast after doing that work, and it's difficult to... Maybe I'd be afraid of what I find. I'm already annoyed with my own voice and video, like editing it, but perhaps that's the honest thing to do. Because one useful thing about doing computer vision about myself is I know what I was thinking at the time. So you can start to connect the particular, the behavioral peculiarities of the way you blink, the way you squint, the way you close your eyes. Talking about details, for example, I just closed my eyes. Is that a blink or no? Figuring that out in terms of timing, in terms of the blink dynamics, is tricky. It's very doable. I think there's universal laws about what is a blink and what is a closed eye and all those things, plus makeup and eyelashes. I actually have annoyingly long eyelashes. I remember when I was doing a lot of this work, I would cut off my eyelashes. Especially, it's funny, female colleagues were like, what the fuck are you doing? No, keep the eyelashes. Because it got in the way, made the computer vision a lot more difficult. Super interesting topics. Yeah, but speaking about the... Still on the topic of the data sets for sports, there's one paper that I actually annotated on Fermat. It was published in 90s, I believe. 90s or 80s, I forget. But the researcher was effectively looking at the hot end phenomena in basketball. So whether the fact that you just made a field goal, if on your next attempt, if you're more likely to make it or not. It was super interesting because he pulled, I think, 100 undergrads from Stanford and Cornell and asking people, do you think that you have a higher likelihood of making a free throw if you just made one? I think 68% said yes. They believe that. And then he looked at the data, and this was back in, as I said, a few decades ago. And so I think he had a data set of about... He looked at it specifically for free throws, and he had a data set of about 5,000 free throws. And effectively, what he found was that, specifically in the case of free throws, for the aggregate data, he didn't find that... He couldn't really spot that correlation, that hot end correlation. So if you made the first one, you weren't more likely to make the second one. What he did find was that they were just better at the second one because you just got maybe a tiny practice and you just attempted once and then you're going to be better at the next one. And then I went and there's a data set on Kaggle that has 600,000 free throws, and I re-ran the same computations and confirmed you can see a very clear pattern that they're just better at their second free throw. That's interesting. That kind of analysis is so awesome because I think with tennis, they have a fault. When you serve, they have analysis of are you most likely to miss the second serve if you missed first, obviously. I think that's the case. That integrates... That's so cool when psychology is converted into metrics in that way. And in sports, it's especially cool because it's such a constrained system that you can really study human psychology because it's repeated, it's constrained. So many things are controlled, which is something you rarely have in the wild psychological experiments. So it's cool. Plus everyone loves it. Sports is really cool to analyze. People actually care about the results. Yeah. I still think... Well, I will definitely publish this work on Messi versus Ronaldo and- I'd love to read it. Objective, fully objective. I'd love to peer review. Yeah, this is very true. This is not past peer review. Let me ask an advice question to the young folks. You've explored a lot of fascinating ideas in your life. You built a startup, worked on physics, worked in computer science. What advice would you give to young people today in high school, maybe early college about life, about career, about science and mathematics? I remember reading that Pongkario was once asked by a young man, a French journal about his advice for young people and what was his teaching philosophy. And he said that one of the most important things that parents should teach their kids is how to be enthusiastic in regards to the mysteries of the world. And that he said striking that balance was actually one of the most important things in education. You want to have your kids be enthusiastic about the mysteries of the world, but you also don't want to traumatize them if you really force them into something. And I think especially if you're young, I think you should be curious and I think you should explore that curiosity to the fullest, to the point where you even become almost as an expert on that topic. And you might start with something that is small. You might start with you're interested in numbers and how to factor numbers into primes. And then all of a sudden you go and you're lost in number theory and you discover cryptography and then all of a sudden you're buying Bitcoin. And I think you should do this. You should really try to fulfill this curiosity and you should live in a society that allows you to fulfill this curiosity, which is also important. And I think you should do this not to get to some sort of status or fame or money, but I think this is the way, this iterative process, I think this is the way to find happiness. And I think this also allows you to find the meaning for your life. I think it's all about being curious and being able to fulfill that curiosity and that path to fulfilling your curiosity. Yeah, the start small and let the fire build is kind of an interesting way to think about it. And you never know where you're going to end up. For Maaz, it's just a really good example. We started by doing this as an internal thing that we did in the company and then we started putting out there and now a lot of people follow it and know about it. And you still don't know where for Maaz's library is going to end up actually. True, exactly. So yeah, I think that would be my piece of advice with very limited experience, of course. Yeah, I agree. I mean, is there something in particular, Joao, from the computer science versus physics perspective? Do you regret not doing physics? Do you regret not doing computer science? Which one is the wiser, the better human being? This is Messi versus Ronaldo. Those are very, I don't know if you would agree, but they're kind of different disciplines. True. Yeah, very much so. I actually, I was, I had that question in my mind. I took physics classes as an undergrad or like besides what I had to take. And it's definitely something that I considered at some point. And that I do feel like later in life, that might be something that, I'm not sure if regret is the right word, but it's kind of something that I can imagine in an alternative universe, what would have happened if I had gone into physics. I try to think that like, well, it depends on what your path ends up being, but that it's not super important, right? Like exactly what you decide to major on. Like I think there's, I think Tim Urban, like the blogger had a good visualization of this where it's like, he has a picture where you have all sorts of paths that you could pursue in your life. And then maybe you're in the middle of it. And so there's maybe some paths that are not accessible to you, but like the tree that is still in front of you gives you a lot of optionality. And so there's two lessons to learn from that. Like we have a huge number of options now, and probably you're just one to reflect, like to try to derive wisdom from the one little path you've taken so far may be flawed because there's all these other paths you could have taken. So it's like, so one, it's inspiring that you can take any path now. And two, it's like the path you've taken so far is just one of many possible ones, but it does seem that like physics and computer science both open a lot of doors and a lot of different doors. True. It's very interesting. It is. Like in this case, like, and especially in our case, because I could see the difference I studied, I went to college in Europe and Joao went to college here in the US. So I could see the difference in like in the European system is more rigid in the sense that when you decide to study physics, you don't have a lot, especially in the early years, you don't have a lot of, you can't choose to take like a class from like computer science course or something like that. You don't have a lot of freedom to explore in that sense in university, as opposed to here in the US where you have more freedom. And I think that's important. I think that's what constitutes a good kind of educational system is one that gravitates towards the interests of a student as you progress. But I think in order for you to do that, you need to explore different areas. And I felt like if I had a chance to take, say more computer science class when I was in college, I would have probably have taken those classes. But yeah, but I ended up like focusing maybe too much in physics. And I think here at least my perception is that you can explore more fields. But there is a kind of it's funny, but physics can be difficult. So I don't see too many computer science people than exploring into physics. It's like the one, not the one, but one of the beneficial things of physics, it feels like it was it Rutherford that said like, basically that physics is the hard thing and everything is easy. So like, there's a certain sense once you figured out some basic like physics, that it's not that you need the tools of physics to understand the other disciplines, it's that you're empowered by having done difficult shit. I mean, the ultimate, I think, is probably mathematics there. Yeah, true. So maybe just doing difficult things, and proving to yourself that you can do difficult things, whatever those are. That's net positive, I believe that positive. Yeah. And I think like, I before I started a company, I had like, I worked in the financial sector for a bit. And like, I think having a physics background, I was I felt I was not afraid of like learning like finance things. And I think like when you come from those backgrounds, you are generally not afraid of stepping into other fields and learning about those because yeah, I feel they've learned a lot of difficult things. And yeah, that's an added benefit, I believe. This was an incredible conversation, Luis, Joao. We started with who we start with Feynman ended up with Messi and Ronaldo. So this is like the perfect conversation is really an honor that you guys would waste all this time with me today. It was really fun. Thanks for talking. Thank you so much for having us. Yeah, thank you so much. Thanks for listening to this conversation with Luis and Joao Batalla. And thank you to Skiff, Simply Safe, Indeed, NetSuite, and 4Sigmatic. Check them out in the description to support this podcast. And now let me leave you with some words from Richard Feynman. Nobody ever figures out what life is all about. And it doesn't matter. Explore the world. Nearly everything is really interesting if you go into it deeply enough. Thank you for listening. I hope to see you next time.
 The following is a conversation with Luis and Joao Batalla, brothers and co-founders of Fermat's
 Library, which is an incredible platform for annotating papers. As they write on the Fermat's
 Library website, Justice Pierre de Fermat scribbled his famous last theorem in the margins,
 professional scientists, academics, and citizen scientists can annotate equations, figures, ideas,
 and write in the margins. Fermat's Library is also a really good Twitter account to follow.
 I highly recommend it. They post little visual factoids and explorations that reveal the beauty
 of mathematics. I love it. Quick mention of our sponsors. Skiff, Simplisafe, Indeed, Nutsweet,
 and FourSigmatic. Check them out in the description to support this podcast.
 As a side note, let me say a few words about the dissemination of scientific ideas. I believe that
 all scientific articles should be freely accessible to the public. They currently are not. In one
 analysis I saw, more than 70% of published research articles are behind a paywall. In case you don't
 know, the funders of the research, whether that's government or industry, aren't the ones putting
 up the paywall. The journals are the ones putting up the paywall while using unpaid labor from
 researchers for the peer review process. Where is all that money from the paywall going?
 In this digital age, the costs here should be minimal. This cost can easily be covered through
 donation, advertisement, or public funding of science. The benefit versus the cost of all
 papers being free to read is obvious. And the fact that they're not free goes against everything
 science should stand for, which is the free dissemination of ideas that educate and inspire.
 Science cannot be a gated institution. The more people can freely learn and collaborate on ideas,
 the more problems we can solve in the world together. And the faster we can drive old ideas
 out and bring new, better ideas in. Science is beautiful and powerful. And its dissemination
 in this digital age should be free. This is the Lex Friedman Podcast, and here's my conversation
 with Luis and Joao Batalla. Luis, you suggested an interesting idea. Imagine if most papers had a
 backstory section, the same way that they have an abstract. So knowing more about how the authors
 ended up working on a paper can be extremely insightful. And then you went on to give a
 backstory for the Feynman QED paper. This is all in a tweet, by the way. We're doing tweet analysis
 today. How much of the human backstory do you think is important in understanding the idea itself
 that's presented in the paper or in general? I think this gives way more context to the work
 of scientists. I think a lot of people have this almost kind of romantic misconception that
 the way a lot of scientists work is almost as the sum of Eureka moments where all of a sudden
 they sit down and start writing two papers in a row and the papers are usually isolated.
 And when you actually look at it, the papers are chapters of a way more complex story.
 And the Feynman QED paper is a good example. So Feynman was actually going through a pretty dark
 phase before writing that paper. He lost enthusiasm with physics and doing physics problems.
 And there was one time when he was in the cafeteria of Cornell and he saw a guy that was throwing
 plates in the air and he noticed that when the plate was in the air, there were two movements
 there. The plate was wobbling, but he also noticed that the Cornell symbol was rotating.
 And he was able to figure out the equations of motions of those plates. And that led him to
 kind of think a little bit about electron orbits in relativity, which led to the paper about
 quantum electrodynamics. So that kind of reignited his interest in physics and ended up publishing
 the paper that led to his Nobel Prize, basically. And I think there are a lot of really interesting
 backstories about papers that readers never get to know. For instance, we did a couple of months ago
 an AMA around a paper, a pretty famous paper, the Gans paper with Ian Goodfellow. And so we did an
 AMA where everyone could ask questions about the paper and Ian was responding to those questions.
 He was also telling the story of how he got the idea for that paper in a bar. So there was also
 an interesting backstory. I also read a book by Cedric Villani. Cedric Villani is this mathematical
 field medalist. And in his book, he tries to explain how he got from a PhD student to the
 field's medal. And he tries to be as descriptive as possible about every single step how he got
 to the field's medal. And it's interesting also to see just the amount of random interactions and
 discussions with other researchers, sometimes over coffee, and how it led to fundamental breakthroughs
 in some of his most important papers. So I think it's super interesting to have that context of
 the backstory. Well, the Ian Goodfellow story is kind of interesting and perhaps that's true for
 Feynman as well. I don't know if it's romanticizing the thing, but it seems like just a few little
 insights and a little bit of work does most of the leap required. Do you have a sense that for a lot
 of the stuff you've looked at, just looking back through history, it wasn't necessarily the grind
 of like Andrew Wiles or the Fermat's Last Theorem, for example. It was more like a brilliant moment
 of insight. In fact, Ian Goodfellow has a kind of sadness to him almost in that at that time in
 machine learning, like at that time, especially in for GANs, you could code something up really
 quickly in a single machine and almost do the invention, go from idea to experimental validation
 and like a single person could do it. And now there's kind of a sadness that a lot of the
 breakthroughs you might have in machine learning kind of require large scale experiments. So it
 was almost like the early days. So I wonder how many low hanging fruit there are in science and
 mathematics and even engineering where it's like, you could do that little experiment quickly. Like
 you have an insight in a bar. Why is it always a bar? But you have an insight at a bar and then
 just implement and the world changes. It's a good point. I think it also depends a lot on the
 maturity of the field. When you look at a field like mathematics, like it's a pretty mature field,
 a field like machine learning. It's growing pretty fast. And it's actually pretty interesting. I
 looked up like the number of new papers on archive with the keyword machine learning and like 50% of
 those papers have been published in the last 12 months. So you can see just the sense. Five zero,
 50%. So you can see the magnitude of growth in that field. And so I think like as fields mature,
 like those types of moments, I think naturally are less frequent. It's just a consequence of that.
 The other point that is interesting about the backstory is that it can really make it more
 memorable in a way. And by making it more memorable, it kind of sediments the knowledge more.
 In your mind, I remember also reading the sort of the backstory to Dijkstra's shortest path algorithm,
 right? Where he came up with it essentially while he was sitting down at a coffee shop in Amsterdam.
 And he came up with that algorithm over 20 minutes. And one interesting aspect is that he didn't have
 any pen or paper at the time. And so he had to do it all in his mind. And so there's only so much
 complexity that he can handle if you're just thinking about it in your mind. And that like,
 when you think about the simplicity of Dijkstra's shortest path finding algorithm, it's knowing that
 backstory helps sediment that algorithm in your mind so that you don't forget about it as easily.
 It might be from you that I saw a meme about Dijkstra. It's like he's trying to solve it and
 he comes up with some kind of random path. And then it's like, my parents aren't home. And then he
 figures out the algorithm for the shortest path. I tried through words to convey memes,
 but that's hilarious. I don't know if it's in post that we construct stories that romanticize it.
 Apparently with Newton, there was no Apple, especially when you're working on problems
 that have a physical manifestation or a visual manifestation. It feels like the world could be
 an inspiration to you. So it doesn't have to be completely on paper. Like you could be sitting
 at a bar and all of a sudden see something in a pattern will spark another pattern and you can
 visualize it and rethink a problem in a particular way. Of course, you can also load the math that
 you have on paper and always carry that with you. So when you show up to the bar, some little
 inspiration could be the thing that changes it. Is there any other people almost on the human side,
 whether it's physics with Feynman, Dirac, Einstein, or computer science, touring, anybody else?
 Any backstories that you remember that jump out? Because I'm also referring to,
 not necessarily these stories where something magical happens, but these are personalities.
 They have big egos. Some of them are super friendly. Some of them are like self-obsessed.
 Some of them have anger issues. Some of them, how do I describe Feynman? But he appears to
 have appreciation of the beautiful in all its forms. It has a wit and a cleverness and a humor
 about him. Does that come into play in terms of the construction of the science?
 I think you brought up Newton. Newton is a good example also to think about his backstory because
 there's a certain backstory of Newton that people always talk about, but then there's a whole
 another aspect of him that is also a big part of the person that he was. But he was really into
 alchemy and that he spent a lot of time thinking about that and writing about it. He took it very
 seriously. He was really into Bible interpretation and trying to predict things based on the Bible.
 And so there's also a whole backstory then. And of course, you need to look at it in the context
 and the time that when Newton lived, but it adds to his personality and it's important to also
 understand those aspects that maybe people are not as proud to teach to little kids,
 but it's important as part of who he was and maybe without those, who knows what he would have done
 otherwise. Well, the cool thing about alchemy, I don't know how it was viewed at the time,
 but it almost like to me symbolizes dreaming of the impossible. Like most of the breakthrough ideas
 kind of seem impossible until they're actually done. It's like achieving human flight. It's not
 completely obvious to me that alchemy is impossible or like putting myself in the mindset of the time.
 And perhaps even still, everything that some of the most incredible breakthroughs
 are would seem impossible. And I wonder the value of believing almost like focusing and dreaming of
 the impossible such that it is actually is possible in your mind and that in itself manifests,
 whether the accomplishing that goal or making progress in some unexpected direction.
 So alchemy almost symbolizes that for me. I distinctly remember having the same thought
 of thinking, you know, when I learned about atoms and that they have protons and electrons, I was
 like, okay, to make gold, you just take whatever has an atomic weight below it and then shove
 another proton in there and then you have a bunch of gold. So like, why don't people do that? It
 seemed like conceptually is like, you know, this sounds feasible. You might be able to do it.
 And you can actually is just very, very expensive.
 Yeah, yeah, exactly. Exactly. So in a sense, we do have alchemy and maybe even back then,
 it wasn't as crazy that he was so into it. But people just don't like to talk about that as much.
 Yeah, but Newton in general was a very interesting fellow.
 Anybody else come to mind in terms of people that inspire you, in terms of people that you just
 are happy that they have once or still exist on this earth?
 I think I mean, Freeman Dyson for me. Yeah, Freeman Dyson was, was I've had a chance to
 actually exchange a couple of emails with him. It was probably one of the most humble scientists
 that I've ever met. And that had a big impact on me. We were trying, we're actually trying to
 convince him to annotate a paper on Fermat's library. And I sent him an email asking him
 if you could annotate a paper. And his response was something like, I have very limited knowledge.
 I just know a couple of things about certain fields. I'm not sure if I'm qualified to do that.
 That was his first response. And this was someone that should have won a Nobel Prize and worked on
 a bunch of different fields, did some really, really great work. And then just the interactions
 that I had with him, every time I asked him a couple of questions about his papers, and
 he always responded saying, I'm not here to answer your questions. I just want to open
 more questions. And so that had a big impact on me. It was like just an example of an extremely
 humble yet accomplished scientist. And Feynman was also a big, big inspiration in the sense that he
 was able to be, you know, again, extremely talented and, and a scientist, but at the same time,
 socially, he was able to, to, he was also really smart from a social perspective. And he was able
 to interact with people. It was also a really good teacher. And it was also to did awesome work in
 terms of explaining physics to the masses and motivating and getting people interested in
 physics. And that for me was, was also a big inspiration. Yeah. I like the childlike curiosity
 of some of those folks that you mentioned, Freeman. I've Daniel Kahneman, I got a chance to meet and
 interact with some of these truly special scientists. What makes them special is that even
 in older age, there are still like, there's still that fire of childlike curiosity that burns.
 And some of that is like not taking yourself so seriously that you think you figured it all out,
 but almost like thinking that you don't know much of it. And that's like step one in having a great
 conversation or collaboration or exploring a scientific question. And it's cool how the very
 thing that probably earned people the Nobel prize or, or work that's seminal in some way,
 it is the very thing that still burns even after they've won the prize. It's cool to see. And
 they're rare humans, it seems. And to that point, I remember like the last email that I sent to
 Freeman Dyson was like in his last birthday. He was really into number theory and primes. So what
 I did is I took like a photo of him, picture, and then I turned that into like a giant prime number.
 So I converted the picture into a bunch of one and eights, and then I moved some numbers around
 until it was a prime. And then I sent him that. Also the visual, like it still looked like the
 picture, it was made up of a prime. That's tricky to do. It's hard to do. It looks harder to do.
 It's harder than it actually is. So the way you do it is like you convert the darker regions into
 eights and the lighter regions in ones. And then there's-
 And then just keep flipping numbers until-
 But there's like some primality tests that are cheaper from a computational standpoint.
 But what it tells you is it excludes numbers that are not prime. Then you end up with a set of
 numbers that you don't know if they are prime or not. And then you run the full primality test on
 that. So you just have to keep iterating on that. And it was, it's funny because when you got the
 picture, it was like, how did you do that? It was super curious to, and then we got into the details.
 And again, this was, it was already 90, I think 92 or something. And that curiosity was still there.
 So you can really see that in some of these scientists.
 So could we talk about Fermat's library?
 Yeah, absolutely.
 What is it? What's the main goal? What's the dream?
 It is a platform for annotating papers in its essence. And so academic papers can be one of
 the densest forms of content out there and generally pretty hard to understand at times.
 And the idea is that you can make them more accessible and easier to understand by adding
 these rich annotations to the site. And so we can just imagine a PDF view on your browser,
 and then you have annotations on each side. And then when you click on them, a sidebar expands,
 and then you have annotations that support LaTeX and Markdown. And so the idea is that you can
 say, explain a tougher part of a paper where there's a step that is not completely obvious,
 or you can add more context to it. And then over time, papers can become easier and easier
 to understand and can evolve in a way. But it really came from myself, Luis, and two other
 friends. We've had this long running habit of kind of running a journal club amongst us. We come from
 different backgrounds. I studied CS, Luis studied physics, and so we'd read papers and present them
 to each other. And then we tried to bring some of that online. And that's when we decided to
 build Fermat's library. Then over time, it kind of grew into something with a broader goal.
 And really what we're trying to do is trying to help move science in the right direction.
 That's really the ultimate goal and where we want to take it now.
 So there's a lot to be said. So first of all, for people who haven't seen it,
 the interface is exceptionally well done. Execution is really important here.
 Absolutely.
 The other thing is just to mention for a large number of people, apparently, which is new to me,
 don't know what LaTeX is. So it's spelled like LaTeX, so be careful Googling it if you haven't
 before. It's a... Sorry, I don't even know the correct terminology.
 Type setting language?
 It's a type setting language where you're basically writing a program that then generates
 something that looks, from a typography perspective, beautiful.
 So a lot of academics use it to write papers. I think there's a bunch of communities that use
 it to write papers. I would say it's mathematics, physics, computer science.
 That's the main thing. Because I'm collaborating currently on a paper with two neuroscientists
 from Stanford. And they don't know that.
 So I'm using Microsoft Word and Mendeley and all of those kinds of things. And I'm being very
 Zen-like about the whole process, but it's fascinating. It's a little heartbreaking, actually,
 because it's funny to say, and we'll talk about open science, actually the bigger mission behind
 it for Mars Library is really opening up the world of science to everybody. It's these silly
 two facts of one community uses LaTeX and another uses Word is actually a barrier between them.
 It's boring and practical, in a sense, but it makes it very difficult to collaborate.
 Just on that, I think there are some people that should have received a Nobel Prize,
 but will never get it. And I think one of those is Donald Knuth, because of tech and LaTeX.
 Because it had a huge impact in terms of just making it easier for researchers to put their
 content out there, making it uniform as much as possible.
 Oh, you mean like a Nobel Peace Prize?
 Maybe a Nobel Peace Prize. Yeah, I think so.
 I mean, he, at a very young age, got the Turing Award for his work in algorithms and so on.
 I think it's in, it might be even the 60s, but I think it's the 70s. So when he was really young,
 and then he went on to do incredible work with his book and with tech that people don't know.
 And going back just on the reason why we ended up, because I think this is interesting,
 the reason why we ended up using the name Fermat's Library, this was because of
 Fermat's Last Theorem. And Fermat's Last Theorem is actually a funny story. So Pierre de Fermat,
 he was like a lawyer, and he wrote like on a book that he had a solution to Fermat's Last Theorem,
 but that didn't fit the margin of that book.
 And so Fermat's Last Theorem basically states that there's no solution. If you have integers a,
 b, and c, there's no solution to a to the power of n plus b to the power of n equals to c to the
 power of n if n is bigger than two. So there's no solutions. And he said that, and that problem
 remained open for almost 300 years, I believe. And a lot of the most famous mathematicians
 tried to tackle that problem. No one was able to figure that out until Andrew Wiles,
 I think was in the 90s, was able to publish the solution, which was, I believe, almost 300 pages
 long. And so it's kind of an anecdote that there's a lot of knowledge and insights that can be
 trapped in the margins, and there's a lot of potential energy that you can release if you
 can release, if you actually spend some time trying to digest that. And that was the origin
 story for the name. You actually can share the contents of the margins with the world.
 Exactly. That could inspire a solution or a communication that then leads to a solution.
 But if you think about papers, like, papers are, as Joao was saying, probably one of the densest
 pieces of text that any human can read. And you have these researchers, like some of the brightest
 minds in these fields, working on new discoveries and publishing these work on journals that are
 imposing them restrictions in terms of the number of pages that they can have to explain a new
 scientific breakthrough. And so at the end of the day, papers are not optimized for clarity and for
 a proper explanation of that content, because there are so many restrictions. And so there's,
 as I mentioned, there's a lot of potential energy that can be freed if you actually
 try to digest a lot of the contents of papers. Can you explain some of the other things? So
 margins, librarian, journal club. So journal club is what a lot of people know us for,
 where we every week we release an annotated paper and in all sorts of different fields,
 but physics, CS, math, margins is kind of the same software that we use to run the journal club and
 to host the annotations. But we've made that available for free to anybody that wants to use
 it. And so folks use it at universities and for running journal clubs. And so we've just made that
 freely available. And then librarian is a browser extension that we developed that is sort of an
 overlay on top of archive. So it's about bringing some of the same functionality around comments,
 plus adding some extra niceties to archive, like being able to very easily extract the references
 of a paper that you're looking at or being able to extract the bibtech in order to cite that paper
 yourself. So it's an overlay on top of archive. Yeah, the idea is that you can have that commenting
 interface without having to leave archive. It's kind of incredible. I didn't know about it.
 And once I learned of it, it's like, holy shit. Why isn't it more popular given how popular
 archive is? Like everybody should be using it. Archive sucks in terms of its interface.
 Or let me rephrase that, it's limited. Archive is a pretty incredible project. And it is,
 in a way, the growth has been completely linear over time. If you look at like number of papers
 published on archive, it's pretty much a straight line for the past 20 years. Especially if you're
 coming from a startup background and then you're trying to do archive, you'd probably try all sorts
 of growth acts and try to then maybe have paid features and things like that. And that would
 maybe ruin it. And so there's a subtle balance there. And I don't know what aspects you can
 change about it. For some tools in science, it just takes time for them to grow. Archive has just
 turned 30, I believe. And for people that don't know, archive is this kind of online repository
 where people put pre-prints, which are versions of the papers before they actually make it to
 journals. ARXIV for people who don't know. And it's actually a really vibrant place to publish
 your papers in the aforementioned communities of mathematics, physics, and computer science.
 It started with mathematics and physics, and then over the last 30 years it evolved and now,
 actually, computer science now, it's a more popular category than physics and math on
 archive. And there's also, which I don't know very much about, like a biology medical version of that.
 Bioarchive, yeah, bioarchive. It's interesting because if you look at these platforms for
 pre-prints, they actually play a super important role. Because if you look at a category like math,
 for some papers in math, it might take close to three years after you click upload paper on the
 journal website and the paper gets published on the website of the journal. So this is literally
 the longest upload period on the internet. And during those three years, like, it's, you know,
 their content is just, you know, locked. And so that's why it's so important for people to have
 websites like archive so that you can share that before it goes to the journal with the rest of
 the world. There was actually an archive that Perelman published, the three papers that led to
 the proof of the Poincare conjecture. And then you have other fields like machine learning,
 for instance, where the field is evolving at such a high rate that people don't even wait before
 the papers go to journals before they start working on top of those papers. So they publish
 papers, so they publish them on archive, then other people see them, they start working on that.
 And archive did a really good job at like building that core platform to host papers. But I think
 there's a really, really big opportunity in building more features on top of that platform,
 apart from just hosting papers. So collaboration, annotations, and like having other things apart
 from from papers like code and other things. Because, for instance, in the field like machine
 learning, there's a really big, you know, as I mentioned, people start working on on top of
 pre prints, and they are assuming that that that pre print is correct. But you really need a way,
 for instance, to maybe it's not peer review, but distinguish what is good work from bad work on
 archive. How do you do that? So like a commenting interface like library, and it's useful for that
 so that you can distinguish that at at in a field that is growing so fast as machine learning.
 And, and then you have platforms that focus for instance on on just biology, bio archive is a good
 example. bio archive is also super interesting, because there, there's actually an interesting
 experiment that was run in the 60s. So in the 60s, the NIH supported this, this, this experiment
 called the information exchange group, which at the time was a way for researchers to share biology
 pre prints via mail, or using libraries. And that project in the 1960s got canceled six years after
 it started. And it was due to intense pressure from the journals to kill that project, because
 they were fearing competition from from the pre prints for the journal industry. Creek was also
 one of the famous scientists that opposed to the information exchange group. And it's interesting
 because right now, if you analyze the number of biology papers that appear first as pre prints,
 it's only 2% of the papers. And it this was almost 50 almost 50 years after that first experiment.
 So you can see like that pressure from the journals to cancel that initial version of a pre print repo
 had a tremendous impact on on on the number of papers that are showing up in biology as
 pre prints. So it delayed a lot that that revolution and but now platforms like bio archive
 are doing that work. But there's still a lot of room for growth there. And I think it's super
 important because those are the papers that are open that everyone can read. Okay, so but if we
 just look at the entire process of science as a big system, can we just talk about how it can be
 revolutionized? So you have an idea, depending on the field, you want to make that idea concrete,
 you want to run a few experiments in computer science, there might be some code, there'd be a
 data set for you know, some of the more sort of biology, psychology, you might be collecting the
 data set that's called, you know, a study, right? So that's part of that, that's part of the
 methodology. And so you are putting all that into a paper form. And then you have some results.
 And then you you submit that to a place for review through the peer review process. And
 there's a process where how would you summarize the peer review process, but it's really just
 like a handful of people look over your paper and comment and based on that decide whether your
 paper is good or not. So there's a whole broken nature to it. At the same time, I love the peer
 review process when I buy stuff on Amazon, like for like the commenting system, whatever that is.
 So okay, so there's a bunch of possibilities for revolutions there. And then there's the other side,
 which is the collaborative aspect of the science, which is people annotating people commenting,
 sort of the low effort collaboration, which is a comment. Sometimes, as you've talked about a
 comment can change everything, but you know, or a higher effort collaboration, like more like maybe
 annotations or even like contributing to the paper, you can think of like collaborative updating of
 the paper over time. So there's all these possibilities for doing things better than they've
 been done. Can we talk about some ideas in this space, some ideas that you're working on some
 ideas that you're not yet working on, but should be revolutionized? Because it does seem that
 archive and like open review, for example, are like the Craigslist of science. Like, like, yeah,
 okay, I'm very grateful that we have it, but it just feels like it's like 10 to 20 years, like it
 doesn't feel like that's a feature. The simplicity of it is a feature. It feels like it's a it's a
 it's a bug. But then again, the pushback there is Wikipedia has the same kind of simplicity to it.
 And it seems to work exceptionally well in the crowdsourcing aspect of it. So sorry,
 this that's a bunch of stuff going on the table. Let's just pick random things that we can talk
 about. Wikipedia, you know, for me, it's the cosmological constant of the internet. It's,
 I think we are lucky to live in the parallel universe where Wikipedia exists. Yes. Because
 if someone had pitched me Wikipedia, like a publicly edited, I seek an encyclopedia,
 like a couple of years ago, like it would be, I don't know how many people would have said that
 that would have survived. I mean, it makes almost no sense. It's like having a Google Doc that
 everybody on the internet can edit and like, they'll be like the most reliable source for
 knowledge. And I don't know how many but hundreds of thousands of topics. Yeah. It's insane. It's
 insane. And like you have, and then you have users, like there's one, a single user that
 edited one third of the articles on Wikipedia. So we have these really, really big power users,
 there are a substantial part of like, what makes Wikipedia successful. And so, like, no one would
 have ever imagined that that could happen. And so that's one thing I completely agree with what you
 just said. I also- Sorry to interrupt briefly. Maybe let's inject that into the discussion of
 everything else. I also believe, I've seen that with Stack Overflow, that one individual or a small
 collection of individuals contribute or revolutionize most of the community. Like, if you create a
 really powerful system for archive or like open review and made it really easy and compelling
 and exciting for one person who is in like a 10X contributor to do their thing,
 that's going to change everything. It seems like that was the mechanism that changed everything for
 Wikipedia. And that's the mechanism that changed everything for Stack Overflow, is gamifying or
 making it exciting or just making it fun or pleasant or fulfilling in some way for those people who are
 insane enough to like answer thousands of questions or write thousands of factoids and like research
 them and check them all those kinds of things or read thousands of papers. Yeah. No, Stack Overflow
 is another great example of that. And it's just, and those are both two incredibly productive
 communities that generate a ton of value and capture almost none of it. And in a way, it's
 very counterintuitive that these communities would exist and thrive. And it's really hard to,
 there aren't that many communities like that. So how do we do that for science? Do you have
 ideas there? Like what are the biggest problems that you see? You're working on some of them.
 Like just on that, there are a couple of really interesting experiments that people are running.
 An example would be like the polymath projects. So this is a kind of a social experiment that was
 created by Tim Gowers, a Fields Medalist. And his idea was to try to prove that is it possible to
 do mathematics in a massively collaborative way on the internet. So he decided to pick a couple
 of problems and test that. And they found out that it actually is possible for specific types
 of problems, namely problems that you're able to break down in little pieces and go step by step.
 You might need, as with open source, you might need people that are just kind of reorganizing
 the house every once in a while. And then people throw a bunch of ideas and then you make some
 progress, then you reorganize, you reframe the problem, you go step by step. But they were
 actually able to prove that it is possible to collaborate online and do progress in terms of
 mathematics. And so I'm confident that there are other avenues that could be explored here.
 Can we talk about peer review, for example?
 Absolutely. I think like in terms of the peer review, I think it's important to look at the
 bigger picture here of like, of what the scientific publishing ecosystem looks like. Because for me,
 there are a lot of things that are wrong about that entire process. So if you look at, for instance,
 what publishing means in like a traditional journal, you have journals that pay authors
 for their articles, and then they might pay like reviewers to review those articles, and finally
 they pay people to, or distributors to distribute the content. In the scientific publishing world,
 you have scientists that are usually backed by government grants, they are giving away their
 work for free in the form of papers. And then you have other scientists that are reviewing their
 work. This process is known as the peer review process, again, for free. And then finally,
 we have government-backed universities and libraries that are buying back all that work
 so that other scientists can read. So this is, for me, it's bizarre. You have the government
 that is funding the research, it's paying the salaries of the scientists, it's paying the
 salaries of the reviewers, and it's buying back all that product of their work again.
 And I think the problem with this system, and it's why it's so difficult to break this
 suboptimal equilibrium, is because of the way academia works right now and the way you can't
 progress in your academic life. And so in a lot of fields, the competition in academia is really
 insane. So you have hundreds of PhD students, they are trying to get to a professor position,
 and it's hyper competitive. And the only way for you to get there is if you publish papers,
 ideally in journals, with a high impact factor. In computer science, it's often conferences
 are also very prestigious, or actually more prestigious than journals now.
 Okay, interesting. So that's the one discipline that you have to
 work with. I mean, that has to do with the thing we've discussed in terms of how quickly the field
 turns around. But like, NeurIPS, CVPR, those conferences are more prestigious, or at the
 very least as prestigious as the journals. But yeah, but doesn't matter, the process is what it is.
 So for people that don't know, the impact factor of a journal is basically the average number of
 citations that a paper would get if it gets published on that journal. But so you can really
 think that the problem with the impact factor is that it's a way to turn papers into accounting
 units. And let me unpack this, because the impact factor is almost like a nobility title. So because
 papers are born with impact, even before anyone reads them. So the researchers, they don't have
 the incentive to care about if this paper is going to have a long term impact on the world.
 What they care, their goal, their end goal is the paper to get published.
 Yes.
 So that they get that value upfront. So for me, that is one of the problems of that. And that
 really creates a tyranny of metrics. Because at the end of the day, if you are a dean, what you
 want to hire is like people, researchers that publish papers on journals with high impact
 factors, because that will increase the ranking of your university and will allow you to charge
 more for tuition, so on and so forth. And especially when you are in super competitive areas,
 that people will try to gamify that system and misconduct starts showing up. There's a
 really interesting book on this topic called Gaming the Matrix. It's a book by a researcher
 called Mario Biagioli. It goes a lot into how the impact factor and metrics affect science
 negatively. And it's interesting to think, especially in terms of citations, if you look
 at the early work of looking at citations, there was a lot of work that was done by a guy called
 Eugene Garfield. And this guy, the early work in terms of citation, they wanted to use citations
 as from a descriptive point of view. So what they wanted to create was a map. And that map would
 create a visual representation of influence. So citations would be links between papers.
 And ideally, what they would show, they would represent is that you read someone else's paper
 and it had an impact on your research. They weren't supposed to be counted.
 I think this inspired Larry and Sergey's work for Google.
 Exactly. I think they even mentioned that. But what happens is as you start counting citations,
 you create a market. And the same way, the work of Eugene Garfield was a big inspiration for Larry
 and Sergey and for the PageRank algorithm that led to the creation of Google. And they even
 recognized that. And if you think about it, it's like the same way there's a gigantic market for
 search engine optimization, SEO, where people try to optimize the PageRank and how high
 the web page will rank on Google, the same will happen for papers. People will try to optimize
 like the impact factors and the citations that they get. And that creates a really big problem.
 And it's super interesting to actually analyze them. If you look at the distribution of the
 high impact, the impact factors of journals, you have like nature with nature, I believe it's like
 in the low 40s. And then you have, I believe science is high 30s. And then you have a really
 good set of good journals that will fall between 10 and 30. And then you have a gigantic tale of
 journals that have impact factor below two. And you can really see two economies here.
 You see the universities that are maybe less prestigious, less known, where the faculty are
 pressured to just publish papers, regardless of the journal. What I want to do is increase the
 ranking of my university. And so they end up publishing as many papers as they can in like
 journals with low impact factor. And unfortunately, this represents a lot of the global south.
 And then you have the luxury good economy. So for instance, for, and there are also problems here
 in the luxury good economy. So if you look at the journal like nature, so with impact factor of like
 in the low 40s, there's no way that you're going to be able to sustain that level of impact factor
 by just grabbing the attention of scientists. What I mean by that is like, for the journals,
 the articles that get published in nature, they need to be New York times great. So they need to
 make it to the, you know, to the, to the big media, they need to be captured by the big media,
 big media. And because that's the only way for you to capture enough attention to sustain that
 level of citations. And that of course creates problems because people then will try to again,
 gamify the system and have like titles or abstracts or that are bigger claim, make the
 claims that are bigger than what is actually can be sustained by the data or the content of the
 paper. And you'll have clickbait titles or clickbait abstracts. And again, this is all a
 consequence of metrics and science of metrics. And, and this is a very dangerous cycle that I think
 it's very hard to break, but it's happening in academia in a lot of fields right now.
 Is it fundamentally the existence of metrics or the metrics just need to be significantly improved?
 Because like I said, the metrics used for Amazon for purchasing, I don't know, computer parts is
 pretty damn good in terms of selecting which are the good ones, which are not in that same way.
 If we had Amazon type of review system in the space of ideas, in the space of science,
 it feels like that those metrics would be a little bit better. Sort of when it's,
 when it's significantly more open to the crowdsource nature of the internet,
 of the, of the scientific internet, meaning as opposed to like my biggest problem with peer
 review has always been that it's like five, six, seven people, usually even less. And it's often,
 nobody's incentivized to do a good job in the whole process. Meaning it's anonymous in a way that
 doesn't incentivize, like doesn't gamify or incentivize great work. And also it doesn't
 necessarily have to be anonymous. Like there has to be, the entire system is,
 doesn't encourage actual sort of rigorous review. For example, like open review
 does kind of incentivize that kind of process of collaborative review, but it's also imperfect.
 It just feels like the thing that Amazon has, which is like thousands of people
 contributing their reviews to a product. It feels like that could be applied to science.
 Where the same kind of thing you're doing with Formal's library, but doing at a scale that's
 much larger. It feels like that should be possible given the number of grad students,
 given the number of general public that's getting, for example, I personally, as a person
 who got an education in mathematics and computer science, like, and I'm not going to say that
 I'm not a person who got an education in mathematics and computer science. Like I can be a quote unquote
 like reviewer on a lot bigger set of things than is my exact expertise. If I'm one of thousands of
 reviewers, if I'm the only reviewer, one of five, then I better be like an expert in the thing.
 But if I, and I've learned this with COVID, which is like, you can just use your basic skills as a
 data analyst and to contribute to the review process and a particular little aspect of a paper
 and be able to comment, be able to sort of draw in some references that challenge the ideas presented
 or to enrich the ideas that are presented. It just feels like crowdsourcing the review process
 would be able to allow you to have metrics in terms of how good a paper is that are much better
 representative of its actual impact in the world, of its actual value to the world, as opposed to
 some kind of arbitrary gamified version of its impact. I agree with that. I think we, there's
 definitely the possibility, at least for more resilient, a more resilient system than what we
 have today. And that's, I think that's kind of what you're describing, Alex. And I mean, to an
 extent, we kind of have like a little bit of a Heisenberg uncertainty principle. When you pick
 a metric, as soon as you do it, then maybe it works as a good heuristic for a short amount of time,
 but soon enough people will start gamifying. But then you can definitely have metrics that are more
 resilient to gamification and they'll work as a better heuristic to try to push you in the
 in the best direction. But I guess the underlying problem you're saying is there's a shortage of
 positions in academia. That's a big problem for me. Yeah. And that, and so they're going to be
 constantly gamifying the metrics. It's a bit of a zero sum game. It's a very competitive, it's what,
 it's a very competitive field. And that's what usually happens in very competitive fields. Yeah.
 But I think some of like the peer review problems, like scale helps, I think. And it's
 interesting to look at like what you're mentioning, breaking it down, maybe in like smaller parts and
 having more people jumping in. But this is definitely a problem. And the peer review problem,
 as I mentioned, is correlated with the problem of like academic career progression and it's all
 intertwined. And it's, that's why I think it's so hard to break it. There are like a couple of really
 interesting things that are being done right now. There are a couple of, for instance,
 journals that are overlay journals on top of platforms like archive and bio archive
 that want to remove like the more traditional journals from the equation. So essentially a
 journal is just a collection of links to papers. And what they're trying to do is like removing
 that middleman and trying to make the review process a little bit more transparent and not
 charging universities. Like there's a couple of, there are a couple of more famous ones. There's
 one discrete analysis in mathematics. There's one called the quantum journal, which we are actually
 working with them. We have a partnership with them for the papers that get published in quantum
 journal. They also get the annotations on formats and they're doing pretty well. They've been able
 to grow substantially. The problem there is getting to critical mass. So it's again, convincing the
 researchers and especially the young researchers that need that impact factor, need those
 publications to have citations to not publish on the traditional journal and go on an open journal
 and publish their work there. I think there are a couple of really high profile scientists,
 people like Tim Gowers, they are trying to incentivize like famous scientists that already
 have tenure and that don't need that to publish that to increase the reputation of those journals
 so that other, maybe younger scientists can start publishing on those as well. So they can try to
 break that vicious cycle of the more traditional journals. I mean, another possible way to break
 this cycle is to like raise public awareness and just by force, like ban paid journals.
 Like what exactly are they contributing to the world? Like basically making it illegal to, forget
 the fact that it's mostly federally funded. So that's a super ugly picture too, but like why
 should knowledge be so expensive? Like where everyone is working for the public good and then
 there's these gatekeepers that, you know, most people can't read most papers without having to
 pay money and that doesn't make any sense. Like that should be illegal.
 I mean, that's what you're saying is exactly right. I mean, for instance, right, I went to school here
 in the US, we studied in Europe and you would sit like, you'd ask me all the time to download
 papers and send it to him because he just couldn't get it and like papers that he needed for his
 research. And so, but he's a student, like he's a grad student. He was a grad student, but that,
 you know, I'm even referring to just regular people. Oh yeah. Okay. That too. Yeah. And I think
 during 2020 because of COVID, a lot of journals put down the walls for certain kind of coronavirus
 related papers, but like that just gave me an indication that like this should be done for
 everything. It's absurd. Like people should be outraged that there is these gates because,
 so the moment you dissolve the journals, then there'll be an opportunity for startups
 to build stuff on top of archive. It'd be an opportunity for like,
 for Ma's library to step up, to scale up to something much even larger. I mean, that was
 the original dream of Google, which I've always admired, which is make the world's information
 accessible. Actually, it's interesting that Google hasn't, maybe you guys can correct me, but
 they have put together Google scholar, which is incredible, but they, and they've the scanning of
 books, but they haven't really tried to make science accessible in the following way. Like
 besides doing Google scholar, they haven't like delved into the papers, right?
 Which is especially curious given what we was saying, right? That it's kind of in their genesis.
 There's this, you know, research that was very connected with our papers reference each other
 and like building a network out of that. Interesting enough, like Google, I think there was a,
 there was not intended. Google plus was like the Google social network that got canceled was used
 by a lot of researchers with who I think was just a, you know, side kind of a side effect,
 but then a lot of people ended up migrating to Twitter, but it was not on purpose. But yeah,
 I agree with you. Like they haven't gone past Google scholar and, you know, why that said,
 Google scholars incredible for people who are not familiar. It's one of the best aggregation of all
 the scientific work that's out there. And especially the network that connects to all of them, what
 sites, what, and also trying to aggregate all of the versions of the papers that are available there
 and trying to merge them in a way that one particular work, even though it's available in
 a bunch of places, conscious, you know, like a central hub of what that work is across the
 multiple versions. But that almost seems like a fun pet project of a couple of engineers within
 within Google, as opposed to a serious effort to make the world science accessible. But going back
 to just the journals, when you're talking about that Lex, I believe that in that front, I think
 we might be past the event horizon. So I think the, the, the model, the business model for the
 journals, you know, doesn't make sense. They're a middle layer that is not adding a lot of value.
 And you see a lot of motions, whereas like in Europe, a lot of the papers that are get,
 that are funded by the European Union, they will have to be open to the public. And I think there's
 a lot of Bill Gates to like the, the, what the Gates Foundation funds, like the demand that it,
 then it will, that it's accessible to everybody. So I think it's, it's the question of time before
 that, that wall kind of falls. And that is going to help in a lot of possibilities. Because, you
 know, imagine if, if you had like the layer of that gigantic layer of papers all available online,
 you know, that unlocks a lot of potential as a platform for people to build things on top of
 that. But I think it's what you're saying. It is weird. Like you can literally go and listen to any
 song that was ever made on your phone, right? You open Spotify and you might not even pay for it.
 You might be on the free version and you can listen to any song that was ever made pretty much.
 But there's like, you, you don't have access to a huge percentage of academic papers, which is just
 like this fundamental knowledge that we're all funding, but you as an individual don't have
 access to it. And somehow, you know, like the problem for music got solved. But for papers,
 it's still like, it's just not yet. It could be ad supported, all those kinds of things. And then
 hopefully that would change the way we do science. This is the most exciting thing for me is,
 especially once I started like making videos and this silly podcast thing, I started to realize like
 that if you want to do science, one of the most effective ways is to do like couple the paper with
 a set of YouTube videos, like explaining it. Like that also seems like there's a lot of room for
 disruption there. What is the paper 2.0 going to look like? I think like LaTeX and the PDF
 seems like if you, it's interesting, if you look at the first paper that got published in nature,
 and if you look at the paper that got published in nature today, look at the two side by side,
 they are fundamentally the same. And even though like the paper that gets published today, you know,
 you get even, even code, like right now people put like code like on, on a PDF. Like, and there are
 so many things that are related to papers today, you know, you use, you have data, you have code,
 you might need videos to better explain the concepts. So it's, it, I think for me, it's natural
 that there's going to be also an evolution there, that papers are not going to be just the static
 PDFs or LaTeX. There's going to be a next, a next interface. So in academia, a lot of things that
 are judged, you're judged by is often quantity, not quality. I wonder if there's an opportunity
 to have like, I tend to judge people by the best work they've ever done as opposed to,
 I wonder if there's a possibility for that to encourage sort of focusing on the quality
 and not necessarily in paper form, but maybe a subset of a paper, subset of idea, almost even
 a blog post or an experiment. Like why does it have to be published in a journal to be legitimate?
 And it's interesting that you mentioned that. I also think like, yeah, it's why, why, why is
 that the only format? Why can't a blog post or we were even experimenting, experimenting with these
 a few months ago, or can you actually like publish something or like a new scientific breakthrough or
 or something that you've discovered in the form of like a set of tweets,
 a Twitter thread? Why can't that be possible? And we were experimenting with that idea.
 We even, yeah, we ran a couple of, like some people submitted a couple of those,
 like I think the limit was three or four tweets. Maybe it's a new way to look at a, you know,
 a proof or something, but I think it serves to show that there should be other ways to publish
 like scientific discoveries that don't fit the paper format. Well,
 but so even with the Twitter thread, it would be, it'd be nice to have some mechanism of
 formalizing it and making it the stat, making it into an NFT. Okay. Maybe like a concrete thing
 that you can reference with a link that's unique because I mean, everything we've been saying,
 all of that while being true, it's also true that the constraints and the formalism of a paper works
 well. It like forces you constraints forces you to narrow down your thing and literally put it on
 paper. But, you know, I agree, make concrete. And that's why, I mean, it's, it's a, it's a
 I mean, it's not broken. It's just could be better. And that's the main idea. I think there's something
 about writing, whether it's a blog post or Twitter thread or a paper, that's really nice to,
 to concretize a particular little idea that they can then be referenced by other ideas,
 then it can be built on top of with other ideas. So let me ask you've read quite a few papers.
 You've annotated quite a few papers. Let's talk about the process itself. How do you advise people
 read papers or maybe you want to broaden it beyond just papers, but just read concrete pieces of
 information to understand the insights that lay within. I would say for paper specifically,
 I would bring back kind of what Louise was talking about it is that it's important to keep in mind
 that papers are not optimized for ease of understanding. And so, right, there's all
 sorts of restrictions and size and format and language that they can use. And so it's important
 to keep that in mind. And so that if you're struggling to read a paper doesn't that might
 not mean that the underlying material is actually that hard. And so that's definitely something that
 especially for us that we read papers and most of the times it'll be papers that are completely
 outside of our comfort zone, I guess. And so it'd be completely new areas to us. So I always try to
 keep that in mind. So there's usually a certain kind of structure like abstract introductions,
 methodology, depending on the community and so on. Is there something about the process of like,
 how to read it, whether you want to skim it to try to find the parts that are easy to understand or
 not, reading it multiple times? Is there any kind of hacks that you can comment on? I remember like
 Feynman had this kind of hack when he was reading papers where you would basically,
 I think I believe he would read the conclusion of the paper and we would try to just see if he
 would be able to figure out how to get to the conclusion in like a couple of minutes by himself.
 And he would read a lot of papers that way. And I think Fermi also did that almost. And Fermi was
 known for doing a lot of back of the envelope calculations. So he was a master at doing that.
 And in terms of like, especially when reading a paper, I think a lot of times people might
 feel discouraged about the first time you read it, you know, it's very hard to grasp or you don't
 understand a huge fraction of the paper. And I think it's having read a lot of papers in my life,
 I think I've in peace with like the fact that you might spend hours where you're just
 reading a paper and jumping from paper to paper, reading something and then you're like,
 paper to paper, reading citations. And like your level of understanding of sometimes of the paper
 is very close to zero percent. And all of a sudden, you know, everything kind of makes sense in your
 mind. And then, you know, you have this quantum jump where all of a sudden you understand the
 big picture of the paper. And this is an exercise that I have to when reading papers, and especially
 like more complex papers, like, okay, you don't understand because you're just going through the
 process and just keep going. And like, and it might feel super chaotic, especially if you're
 jumping from reference to reference, you know, you might end up with like 20 tabs open, and you're
 reading a ton of other papers, but it's just trusting that process that at the end, like you'll
 find light. And I think for me, that's a good framework. When reading a paper, it's hard,
 because you know, you might end up spending a lot of time and you looks like you're lost. But,
 but that's the process to actually understand what they're talking about in a paper.
 Yeah, I think that process, I enjoy, I've found a lot of value in the process, especially for
 things outside my field, reading a lot of related work sections, and kind of going down that path of
 getting a big context of the field, because what's, especially when they're well written,
 there's opinions injected into the related work, like what work is important, what is not. And if
 you read multiple related work sections that cite or don't cite each other, like the papers,
 you get a sense of where the field, where the tensions of the field are, where the, where the
 field is striving. And that helps you put into context, like whether the work is radical,
 whether it's overselling itself, whether it's underselling itself, all those things.
 And on, added on top of that, I find that often the related work section is the most kind of
 accessible and readable part of a paper, because it's kind of, it's brief to the point, it's trying,
 like summarizing, it's almost like a Wikipedia style article. The introduction is supposed to be
 a compelling story or whatever, but it's often like overselling, there's like an agenda introduction.
 The related work usually has the least amount of agenda, except for the few, like elements where
 you're trying to talk shit about previous work, where you're trying to sell that you're doing much
 better. But other than that, when you're just painting where the, where the field came from,
 or where the field stands, that's really valuable. And also again, just to agree with
 what to agree with, fine. And the conclusion, but it's like, I get a lot of value from the
 breadth first search, kind of read the conclusion, then read the related work, and then
 go through the references in the related work, read the conclusion, read the related work,
 and just go down the tree until you like hit dead ends or run out of coffee. And then through that
 process, you go back up the tree and now you can see the results in their proper, in their proper
 context, unless of course the paper's truly revolutionary, which even that process will
 help you understand that is in fact truly revolutionary. You've also, you talked about
 just following your Twitter thread in depth first search. You talked about that you read the book on
 Grisha Perlman, Gurgurur Perlman, and then you had a really nice Twitter thread on it, and
 you were taking notes throughout. So at a high level, is there suggestions you can give on how
 to take good notes? Whether it's we're talking about annotations or just for yourself to try to
 put on paper ideas as you progress through the work in order to then like understand the work
 better. For me, I always try not to underestimate how much you can forget within six months after
 you've read some. Oh, I thought you were going to say five minutes, but yeah, six months is good.
 Yeah, or even shorter. And so that's something that I always try to keep in mind. And it's
 often, I mean, every once in a while I'll read back a paper that I annotated on Fermat and it's,
 and I'll read through my own annotations and it's, I've completely forgotten what I had written.
 But it also, it's interesting because in a way after you just understood something,
 you're kind of the best possible teacher that can teach your future self. You know,
 after you've forgotten it, you can, you're kind of your own best possible teacher at that moment.
 And so it's, it can be great to try to capture that. It's brilliant. It just made me kind of
 realize it's really nice to put yourself in the position of teaching an older version of yourself
 that returns to this paper, almost like thinking it literally. That's under explored, but it's,
 it's super powerful because you were the person that you can, like, if you, if you look at the
 scale from like one, not knowing anything about the topic and 10, like you are the one that
 progressed from one to 10 and you know which steps you struggled with. So you are the really
 the best person to help yourself make that transition from one to 10. And a lot of the times,
 like, and we don't, I really believe that the framework that we have to like expose ourselves
 to like be talking to like us when we were an expert, when we were taking that class and we
 knew everything about quantum mechanics. And then six months later, you don't remember half of the
 things. How could we make it easier for like to have those conversations between you and your
 past self, past expert self? I think there might be, you know, it's an under explored idea. I think
 notes on paper are probably not the best way. I'm not sure if it's a combination of like video,
 audio, where it's like you have a guided framework that you follow to extract information from
 yourself so that you can later kind of revisit to make it easier to remember. But that's, I think
 it's an interesting idea worth exploring that not, I haven't seen a lot of people kind of
 trying to distill that problem. Yeah, I'm creating the kind of tools I find if I record,
 it sounds weird, but I'll take notes. But if I record audio, like little clips of thoughts,
 like rants, that's really effective at capturing something that notes can't. Because when I replay
 them, for some reason, it loads my brain back into where I was when I was reading that in a way that
 notes don't. Like when I read notes, I'll often be like, whoa, what? What was I thinking there? But
 when I listen to the audio, it brings you right back to that place. So there might, and maybe with
 video, with visual, that might be even more powerful. I think so. Yeah. And I think just the
 process of verbalizing it, that alone kind of makes you have to structure your thought and put
 it in a way that somebody else could come and understand it. And just the process of that is
 useful to organize your thoughts. And yeah, just that alone. Does the Fermat's Library Journal Club
 have a video component or no? Not natively. Sometimes we'll include videos, but it's always
 embedded. Do people like build videos on top of it to explain the paper? Because you're doing all
 the hard work of understanding deeply the paper. Not, we haven't seen that happening too much,
 but we were, we were actually playing around with the idea of creating some sort of podcast version
 where we try to distill the paper on an audio format that not maybe you could have access.
 Might be tricky. Might be trickier, but there are definitely people that could be interested
 in the paper and that topic, but are not willing to read it. But they might listen to a 30 minute
 episode on that paper. Yes. You could reach more people and you might even bring the authors to
 the conversation, but it's tricky. And especially for like more technical papers. We've thought
 about doing that, but we haven't like converged. Sure, if you have any tips. Well, I'm going to
 take that as a small project to take one of the Fermat's almost like half advertisement and half
 as a challenge for myself to take, take one of the annotated papers and like use it as a basis
 for creating a quick video. I, I've seen like, hopefully I'm saying the name correctly, but
 machine learning street talk. I think that's the name of the show. The, that I recommend highly,
 that's the right name, but they, they do exactly that, which is multiple hour breakdown of a paper
 with video component. Sometimes with authors, people love it. It's very effective. There's,
 there's also I've seen, I haven't seen the entire in its entirety, but I've seen like the,
 the founder of comma.ai George. Yeah, George. I've seen him like just taking a paper and then,
 you know, distilling the paper and, and coding it, coding it sometimes during 10 hours. Yeah. And
 he was able to, you know, get a lot of people interested in, in that and viewing him.
 So I'm a huge fan of that. Like, uh, it's George is a personality. I think a lot of people like
 listen to this podcast for the same reason. It's not necessarily the content is they, they, they,
 they like to listen to like a, a silly Russian who's has a childlike brain and mumbles and
 all those like struggle with ideas. Right. And George is a mad man who people just enjoy. Like,
 how is he going to struggle in implementing this particular paper? How's he going to struggle with
 this idea? It's fun to watch. And that actually pulls you in the personality is important there.
 True. But there's, there's, you know, I agree with you, but there also it's visible. Like it's,
 there's an extraordinary ability that is there, like is talented and you need to have,
 there's a craft and this guy definitely has talent and he's doing something that is not easy.
 And I think that also draws the attention of people. Oh yeah. And, and, and it's like the,
 the other day we were actually, we ran into this YouTube channel of this guy that was restoring
 art. Right. And, and, and it was basically just a video of him. Like the production is really,
 like really well done. And he's just him taking really old pieces of, of art, like, and then
 paintings and then restoring them. But he's really good at that. And he describes that process and
 that draws attention, draws the attention of people, regardless of your craft beat,
 like annotating a paper, like restoring it. Yeah. Like George is incredibly good at programming
 like quick, like, you know, those, uh, competitive programmers, like top coder and all those kinds
 of stuff. He has the same kind of element where the brain just jumps around really quickly. And
 that's, uh, yeah, just like, it's also motivation. It's yeah, it's motivating, but, but you're right.
 In, in watching people who are good at what they do, it's motivating. Even if the thing you're
 trying to do is not what they're doing, it's just like contagious when they're really good at it.
 And the same kind of analysis with the paper, I think, so not just like the final result,
 but the process of struggling with it. That's really interesting. Yeah. I think, I mean,
 I think Twitch proved that like, you know, that there's really a market for, for that, for watching
 people do things that they're really good at. And, and you'll just watch it. You will enjoy that,
 that might even spike your interest in that specific topic. And yeah, and people,
 people will enjoy watching sometimes hours on ends of great craftsmen. Do you mind if we talk about
 some of the papers, do any papers come to mind that have been annotated on Formaz library?
 The papers that we annotated can be about completely random topics, but that's part of
 what we enjoy as well. It forces you to explore these topics that otherwise maybe you'd never
 run into. And so, so the ones that come to mind are, to me are fairly random, but one that I
 really enjoyed learning more about is a paper written by mathematician, actually, Tom Apostol,
 and about a tunnel in a Greek island off the coast of Turkey. So it's very random. So this,
 okay, so what's interesting about this tunnel? So this tunnel was built in the sixth century BC.
 And, and it was built in this, in the island of Samos, which is, as I said, off the coast of Turkey.
 And right there, the city on one side, then the other mountain, and then they had a bunch of
 springs on the other side, and they wanted to bring water into the city. Building an aqueduct
 would be pretty hard because of the way the mountain was shaped. And it would also, you know,
 if they, if they were under a siege, like they could just easily destroy that aqueduct, and then
 the water wouldn't have any water supply, the city wouldn't have any water supply. And so they
 decided to build a tunnel. And they decided to try to do it quickly. And so the, they started digging
 from both ends at the same time, through the mountain, right? And so, like, when you start
 thinking about this, it's a fairly difficult problem. And this is like sixth century BC. So
 you had very limited access to, you know, the mathematical tools that you had at the time were
 very limited. And so what this paper is about is about the story of how they built it, and about
 the fact that for about 2000 years, kind of the accepted, the accepted explanation of how they
 built it was actually wrong. And so this tunnel has been famous for a while. There are a number
 of historians that talked about it since ancient Egypt. And the method that they described for
 building it is, was just wrong. And so these researchers went there and were able to figure
 that out. And so basically, kind of the way that they thought they had built it was basically,
 if you can imagine looking at the mountain from the top, and you have the mountain, then you have
 both entrances. And so what they thought, and this is what the ancient historians described,
 is that they effectively tried to draw a right angle, a right angle triangle with the two
 entrances at each end of the hypotenuse. And the way they did is like they would go around the
 mountain and kind of walking in a grid fashion. And then you can, you can figure out the two sides
 of the triangle. And then after you have that triangle, you can effectively draw two smaller
 triangles at each entrance that are proportional to that big triangle. And then you kind of have
 arrows pointing in each way. And then you can, you know, at least that these, that you have a line
 going through the mountain that connects both entrances. The issue with that is like, once you,
 once you go to this mountain and you start thinking of doing this, you realize that,
 especially given that the tools that they had at the time, that your error margin would be too small.
 You wouldn't be able to do it. You, you, you, the, just the fact of, of trying to, to build this
 triangle in that fashion, the error would accumulate and you would end up missing. You'd
 start building these tunnels and they would miss each other. So the task ultimately is to figure
 out like really perfectly as close, as close as possible to the direction you should be digging.
 First of all, that it's possible to have a straight line through and then what that the direction
 would be. And then you are trying to infer that by constructing a right triangle by doing, I, I'm
 not exactly sure about how to do that rigorously, like by tracing the mountain, by walking along the
 mountain, how to, you said grids. Yeah. You, you kind of walk as if you were in a, in a grid. And
 so you just walk in right angles. So, right. So you have to walk really precisely then you have to
 you have to use tools to measure this and the terrain is probably, yeah, very, so this makes
 more sense in 2d and 3d gets even weirder. So, okay. Gotcha. But so this method was described by
 like an ancient Egyptian historian, I think hero of Alexandria. And, and then for about like, yeah,
 for about 2000 years, that's, that's how, like, that's how we thought that they had built this,
 this tunnel. And then, and yeah, and then these researchers went there and found out that actually,
 they must have had to use other methods. And then in this paper, they describe these, these other
 methods. And of course, they can't know for sure. But there's, they presented a bunch of plausible
 alternatives. The one that for me was is the most plausible is that what they probably must have done
 is to use something that is similar to an iron sight on a rifle, the way you can line up your
 rifle with a target off in the distance by by having an iron sight. And, and they they must have
 done something similar to that effectively with three with three sticks. And that way they were
 able to line up sticks along the side of the mountain that were all on the same height. And
 so that then you could get to the other side and you could cut and then you could draw that line.
 So this for me is the most plausible way that they might have done that. And they, but then they,
 they, they described this in detail and other possible approaches in this paper.
 So this is a mathematician doing this?
 Yeah, this is a mathematician that did this.
 Which I suppose is the right mindset instead of skills required to solve an ancient problem.
 Right?
 Yeah.
 Yeah, there's mathematicians and engineers, a lot of things.
 Because they didn't have computers or drones or LIDAR back then or whatever technology you would
 use modern day for civil engineering.
 Yeah. And another fascinating thing is that like, you know, after effectively after the
 the downfall of the Roman civilization, people didn't build tunnels for about a thousand years.
 We go a thousand years without tunnels. And then like only in like in late middle ages that we
 start doing them again. But here is the tunnel, like sixth century BC, like incredibly limited
 mathematics. And they build it in this way. And it was a mystery for a long time exactly how they
 did it. And then these mathematicians went there and basically with no archaeology kind of background
 were able to figure it out.
 How do annotations for this paper look like? What is it? What's a successful annotation
 for a paper like this?
 Yes. So sometimes you're for this paper, sometimes adding some more context on a specific
 part. Like sometimes they mentioned, for instance, these instruments that were common in ancient
 Greece and ancient Rome for building things. And so in some of those annotations, I described
 these instruments in more detail and how they worked because sometimes it can be hard to
 visualize these. Then this paper, I forget exactly when this was published, I believe maybe
 maybe the 70s. But then there was further research into this tunnel and more interesting
 other interesting aspects about it. I add those to that paper as well. There's historical context
 that I also go into there. For instance, the fact that as I said, that effectively after the
 downfall of the Roman Empire, no tunnels were built. Like that's something that I added to
 the paper as well.
 Yeah. So when other people look at the paper, how do they usually consume the annotations?
 So is there a commenting feature? I mean, this is a really enriching experience the way you
 read a paper. What aspects do people usually talk about that they value from this?
 So anybody can just go on there and either add a new annotation or add a comment to an
 existing annotation. And so you can start a thread within an existing annotation.
 And that's something that happens relative frequency. And then because I was the original
 author of the initial annotation, I get pinged. And so oftentimes I'll go back and add on to
 that thread.
 How'd you pick the paper? I mean, first of all, this whole process is really exciting. I'm
 going to, especially after this conversation, I'm going to make sure I participate much more
 actively on papers that I know a lot about and on paper I know nothing about.
 I should get this when I say the paper.
 When I say the paper. I would love to. I also, I mean, I realized that there's a,
 like it's an opportunity for people like me to publicly annotate a paper.
 Like, or do an AMA around the paper.
 Yeah, exactly. But yeah, but like be, be in the conversation about a paper. It's like a place to
 have a conversation about an idea. The other way to do it that's much more ad hoc is on Twitter.
 Right. But this is more like formal and you could actually probably integrate the two.
 They have a conversation about the conversation. So the Twitter is the conversation about a
 conversation. And the main conversation is in the space of annotations.
 There's an interesting effect that we see sometimes with the annotations on our papers is that a lot
 of people, especially if we, the annotations are really well done. People sometimes are afraid
 of adding more annotations because they see that as a kind of a finished work. And so they don't
 want to pollute that or, and especially if it's like a silly question. This is, I don't think
 that's good. I think, you know, we should as much as possible try to lower the barrier for someone
 to jump in and ask questions. I think it only, like most of the times it adds value, but it's
 some feedback that we got from users and readers. I'm not exactly sure how to
 to kind of fight that, but. Well, I think I, I think if I serve as an inspiration in any way
 is by asking a lot of dumb questions and saying a bunch of dumb shit all the time.
 And hopefully that inspires the rest of the other folks to do the same because that's the only way
 to knowledge, I think, is to be willing to ask the dumb questions. And there are papers that are like,
 you know, we have a lot of papers on Formaz where it's just one page or really short papers.
 And you, we have like the shortest paper ever published in a math journal, like with like just
 a couple of words. One of my favorite papers on the platform is actually a paper written by
 Enrico Fermi. And the title of the paper is my, I think is my observations at Trinity.
 So basically Fermi was part of the Manhattan project. So he was in New Mexico when they
 exploded the first atomic bomb. And so it was a couple of miles away from the explosion.
 And he was probably one of the first persons to calculate the energy of the explosion.
 And so the way he did that was he took a piece of paper and he tore down a piece of paper in
 little pieces. And when the bomb exploded, the Trinity bomb was the name of the bomb,
 like he waited for the blast to arrive at where he was. And then he threw those pieces of paper
 in the air and he calculated the energy based on the displacement of the paper, the pieces of paper.
 And then he wrote a report, which was classified until like a couple of years ago, one page report,
 like calculating the energy of the explosion. So bad ass. And we actually went there and kind of
 unpacked and I think he just mentions basically the energy and we actually went and one of the
 annotations is like explaining how he did that. I wonder how accurate he was. It was maybe, I think,
 maybe like 20 or 25% off. Then there was another person that actually calculated the energy based
 on images after the explosion and the rate at which the mushroom of the explosion expanded.
 And it's more accurate to calculate the energy based on that. And I think it was like 20, 20%
 off. But it's really interesting because Fermi was known for all these, being a master at these back
 of the envelope calculations, always like the Fermi problems are well known for that. And it's
 super interesting to see like that just one page report and it was also actually classified. And
 it's interesting because a couple of months ago when the Beirut explosion happened, there was a
 video circulating of this, a bride that was doing a photo shoot when the explosion in Beirut happened.
 And so you can see a video of her with the wedding dress and then the explosion happens and the blast
 arrives at where she was. She was a couple of miles away from the blast. And you can see like
 the displacement of the dress as well. And I actually looked and that video went viral on
 Twitter. And I actually looked at that video and based, I used the same techniques that Fermi used
 to calculate the energy of the explosion based on the displacement of the dress. And you could
 actually see where she was at the distance from the explosion because there was a store behind her
 and you could look the name of the store. And so I calculated that it was the distance and then you
 can... Based on the distance where she was from the explosion and also on the displacement of the
 dress. Because when the blast happens, you can see the dress going back and then going back to the
 original position. And by just looking at how much the dress moved, you can estimate the energy of
 the explosion. I assume you published this? On Twitter it was just a Twitter thread. But actually
 a lot of people share that and it was picked up by a couple of news outlets. I was hoping it would
 be like a formal title and it would be an archive. No, no, no. It may be submitted. Just the Twitter
 thread. But it was interesting because it was exactly the same method that Fermi used. Is there
 something else that jumps to mind? Is there something... I know like in terms of papers,
 I know the Bitcoin paper is super popular. Is there something interesting to be said about
 any of the white papers in the cryptocurrency space? Yeah, the Bitcoin paper was the first paper
 that we put on Fermat. Why that choice as the first paper? It was a while ago and it was one
 of the papers that I read and then kind of explained it to Louise and her two other friends
 that do this journal club with us. I did some research in cryptography as an undergrad and so
 it was a topic that I was interested in. But even for me, I had that background. But reading the
 Bitcoin paper, it took me a few reads to really kind of wrap my head around it. It uses very
 Spartan precise language in a way. It's like you feel like you can't take any word out of it
 without something falling apart. And it's all there. I think it's a beautiful paper and it's
 very well written, of course. But we wanted to try to make it accessible so that anybody that maybe
 is an undergrad in computer science could go on there and know that you have all the information
 in that page that you're going to need to understand the mechanics of Bitcoin.
 And so I explain the basic public key cryptography that you need to know in order to understand it.
 You can explain, okay, what are the properties of a hash function and how they are useful in
 this context. Explain what a Merkle tree is. So a bunch of those basic concepts that maybe if you're
 reading it for the first time and you're an undergrad and you don't know those terms,
 you're going to be discouraged because maybe, okay, now I have to go and Google around until
 I understand these before I can make progress in the paper. And this way it's all there.
 So there's a magic to also to the fact that over time more people went on there and added
 further annotations. So the idea that the paper gets easier and more accessible over time,
 but that's still you're still looking at the original content the way the author intended it
 to be. But there's just more context and the toughest bits have more in-depth explanations.
 I think there's just so many interesting papers there. I remember reading the paper that was
 written by Freeman Dyson on the first time that he explained, he came up with the concept of the
 Dyson sphere and he put that out. Again, it's a one-page paper and what he explained was that
 eventually if a civilization develops and grows, there's going to be a point where
 when the resources on the planet are not enough for the energy requirements of that civilization.
 So if you want to go, the next step is you need to go to the next star and extract energy from that
 star. And the way to do it is you need to build some sort of cap around the star that extracts
 the energy. So he theorized this idea of the Dyson sphere and he went on to kind of analyze
 how you would build that, the stability of that sphere, like if something happens, if there's
 like a small oscillation with that sphere collapsing to the star or no, what would happen.
 And even went on to kind of say that a good way for us to look for signs of intelligent life out
 there is to look for signals of these Dyson spheres. And because according to the second
 law of thermodynamics, there's going to be a lot of infrared radiation that is going to be emitted
 as a consequence of extracting energy from the star. And we should be able to see those signals
 of like infrared if we look at the sky. But all these like from the introduction of the concept,
 like how to build a Dyson sphere, the problems of like having a Dyson sphere,
 how to detect how that could be used as a signal for intelligence life.
 Wait, really? That's all in the paper?
 All in one page paper. And it's like, for me, it's beautiful.
 Where was this published?
 It's fascinating that papers like that could be, I mean, the guts it takes to put that all together
 in a paper form. You know, that kind of challenges our previous discussion of paper. I mean, papers
 can be beautiful. You can play with the format, right?
 But there's a lot to unpack there. That's like the starting point. But it's beautiful that you're
 able to put that in one page and then people can build on top of that.
 But the key ideas are there.
 Yeah, exactly.
 What about, have you looked at any of the big seminal papers throughout the history of science?
 Like you look at simple like Einstein papers. Have any of those been annotated?
 Yeah, yeah, no. We have some more seminal papers that people have heard about. You know, we have
 the DNA double elix paper on there. We have the Higgs boson paper. Yeah, there's papers that we
 know that they're not going to be finding out about them because of us. But it's papers that we think
 should be more widely read and that folks would benefit from having some annotations there.
 And so we also have a number of those.
 A lot of like discovery papers for fundamental particles and all that. We have a lot of those
 on Formas library. I would like to, we haven't annotated that one, but I'd like to on the Riemann
 hypothesis. That's a really interesting paper as well. But we haven't annotated that one,
 but there's a lot of like more historical landmark papers on the platform.
 Have you done Poincare conjecture with Perlman?
 That's too much.
 Too much for me. But it's interesting that, you know, and going back to our discussion,
 like the Poincare paper was like published on archive and it was not on a journal. Like
 the three papers. And what do you make of that? I mean, he's such a fascinating human being.
 Exactly.
 I mentioned to you offline that I'm going to Russia. He's somebody I'm really...
 You should try to interview him.
 Yeah. Well, so I definitely will interview him. And I believe I will. I believe I can.
 I just don't know how to... I know where he lives. So here, okay. My hope is, my conjecture is that
 if I just show up to the house and look desperate enough that, or threatening enough for some
 combination of both, that like the only way to get rid of me is to just get the thing done.
 That's the hope.
 It's actually interesting that you mentioned that because I, after I... So a couple of weeks ago,
 I was searching for like stuff about Perlman online and ended up on this Twitter account of
 this guy that claims to be Perlman's assistant. And he's like, he has been posting a bunch of
 pictures next to Perlman. You can see Perlman in a library and he's next to him taking a selfie,
 or Perlman walking on the street and like, maybe you could reach out to this assistant and I'll
 send you this Twitter account.
 So maybe you're onto something.
 No, but going back to Perlman, it's super interesting because the fact that he published
 the proofs on archive was also a way for him to... Because he really didn't like the scientific
 publishing industry and the fact that you had to pay to get access to articles. And that was a
 form of like protest. And that's why he published those papers there. I mean, I think Perlman is
 just a fascinating like character. And for me, it's this kind of ideal of platonic ideal of what
 a mathematician should be. It's someone that is, just cares about, deeply cares about mathematics.
 You know, he cares about fair attribution of disregards money. And the fact that he published
 on archive is a good example of that.
 What about the Fields Medal that he turned down the Fields Medal? What do you make of that?
 Yeah, I mean, if you look at the reasons why he rejected the Fields Medal. So Perlman did a
 post-doc in the US and when he came back to Russia...
 Do you know how good his English is?
 I think it's fairly good.
 I think it's pretty good, right?
 I think it's really good. Especially in the US.
 Because he's given lectures in the American University.
 But I haven't been able to...
 Listen to anything.
 Well, certainly not listen, but I haven't been able to get anybody... Because I know a lot of
 people have been to those lectures. I'm not able to get a sense of like, yeah, but how strong is
 the accent? What are we talking about here? Is this going to have to be in Russian? Is it going to
 have to be in English? It's fascinating. But he writes the papers in English.
 So... But there's such a fascinating character and there are a couple of examples like him.
 I think in 28 or 29, he proved like a really famous conjecture called the Soul Conjecture.
 I believe it was like in a very short four-page proof of that. It was a really big breakthrough.
 Then he went to Princeton to give a lecture on that. And after the lecture, the chair of the
 math department at Princeton, a guy called Peter Sarnak, went up to Perlman and was trying to
 recruit him, trying to offer him a position at Princeton. And at some point he asked for
 Perlman's resume. And Perlman responded saying, just gave a lecture on like these really tough
 problem. Why do you need my resume? I'm not going to send you. I just proved my value.
 But going back to the Fields Medal, when Perlman went back to Russia, he arrived at a time where
 the salary of post-docs was so much off in regards to inflation that they were not making any money.
 Like people didn't even bother to pick up the checks at the end of the month because they were
 just like ridiculous. But thankfully he had some money that he had gained while he was doing his
 post-docs. So he just concentrated on like the Poincare Conjecture problem, which when he took
 that, he took it after it was reframed by this mathematician called Richard Hamilton, which
 posed the problem in a way that it turned into this super like math Olympiad problem with like
 perfect boundaries, well-defined, and that was perfect for Perlman to attack. And so he spent
 like seven years working on that. And then in 2002, he started publishing those papers on archive.
 And people started jumping on that, reading those papers, and there was like a lot of excitement
 around that. A couple of years later, there were two researchers, I believe they were from Harvard,
 that took Perlman's work, they sent some of the edges, and they republished that, saying that
 based on Perlman's work, they were able to figure out the Poincare Conjecture. And then there was,
 at the time at the International Conference of Mathematics in 2006, I believe that's when they
 were going to give out the Fields Medal, there was a lot of debate of like, oh, who's, like,
 who should get the credit for solving this big problem? And for Perlman, it felt really sad that
 people were even considering that he was not the person that solved that. And the claims that those
 like researchers, when they published after Perlman, there were false claims that they were the ones,
 they just sent it a couple of edges, like Perlman did all the really hard work. And so just the fact
 that they doubted that Perlman had done that, like was enough for him to say, I'm not interested in
 this prize. And that was one of the reasons why he rejected the Fields Medal. He also rejected the
 Clay Prize. So the Poincare Conjecture was one of the Millennium Prizes. There was a lot of
 Millennium Prizes. There was a million dollar prize associated with that problem. And that had to do
 with the fact that for them to attribute that prize, I think it had to be published on a journal.
 Yes. The proof. And again, Perlman's principles of like interfered here. And he also just didn't care
 about the money. He was like, Clay, I think was a businessman and he's like, doesn't have to do
 anything with mathematics. I don't care about these. Like, that's one of the reasons why he
 rejected it. Yeah, it's hard to convert into words, but at MIT, I'm distinctly aware of the
 distinction between when I enter a room, there's a certain kind of music to the way people talk when
 we're talking about ideas versus what that music sounds like when we're talking,
 when it's like bickering in the space of like, whether it's politics or funding or egos,
 it's a different sound to it. And I'm distinctly aware of the two. And I kind of sort of, to me,
 personally, happiness was just like swimming around the one that like is the political stuff
 or the money stuff and all that, or egos. And I think that's probably what Perlman is as well.
 Like the moment he senses there's any, as with the Fields Medal, like the moment you start
 to have any kind of drama around credit assignment, all those kinds of things,
 it's almost not that it's important who gets the credit. It's like the drama in itself gets in the
 way of the exploration of the ideas or the fundamental thing that makes science so damn
 beautiful. And you can really see that this is also a product of that Russian school of like doing
 science. And you can see that people were, during the Cold War, a lot of mathematicians,
 they were not making any money. They were doing math for the sake of math, like for the intellectual
 pleasure of like solving a difficult problem. And even if it was a flawed system and there were a
 lot of problems with that, they were able to actually achieve these. And there were a lot of,
 and Perlman for me is the perfect product of that. He just cared about like working on tough problems.
 He didn't care about anything else. It was just math, you know, pure math.
 Yeah, there's like for the broader audience, I think another example of that is like professional
 sports versus Olympics. Especially in Russia, I've seen that clear distinction where, because the
 state manages so much of the Olympic process in Russia, as people know with the steroids, yes,
 yes, yes. But outside of the steroids thing is like the athlete can focus on the pure artistry of the
 sport. Like not worry about the money, not just in the way they talk about it, the way they think
 about it, the way they define excellence versus like in the, perhaps a bit of a capitalist system
 in the United States with American football, with baseball, with basketball. So much of the discussion
 is about money. Now, of course, at the end of the day, it's about excellence and artistry and all
 of that. But when the culture is so richly grounded in discussions of money and
 sort of this capitalistic like merch and businesses and all those kinds of things,
 it changes the nature of the activity. And it's in a way that's hard again to describe in words,
 but when it's purely about the activity itself, it's almost like you quiet down all the noise,
 enough to hear the signal, enough to hear the beauty. Like whenever you're talking about the
 money, that's when the marketing people come and the business people, the non-creatives come and
 they fill the room and they create drama and they know how to create the drama and the noise,
 as opposed to the people who are truly excellent at what they do, the person in their arena,
 right? Like when you remove all the money and you just let that thing shine, that's when the
 true excellence can come out. And that was of the few things that worked with the communist system
 in the Soviet Union, to me at least, as somebody who loves sport and loves mathematics and science,
 that worked well, removing the money from the picture. Not that I'm saying poverty is good for
 science. There's some level in which not worrying about money is good for science. It's a weird,
 I'm not exactly sure what to make of that because capitalism works really damn well, but it's tricky
 how to find that balance. One Fields Medalist that is interesting to look at, and I think you
 mentioned it earlier, but it's Cedric Villani, which is, might be the only Fields Medalist that
 is also a politician now. But so it's this brilliant French mathematician that won the
 Fields Medal. And after that, he decided that one of the ways that he could have the biggest
 leverage in pushing science in the direction that he thinks science should go, would be to try to
 go into politics. And so that's what he did. And he is ran, I'm not sure if he has won any election.
 I think he's running for mayor of Paris. For mayor of Paris or something like that. But it's this
 brilliant mathematician that before winning the Fields Medal, had only been just a brilliant
 mathematician. But after that, he decided to go into politics to try to have an impact and try to
 change some of the things that he would complain about before. So there's that component as well.
 Yeah. And I've always thought mathematics and science should be like, James Bond
 would in my eyes, I think be sexier if he did math. We should as a society put excellence in
 mathematics at the same level as being able to kill a man with your bare hands. Those are both
 useful features, that's admirable. It's like, oh, that makes the person interesting. Being extremely
 well-read about history or philosophy, being good at mathematics, being able to kill a man with
 bare hands, those are all the same in my book. So I think all are useful for action stars.
 And I think the society will benefit for giving more value to that. One of the things that bothers
 me about American culture is the, I don't know the right words to use, but the nerdiness associated
 with science. I don't think nerd is a good word in American culture because it's seen as weakness.
 There's images that come with that. And it's fine. You could be all kinds of shapes and colors,
 and personalities, but to me, having sophisticated knowledge in science, being good at math
 doesn't mean you're weak. In fact, it could be the very opposite. And so it's an interesting thing
 because it was very much differently viewed in the Soviet Union. So I know for sure as an existence
 proof that it doesn't have to be that way. I also feel like we lack a lot of role models.
 If you ask people to mention one mathematician that they know that is alive today, I think a lot
 of people would struggle to answer that question. And I also think, I love Neil deGrasse Tyson.
 But having more role models is good, different kinds of personalities. He has fun, and it's like
 Bill Nye, the science guy. I don't know if you guys know him. That spectrum.
 Yeah, but there's not, Feynman is no longer there. Those kinds of personalities.
 Carl Sagan, man.
 Even Carl Sagan, yeah. Like a seriousness that's not playful.
 Not apologetic.
 Yeah, exactly. Not apologetic about being knowledgeable. And in fact, the kind of energy
 where you feel self-conscious about not having thought about some of these questions.
 Just like when I see James Bond, I feel bad about that I don't have never killed a man.
 I need to make sure I fix that. That's the way I feel. So the same way I want to feel that way.
 With Carl Sagan talks, I feel like I need to have that same kind of seriousness about science.
 If I don't know something, I want to know it well. What about Terrence Tau?
 He's kind of a superstar. What are your thoughts about him?
 True. I think he's a superstar.
 What about Terrence Tau? He's kind of a superstar. What are your thoughts about him?
 True. He's probably one of the most famous mathematicians alive today.
 And regardless of like, he won a Fields Medal. He's a really smart and talented mathematician.
 It's also like a big inspiration for us, at least for some of the work that we do with
 Formaz Library. So Terrence Tau is known for having a big blog and he's pretty open about
 his research. And he also tries to make his work as public as possible through his blog posts.
 In fact, there's a really interesting problem that got solved a couple of years ago.
 So Tau was working on an Erdos problem, actually. So Paul Erdos was this mathematician
 from Hungary. And he was known for a lot of things, but one of the things that he was also
 known for was the Erdos problem. So he was always creating these problems and usually
 associating prizes with those problems. And a lot of those problems are still open.
 Some of them will be open for maybe a couple hundred years. And I think that's actually
 an interesting hack for him to collaborate with future mathematicians. His name will keep coming
 up for future generations. So Tau was working on one of these problems called the Erdos discrepancy
 and he published a blog post about that problem and he reached a dead end.
 And then all of a sudden there was this guy from Germany that wrote a comment on his blog post
 saying, okay, so this problem is a Sudoku flavor and some of the machinery that we're using to
 solve Sudoku could be used here. And that was actually the key to solve the Erdos discrepancy
 problem. So there was a comment on his blog and I think that for me is an example of how to do,
 again, going back to collaborative science online and the power that it has. But Tau is also pretty
 public about some of the struggles of being a mathematician. And even he wrote about some of
 the unintended consequences of having extraordinary ability in a field and used himself as an example.
 When he was growing up, he was extremely talented in mathematics from a young age. Like Tau was
 a person, he won a medal in like one of the IMOs at the age, I think was a gold medal at the age
 of 10 or something like that. And so he mentioned that when he was growing up, like, and especially
 in college, when he was in a class that he enjoyed, it just came very natural for him and he
 didn't have to work hard to just ace the class. And when he found that the class was boring,
 like it didn't work and he barely passed. I think in college he almost failed two classes.
 And he was talking about that and how he brought those studying habits or like in existence of
 studying habits when he went to Princeton for his PhD. And in Princeton, when he started kind of
 delving into more complex problems in classes, he struggled a lot because he didn't have those
 habits. Like he wasn't taking notes and he wasn't studying hard when he faced problems.
 And he almost failed out of his PhD, he almost failed his PhD exam. And it talks about like
 having this conversation with his advisor and the advisor pointing out like, you're not,
 this is not working, you might have to get out of the program. And like how that was a kind of a
 turning point for him. And like it was super important in his career. So I think Tao is also
 like this figure that apart from being just an exceptional mathematician, he's also pretty open
 about what it takes to be a mathematician and some of the struggles of these type of careers. And I
 think that's super important. In many ways, he's a contributor to open science and open humanity.
 He's being an open human. True. By communicating, Scott Aronson is another in computer science
 world who's very different style, very different style. But there's something about a blog that
 is authentic and real and just gives us a window into the mind and soul of these brilliant folks.
 So it's definitely a gift. Let me ask you about Formaz Library on Twitter, which I mean, I don't
 know how to describe it. People should definitely just follow Formaz Library on Twitter. I keep
 following and unfollowing Formaz Library because it gives, when I follow it leads me down rabbit
 holes often that are very fruitful. But anyway, so the posts you do on Twitter are just these
 beautiful, are things that reveal some beautiful aspects of mathematics. Is there something you
 could say about the approach there? And maybe broadly what you find beautiful about mathematics
 and then more specifically how you convert that into a rigorous process of revealing that in tweet
 form. That's a good point. I think there's something about math that a lot of the mathematical
 content and your papers are like little proofs, has in a way sort of an infinite of life. What
 I mean by that is that if you look at like Euclid's elements, it's as valid today as it was when he was
 created like 2000 years ago. And that's not true for a lot of other scientific fields.
 And so in regards to Twitter, I think there's also a very, it's a very underexplored platform
 from a learning perspective. I think if you look at content on Twitter, it's very easy to consume.
 It's very easy to read. And especially when you're trying to explain something, you know,
 we humans get a dopamine hit if we learn something new. And that's a very, very powerful feeling.
 And that's why people go to classes when you have a really good professor, it's looking for those
 dopamine hits. And that's something that we try to explore when we're producing content on Twitter.
 Imagine if we could, if you would on a line to a restaurant, you could go to your phone to learn
 something new instead of going to a social network. And I think it's very hard to sometimes to
 kind of provide that feeling because you need to sometimes digest content and put it in a way
 that it fits 280 characters. And it requires a lot of sometimes time to do that, even though
 it's easy to consume, it's hard to make. But once you are able to provide that Eureka moment to
 people, like that's very powerful to get that whole dopamine hit and like you create this feedback
 cycle and people come back for more. And in Twitter compared to like, you know, an online course
 for a book, you have a 0% dropout. So people will read the content. So that it's like, it's part of
 the creators, like the person that is creating the content, if you're able to actually read the
 content, if you're able to actually get that feedback cycle, it's super, super powerful.
 Yeah, but some of this stuff is like, like, how the heck do you find that? And I don't know why
 it's so appealing. Like, this is from a, what is it? A couple days ago, I'll just read out the
 number 23456789 is the largest prime number with consecutive increasing digits.
 I mean, that is so cool. That's like some weird like glimpse into some deep universal truth,
 even though it's just a number. I mean, that's like so arbitrary. Like, why, why is it so pleasant
 that that's a thing, but it is in some way, it's almost like it is a little glimpse at some much
 bigger, like, and I think like, especially if we're talking about science, there's something
 unique about you go and with a lot of the tweets, you go sometimes from a state of not knowing
 something to knowing something. And that is very particular to science, science, math, physics,
 and that again is extra extremely addictive. And that's, that's how I feel about that. And
 that's why I think people engage so much with with our tweets and go into rabbit holes. And then
 they, you know, we start with prime numbers and all of a sudden you are spending hours reading
 number theory things and you go into Wikipedia and you lose a lot of time there. But
 the variety is really interesting too. There's human things, there's, there's physics things,
 there's like numeric things, like I just mentioned, but there's also more rigorous
 mathematical things. There's stuff that's tied to the history of math and the proofs and
 there's visual, there's animations that are looping animations that are incredible. There are
 animations that are incredible that reveal something. There's Andrew Wiles on Being Smart.
 This is just me now like ignoring you guys and just going through. Oh yeah, we're a bit like
 math drug dealers. We're just trying to get you hooked. We're trying to give you that hit
 and trying to get you hooked. Yes, some people are brighter than others, but I really believe
 that most people can really get to quite a good level of mathematics if they're prepared to deal
 with these psychological issues of how to handle the situation of being stuck. Yeah, there's some
 truth to that. That's truth. I feel that's like really, it's some truth in terms of research and
 also about startups. You're stuck a lot of the time before you get to a breakthrough and it's
 difficult to endure that process of like being stuck because you're not trained to be in that
 position. I feel, yeah, that's yeah, most people are broken by the stuckness or like they're
 distracted. Like I've been very cognizant of the fact that more and more social media becomes a
 thing. Like distractions become a thing that that moment of being stuck is your mind wants to
 to go do stuff that's unrelated to being stuck and you should be stuck. I'm referring to small
 stucknesses. Like you're like trying to design something and it's a dead end, basically little
 dead ends. Dead ends of programming, dead ends in trying to think through something. And then your
 mind wants to like, this is the problem with this work life balance culture is like take a break.
 Like as if taking a break will solve everything. Sometimes it solves quite a bit, but like sometimes
 you need to sit in the stuckness and suffer a little bit and then take a break. But you definitely
 need to be this. And like most people quit from that psychological battle of being stuck. So
 success is people who persevere through that. Yeah, yeah. And in the creative process, that's
 also true. The other day I was I think was reading about this, what is his name, Ed Sheeran, like the
 musician, was talking a little bit about the creative process and he was using this analogy
 of a faucet like where when you turn on a faucet it's like dirty water coming out in the beginning.
 And you just have to keep trusting that at some point your clean, clear water will come out.
 But you have to endure that process. Like in the beginning it's going to be dirty water and just
 embrace that. Yeah, actually this the entirety of my YouTube channel and this podcast have been
 following that philosophy of dirty water. Like I've been, you know, I do believe that. Like you have
 to get all the crap out of your system first. And sometimes it's all crappy work.
 I tend to be very self-critical, but I do think that quantity leads to quality for some people.
 It does for the way my mind works. It's like just keep putting stuff out there, keep creating,
 and the quality will come as opposed to sitting there waiting, not doing anything until
 the thing seems perfect because the perfect may never come.
 But just on like on our Twitter profile, I really, and sometimes when you look on some of those
 tweets, they might seem like pretty kind of, you know, why is this interesting? It's like so raw,
 like it's just a number. But I really believe that especially with math or physics,
 it is possible to get everyone to love math or physics, even if you think you hate it.
 It's not a function of the student or the person that is on the other side. I think it's just
 purely a function of like how you explain hidden beauty that they hadn't realized before.
 It's not easy, but I think it's like a lot of the times is on like on the creator's side
 to be able to like show that beauty to the other person.
 I think some of that is native to humans. We just have that curiosity and you look at small
 toddlers and babies and like them trying to figure things out and there's just something that is born
 with us that we want for that understanding. We want to figure out the world around us.
 And so, yeah, it shouldn't be like whether or not people are going to enjoy it.
 I also really believe that everybody has that capacity to fall in love with math and physics.
 You mentioned startup. What do you think it takes to build a successful startup?
 Yeah, that it's what Louise was saying that you need to be able to endure being stuck.
 And I think the best way to put it is that startups don't have a linear reward function.
 You oftentimes don't get rewarded for effort. And in most of our lives, we go through
 these processes that do give you those small rewards for effort. In school, you study hard.
 Generally, you'll get a good grade and then you get like good grades or you get grades every
 semester. And so you're slowly getting rewarded and pushed in the right direction. For startups,
 and startups are not the only thing that is like this, but for startups, you can put in a ton of
 effort into something and then get no reward for it. It's like Sisyphus boulder where you're
 pushing that boulder up the mountain and you get to the top and then it just rolls all the way back
 down. And so that's something that I think a lot of people are not equipped to deal with and can be
 incredibly demoralizing, especially if that happens more than a few times. But I think it's
 absolutely essential to power through it because by the nature of startups, it's oftentimes you're
 dealing with non-obvious ideas and things that might be contrarian. And so you're going to run
 into that a lot. You're going to do things that are not going to work out and you need to be
 prepared to deal with that. But if we're not coming out of college, you're just not equipped.
 I'm not sure if there's a way to train people to deal with those non-linear reward functions,
 but it's definitely, I think, one of the most difficult things about doing a startup. And
 also happens in research sometimes. We're talking about the default state is being stuck.
 You try things, you get zero results, you close doors, you're constantly closing doors until you
 find something. And yeah, that is a big thing. What about at this point when you're stuck,
 there's a kind of decision whether if you have a vision to persist through with this direction
 that you've been going along or what a lot of startups do or businesses is pivot. How do you
 decide whether to give up on a particular flavor of the way you've imagined the design and to
 adjust it or completely alter it? I think that's a core question for startups that I've asked
 myself exactly. And I've never been able to come up with a great framework to make those decisions.
 I think that's really at the core of a lot of the toughest questions that people that
 started a company have to deal with. I think maybe the best framework that I
 was able to figure out is when you run out of ideas. You're exploring something, it's not
 working, you try it in a different angle, you try a different business model. When you run out of
 ideas, you don't have any more cards, just switch. It's not perfect. Because you have a lot of stories
 of startups where people kept pushing and then that paid off. And then you have philosophies like
 fail fast and pivot fast. So it's hard to balance these two worlds and understand what is the best
 framework. If you look at Formal's library, you're maybe you can correct me, but it feels like
 you're operating in a space where there's a lot of things that are broken or could be significantly
 improved. So it feels like there's a lot of possibilities for pivoting. Or like how do you
 revolutionize science? How do you revolutionize the aggregation, the animalistic, the animalistic
 the annotation, the commenting, the community around information of knowledge, structured
 knowledge. I mean, that's kind of what like Stack Overflow and Stack Exchange has struggled with
 to come up with a solution. And they've come up, I think, with an interesting set of solutions
 that are also, I think, flawed in some ways, but they're much, much better than the alternatives.
 But there's a lot of other possibilities. If we just look at papers, as we talked about,
 there's so many possible revolutions and there's a lot of money to be potentially made
 in those revolutions. Plus coupled with that, the benefit to humanity. And so like you're sitting
 there, like, I don't know how many people are legitimately from a business perspective playing
 with these ideas. It feels like there's a lot of ideas here. True, there is. Are you right now
 grinding in a particular direction? Like, is there a like a five year vision that you're thinking in
 your mind? For us, it's more like a 20 year vision, in the sense that we've consciously tried to make
 the decision of so we run from outs as it's a side project. And it's a separate in the sense like,
 it's not what we're working on full time. And, but our thesis there is that we actually think that
 it's that's a good thing, at least for this stage of from our library. And also because some of
 these projects, you just, if you're coming from a start from a startup framework, you probably try to
 try to fit every single idea into something that can change the world within three to five years.
 And there's just some problems that take longer than that. Right. And so, you know, we're talking
 about archive, and I'm very doubtful that you could grow, like archive into what it is today,
 like within two or three years, no matter how much money you throw at it, there's just some things
 that can take longer. But you need to be able to power through the time that it takes. But if you
 look at it as okay, this is a company, this is a startup, we have to grow fast, we have to raise
 money, then, then sometimes you might forgo those ideas because of that, because they don't very
 well fit into the typical startup framework. And so for us, for Mats, it's something that we're
 okay with growing, with having it grow slowly, and maybe taking many years. And that's why we
 think it's, it's not a bad thing that it is a side project, because it makes it much more
 acceptable in a way, and that to be able to be okay with that. That said, I think what happens is,
 if you keep pushing new little features, new little ideas, I feel like there's like,
 certain ideas will just become viral. Like, and then you just won't be able to help yourself, but
 it'll revolutionize things. It feels like there needs to be, not needs to be, but there's
 opportunity for viral ideas to change science. Absolutely. And maybe we don't know what those
 are yet. It might be a very small kind of thing. Maybe you don't even know if, should this be a
 for-profit company doing these? That's the Wikipedia question. Yeah. There are a lot of questions,
 like really fundamental questions about this space that we've talked about. I mean, you take
 Wikipedia and you try to run it as a startup, and by now, we'd have a paywall, you'd be paying 90,
 99 a month to read more than 20 articles. Or, I mean, that's one view. Yeah. The other,
 the ad-driven model. So they rejected the ad-driven model. I don't know if we could,
 I mean, this is a difficult question. You know, if Archive was supported by ads,
 I don't know if that's bad for Archive, if Fermat's Library was supported by ads. I don't know,
 it's not trivial to me. Unlike, I think, a lot of people, I'm not against advertisements. I think
 ads, when done well, are really good. I think the problem with Facebook and all the social networks
 are the way, the lack of transparency around the way they use data and the lack of control the
 users have over their data, not the fact that data is being collected and used to sell advertisements.
 It's a lack of transparency, lack of control over their data. And I think that's a big
 problem. If you do a good job of that, I feel like it's a really nice way to make stuff free.
 Yeah, it's like Stack Overflow, right? I think they've done a good job with that,
 even though, as we said, they're capturing very little of the value that they're putting out there,
 right? But it makes it a sustainable company and they're providing a lot of,
 it's a fantastic and very productive community. Let me ask a ridiculous tangent of a question,
 Luis. You wrote a paper on Game of Thrones, Battle of Winterfell, just as a side note.
 I'm sorry, I'm sure you've done a lot of ridiculous stuff like this. I just noticed
 that particular one. By ridiculous, I mean ridiculously awesome. Can you describe the
 approach in this work, which I believe is a legitimate publication?
 So going back to the original, like when we were talking about the backstory of papers and the
 importance of that. So this was actually when the last season of the show was airing, this was
 during a company lunch. In the last season, there's a really big battle against the forces of evil
 and the forces of good, and it's called the Battle of Winterfell. And in this battle,
 there are these two armies and there's a very particular thing that they have to take into
 account is that in the army of dead, if someone dies in the army of the living, that person is
 going to be reborn as a soldier in the army of the dead. And so that was an important thing to
 take into account. And the initial conditions, as you specify, it's about a hundred thousand on each
 side. Exactly. So I was able to, based on some images on previous episodes, to figure out what
 was the size of the armies. And so what we were theorizing was like, how many soldiers does a
 soldier on the army of the living has to kill in order for them to be able to destroy the army of
 the dead without losing? Because every time one of the good soldiers dies, it's going to turn into
 the other side. And so we were theorizing that and I wrote a couple of differential equations and
 I was able to figure out that based on the size of the armies, I think, was the ratio had to be
 like 1.7. So it had to kill like 1.7 soldiers of the army of the dead in order for them to win
 the battle. Yeah, that's science. It's the most powerful. And this is also somehow a pitch
 for like a hiring pitch in a sense, like this is the kind of important science you do at lunch.
 Exactly. Well, it turned out to be, you know, for people that have watched these shows, it's like
 they know that every time you try to predict something that is going to happen, you're going
 to fail miserably. And that's what happened. So it was not at all important for the show. But
 we ended up like putting that out and there was a lot of people that shared that. I think it was
 some like elements of the cast of the show that actually retweeted that and shared that.
 That was fun. I would love if this kind of calculation happened during the making of the
 show. I love it, like in, for example, I now know Alex Garland, the director of Ex Machina.
 And I love it. He doesn't seem to be some, not many people seem to do this, but I love it when
 directors and people who wrote the story really think through the technical details. Like whether
 it's knowing like how things, even if it's science fiction, if you were to try to do this, how would
 you do this? Like Stephen Wolfram and his son were collaborating with the movie Arrival in designing
 the alien language of how you communicate with the aliens. Like how would you really have a math
 based language that could span the alien and being and the human being. So I love it when they have
 that kind of rigor. The Martian was also big on that. Like the book in the movie was all about like
 can we actually, is this plausible? Can this happen? It was all about that. And that can really
 bring you in. Like sometimes the small details, I mean the guy that wrote the Martian book is
 another book that is also filled with those like things that when you realize that, okay, these are
 grounded in science can just really bring you in. He has a book about a colony on the moon.
 A colony on the moon and he goes about like all the details that would be required about setting
 up a colony in the moon and like things that you wouldn't think about. Like the fact that
 it's hard to bring like air to the moon so they wouldn't like, how do you make that
 breathable, that environment breathable? You need to bring oxygen, but like you probably wouldn't
 bring nitrogen. So what you do is like instead of having an atmosphere that is 100% oxygen,
 you like decrease the pressure so that you have the same ratio of oxygen on earth, but like lowering
 the pressure here. And so like things like water boils at the lower temperature. So people would
 have coffee and the coffee would be colder. Like there was a problem in this environment in the
 moon. So like, and these are like small things in the book, but I studied physics. So like when I
 read these, that throws me into like tangents and I start researching that. And it's like,
 I really like to read books and watch movies when they go to that level of detail about science.
 Yeah. I think Interstellar was one where they also consulted heavily with a number of
 physicists. I think even resulted in a couple of papers about like the black hole visualizations
 and yeah, but there's even more examples of interesting science around like these fantasy.
 We were reading at some point, like these guys that were trying to figure out if the Tolkien's
 middle earth, if it was round, if it was like a sphere, if it was like a flat earth.
 Based on the map. Based on the map and some of the references in the books. And so,
 yeah, we actually, I think we tweeted about that. Yeah, we did. Based on the distance between the
 cities, you can actually prove that that could be like a map of a sphere or like a spheroid.
 And you can actually calculate the radius of that planet.
 That's fascinating. I mean, yeah, that's fascinating. But there's something about
 like calculating the number, like exactly the calculation you did for the battle of Winterfell
 is something fascinating about that because that's not like being, that's very mathematical
 versus like grounded in physics. And that's really interesting. I mean, that's like injecting
 mathematics into fantasy. There's something magical about that. And that for me, that's why
 I think it's also when you look at things like Fermat's Last Theorem, like problems that are very
 kind of self-contained and simple to state. I think like that's the same with that paper. It's
 very easy to understand the boundaries of the problem. And that for me, that's why those,
 that's why math is so appealing and those like problems are also so appealing to the general
 public. It's not that they look simple or that people think that they are easy to like solve,
 but I feel that a lot of the times they are almost intellectually democratic because everyone
 understands the starting point. You know, you look at Fermat's Last Theorem, everyone understands
 like, this is the universe of the problem. And the same maybe with that paper, everyone understands,
 okay, these are the starting conditions. And yeah, the fact that it becomes intellectually
 democratic, and I think that's a huge motivation for people. And that's why so many people gravitate
 towards these like Riemann hypotheses or Fermat's Last Theorem or that simple paper, which is like
 just one page, it was very simple. And I just talked to somebody, I don't know if you know who
 he is, Jaco Willink, who is this person who among many things loves military tactics. So
 he would probably either publish a follow-on paper, maybe you guys should collaborate,
 but he would see the fundamental, the basic assumptions that you started that paper with as
 flawed, because you know, there's like dragons too, right? There's like, like you have to integrate
 tactics, because it's not a homogeneous system. I don't take into account the dragons and like,
 and he would say tactics fundamentally change the dynamics of the system. And so like,
 that's what happened. So yeah, so at least from a scientific perspective, he was right,
 but he never published. So there you go. Let me ask the most important question. You guys are
 from Portugal, both in Portugal. So who is the greatest soccer player, footballer of all time?
 Yeah, I think we're a little bit biased on this topic, but I mean, I have a huge, I have a tremendous
 respect for what- Here we go. This is the political interest.
 You can convince it. I mean, I have tremendous respect for what Ronaldo has achieved in his career.
 And I think soccer is one of those sports where I think you can get to maybe be one of the best
 players in the world. If you just have like natural talent, and even if you don't put a lot
 of hard work and discipline into soccer, you can be one of the best players in the world.
 And I think Ronaldo is kind of like, of course he's naturally talented, but he also-
 Can Ronaldo say the football from Portugal? Exactly, from Portugal, and not the Brazilian
 in this case. And so, and Ronaldo came from nothing. He's known from being probably one
 of the hardest working athletes in the game. And I see that sometimes a lot of these discussions
 about the best player, a lot of people tend to gravitate towards like, this person is naturally
 talented and the other person has to work hard. And so, as if it was bad, if he had to work hard
 to be good at something. And I think that so many people fall into that trap. And the reason why so
 many people fall into that trap is because if you're saying that someone is good and achieved a lot
 of success by working hard, as opposed to achieving success because he has some sort of God-given
 natural talent, then you can't explain why the person was born with that. What does it tell you
 about you? It tells you that maybe if you work hard on a lot of fields, you could have, could
 accomplish a lot of great things. And I think that's hard to digest for a lot of people.
 And now Ronaldo is inspiring that- I think so.
 So you find hard work inspiring, but he's way too good looking.
 That's the- I don't like him probably.
 No, I like the part of the hard work and like of him being like one of the hardest working
 athletes in soccer. So he's to you the greatest of all time? Is he up to- is he- will be number-
 Okay. I agree.
 Do you agree with this thing? I agree. I agree.
 Well, I definitely disagree. I mean, I like him very much. He works hard. I admire, I admire,
 you know, like he's incredible goal scorer, right? But I- so first of all, Leo Messi,
 and there was some confusion because I've kept saying Mardona is my favorite player, but I think
 Leo has surpassed them. So it's Messi, then Mardona, then Pelé for me. But the reason is,
 there's certain aesthetic definitions of beauty that I admire, whether it came by hard work
 or through God-given talent or through anything. It doesn't really matter to me. There's certain
 aesthetic, like genius. When I see it to me, and especially it doesn't have to be consistent.
 It isn't in the case of Messi and in the case of Ronaldo, but just even moments of genius,
 which is where Mardona really shines. It- I-
 Even if that doesn't translate into like results and goals being scored.
 Right, right. And that's the challenge. I'm like, they did that.
 Because that's where people that tell me that Leo Messi's never, even on strong teams, have led his,
 the national team, people aspire to the world cup, right? That's really important. And to me,
 no, it's the moment, like winning to me was never important. What's more important is the moments
 of genius. And, but you're talking to the human story and yeah, Cristiano Ronaldo,
 definitely has a beautiful human story. Yeah. And I think you can't, for me, it's hard to decouple
 those two. I don't just look at the list of achievements, but I like how he got there and
 how he keeps pushing the boundaries at like almost 40. And how that sets up an example.
 Like maybe 10 years ago, I wouldn't have ever imagined that like one of the top players in
 the world could be a top player at like 37 or-
 But so, and there's an interesting, the human story is really important, but like,
 if you look at Ronaldo, he's like, he's somebody like kids could aspire to be. But at the same
 time, I also like Maradona, who like is a tragic figure in many ways. It's like the, you know,
 the drugs, the temper, all of those things. That's beautiful too. Like I don't necessarily think to
 me first, the flaws are beautiful too in athletes. I don't think you need to be perfect. I don't
 think you need to be perfect from a personality perspective. Those flaws are also beautiful.
 So, but yeah, there is something about hard work and there's also something about the being an
 underdog and being able to carry a team. That's an argument from Maradona. I don't know if you can
 make that argument for Messi and Ronaldo either, because they've all played on superstar teams for
 most of their lives. So I don't know how, you know, it's difficult to know how they would do
 when they had to work, like did what Maradona had to do to carry a team on his shoulders.
 True.
 And Pelé did as well, depending on the context.
 Maybe you could argue that with the Portuguese national team, but we have a good team.
 Yeah, but maybe what Maradona did with Napoli and a couple other teams, it's incredible.
 It speaks to the beauty of the game that we're talking about all these different players that
 have, or especially if you're comparing Messi and Ronaldo that have such different styles of play
 and also even their bodies are so different. But these two very different players can be at the top
 of the game. And there are not a lot of other sports where you have that, you know? You have
 kind of a mental image of a basketball player and the top basketball players kind of fit that mental
 image and they look a certain way. But for soccer, it's not so much like that. And I think that's
 beautiful. That really adds something to the sport.
 Well, do you play soccer yourself? Have you played that in your life? What do you find beautiful about the game?
 Yeah, I mean, it's one of the, I'd say it's the biggest sport in Portugal. And so growing up, we played a lot.
 Did you see the paper from DeepMind? I didn't look at it where they're like doing some analysis on
 soccer strategy.
 Yeah, interesting. I saved that paper. I haven't read it yet. It's actually, when I was in college,
 I actually did some research on applying machine learning and statistics in sports.
 In our case, we're doing it for basketball. But what they're effectively trying to do was,
 have you ever watched Moneyball? So they're trying to do something similar to, right?
 Taking that, in this case, basketball, taking a statistical approach to basketball.
 The interesting thing there is that baseball is much more about having these discrete events that
 happen kind of in similar conditions. And so it's easier to take a statistical approach to it.
 Whereas basketball, it's a much more dynamic game. It started to measure, it started to replicate these
 conditions. And so you have to think about it in a slightly different way. And so we were doing work
 on that and working with the Celtics to analyze the data that they had. They had these cameras in
 the arena. They were tracking the players. And so they had a ton of data, but they didn't really
 know what to do with it. And so we were doing work on that. And soccer is maybe even a step further.
 It's a game where you don't have as many... In basketball, you have a lot of field goals.
 And so you can measure success. Soccer, it's more of a process on the field.
 You have a process almost where it's like you have a goal or two in a game.
 In terms of metrics, I wonder if there's a way... And I've actually have thought about this in the past,
 never coming up with any good solution. If there's a way to definitively say whether it's messy or
 not, they're the greatest of all time. Like honestly, sort of measure, like convert the
 game of soccer into metrics, like you said, baseball. But like those moments of genius,
 like if it's just about goals or passes that led to goals, that feels like it doesn't capture
 the genius of the play. There'll be like... You have more metrics, for instance, in chess.
 And you can try to understand how hard of a move that was. There's like Bobby Fisher has this move
 that I think it's called the move of the century where you have to go so deep into the
 tree to understand that that was the right move and you can quantify how hard it was.
 So it'd be interesting to try to think of those type of metrics, but say for soccer.
 Computer vision unlocks some of that for us. That's one possibility.
 I have a cool idea, a computer vision product that you could build for soccer.
 Let's go. I'm taking notes.
 If you could detect the ball and imagine that... It seems like totally doable right now,
 but if you could detect when the ball enters one of the goals and just have a crowd cheering for
 you when you're playing soccer with your friends every time you score a goal, or you had the
 Champions League song going on and having that... You go play soccer with your friends,
 you just turn that on and there's a computer vision program analyzing...
 Detects the ball.
 Detects the ball every time there's a goal. If you miss, the fans are reacting to that.
 And then...
 It should be pretty simple by now. I think there's an opportunity there.
 Just throwing that.
 I'm going to go all out. But by the way, I did... I've never released... I was thinking of just
 putting it on GitHub, but I did write exactly that, which is the trackers for the players,
 for the bodies of the player. This is the hard part, actually.
 The detection of player bodies and the ball is not hard. What's hard is very robust tracking
 through time of each of those. So I wrote a track that's pretty damn good.
 Is that open source? You open source?
 No, I've never released it. Because I thought I need to...
 This is the perfection thing. Because I knew it was going to be like...
 It's going to pull me in and it wasn't really that done.
 And so I've never actually been part of a GitHub project where it's really active development.
 And I didn't want to make it... I knew there's a non-zero probability that it will become my life
 for like a half a year. Just how much I love soccer and all those kinds of things.
 And ultimately, it will be all for just the joy of analyzing the game, which I'm all for.
 I remember you also... In one of the episodes, you mentioned that you did also a lot of high
 tracking analysis on Joe Rogan's... That was the research side of my life.
 Interesting. And you have that library, right? You kind of downloaded all the episodes.
 Yep. Allegedly. And of course I didn't, if you're a lawyer and listening to this.
 No, I was listening to the episode where you mentioned that. And actually,
 there was something that I might ask you for access to that, to like allegedly that library.
 But I was doing some... Not regarding like eye tracking, but I was playing around with
 analyzing the distribution of silences on one of the Joe Rogan episodes. So like,
 I did that for the Elon conversation, where it's like, you just take all the silences,
 after Joe asked the question and Elon responded, and you plot that distribution and see how that
 looks like. Yeah. I think there's a huge opportunity, especially with long form podcasts,
 to do that kind of analysis bigger than Joe. Exactly. But...
 It has to be a fairly unedited podcast so that you don't get the silences.
 So one of the benefits I have doing this podcast is like, what we're recording today is there's
 individual audio being recorded. So like I have the raw information. When it's published,
 it's all combined together and individual video feed. So even when you're listening,
 which I usually do, I only show one video stream, I'll know, I can track your blinks and so on.
 Interesting. But ultimately, the hope is you don't need that raw data, because if you don't need the
 raw data for whatever analysis you're doing, you can then do a huge number of podcasts. Because
 it's quickly growing now, the number, especially comedians. There's quite a few comedians with
 long form podcasts, and they have a lot of facial expressions, they have a lot of fun and all those
 kinds of things. And it's prone for analysis. Yeah. There's so many interesting things that...
 That idea actually sparked because I was watching a Q&A by Steve Jobs, and I think it was at MIT.
 And then people did a talk there, and then the Q&A started and people started asking questions.
 I was working while listening to it, and someone asked a question, and he goes on a 20 second
 silence before answering the question. I had to check if the video hadn't paused or something.
 And I was thinking about if that is a feature of a person, how long on average you take
 to respond to a question, and if it's like... Oh, that's fascinating.
...has to do with how thoughtful you are, and if that changes over time.
 But it also could be... This is a really fascinating metric, because it also could be...
 It's certainly a feature of a person, but it's also a function of the question.
 If you normalize to the person, you can probably infer a bunch of stuff about the question. So it's
 a nice flag. It's a really strong signal, the length of that silence relative to the usual
 silence they have. So one, the silence is a measure of how thoughtful they are, and two,
 the particular silence is a measure of how thoughtful the question was.
 Thoughtful the question was. It's really interesting.
 I just analyzed Elon's episode, but I think there's room for exploration there.
 I feel like the average for comedians would be... The time would be so small,
 because you're trained to... I would think you're reacting to hecklers. You're reacting
 to all sorts of things. You have to be so quick.
 Maybe. Yeah. But some of the greatest comedians are very good at sitting in the silence.
 There's Louis C.K. They play with that, because you have a rhythm. Like Dave Chappelle, a comedian
 who did Joe's show recently, he has... Especially when he's just having a conversation,
 he does long pauses. It's kind of cool. It's one of the ways to have people hang in your word,
 is to play with the pauses, to play with the silences and the emphasis and mid-sentence.
 There's a bunch of different things that... It'd be interesting to really analyze, but still soccer,
 to me, is that one. I just want a conclusive, definitive statement about... Because there are
 so many soccer highlights of both Messi and Ronaldo. I just feel like the raw data is there.
 I'm just... And decide.
 Because you don't have that with Pelé and Maradona.
 True. Yeah, true.
 But here's a huge amount of high-dev data. The annoying, the difficult thing. This is
 really hard for tracking. This is actually where I kind of gave up. I didn't really give much effort,
 but I gave up... The way that highlights, or usually football match, are filmed, is they
 switch the camera. They'll do a different switch of perspective. It's a really interesting computer
 vision problem. When the perspective is switched, you still have a lot of overlap about the players,
 but the perspective is sufficiently different that you have to recompute everything.
 There are two ways to solve this. One is doing it the full way, where you're constantly doing the
 slam problem. You're doing a 3D reconstruction the whole time and projecting into that 3D world.
 But there could be some hacks. I wonder some trick where you can hop. When the perspective shifts,
 do a high-probability tracking hop from one object to another.
 You're seeing.
 I thought, especially in exciting moments when you're passing players, you're doing a single ball
 dribble across players, and you switch perspective, which is what they often do when you're making a
 run on goal. If you switch your perspective, it feels like that's going to be really tricky to get
 right automatically.
 But in that case, for instance, I feel like if somebody released that data set, or it's like
 you just have this data set, a master data set of all these games from, say, Ronaldo and Messi,
 and you just add that in whatever CSV format. And some publicly available data set like that,
 I feel like people would just... There would be so many cool things that you could do with it,
 and you just set it free, and then the world would do its thing, and then interesting things
 would come out of it.
 By the way, I have this data set. So the two things I've did,
 of this scale, is soccer. So it's body pose and ball tracking for soccer.
 And then it's pupil tracking and blink tracking for... It was Joe Rogan and a few other podcasts
 that I did. So those are the two data sets I have.
 Did you analyze any of your podcasts?
 No, I think I really started doing this podcast after doing that work, and it's difficult to
... Maybe I'd be afraid of what I find. I'm already annoyed with my own voice and video,
 like editing it, but perhaps that's the honest thing to do. Because one useful thing about
 doing computer vision about myself is I know what I was thinking at the time. So you can start to
 connect the particular, the behavioral peculiarities of the way you blink, the way you squint,
 the way you close your eyes. Talking about details, for example, I just closed my eyes.
 Is that a blink or no? Figuring that out in terms of timing, in terms of the blink dynamics,
 is tricky. It's very doable. I think there's universal laws about what is a blink and what
 is a closed eye and all those things, plus makeup and eyelashes. I actually have annoyingly long
 eyelashes. I remember when I was doing a lot of this work, I would cut off my eyelashes.
 Especially, it's funny, female colleagues were like, what the fuck are you doing? No, keep the
 eyelashes. Because it got in the way, made the computer vision a lot more difficult.
 Super interesting topics.
 Yeah, but speaking about the... Still on the topic of the data sets for sports, there's
 one paper that I actually annotated on Fermat. It was published in 90s, I believe. 90s or 80s,
 I forget. But the researcher was effectively looking at the hot end phenomena in basketball.
 So whether the fact that you just made a field goal, if on your next attempt,
 if you're more likely to make it or not. It was super interesting because he pulled,
 I think, 100 undergrads from Stanford and Cornell and asking people, do you think that
 you have a higher likelihood of making a free throw if you just made one? I think 68% said yes.
 They believe that. And then he looked at the data, and this was back in, as I said, a few decades ago.
 And so I think he had a data set of about... He looked at it specifically for free throws,
 and he had a data set of about 5,000 free throws. And effectively, what he found was that,
 specifically in the case of free throws, for the aggregate data, he didn't find that... He couldn't
 really spot that correlation, that hot end correlation. So if you made the first one,
 you weren't more likely to make the second one. What he did find was that they were just better
 at the second one because you just got maybe a tiny practice and you just attempted once and then
 you're going to be better at the next one. And then I went and there's a data set on Kaggle
 that has 600,000 free throws, and I re-ran the same computations and confirmed you can see a
 very clear pattern that they're just better at their second free throw. That's interesting.
 That kind of analysis is so awesome because I think with tennis, they have a fault. When you
 serve, they have analysis of are you most likely to miss the second serve if you missed first,
 obviously. I think that's the case. That integrates... That's so cool when psychology
 is converted into metrics in that way. And in sports, it's especially cool because it's such
 a constrained system that you can really study human psychology because it's repeated, it's
 constrained. So many things are controlled, which is something you rarely have in the wild
 psychological experiments. So it's cool. Plus everyone loves it. Sports is really cool to
 analyze. People actually care about the results. Yeah. I still think... Well, I will definitely
 publish this work on Messi versus Ronaldo and- I'd love to read it.
 Objective, fully objective. I'd love to peer review.
 Yeah, this is very true. This is not past peer review. Let me ask an advice question
 to the young folks. You've explored a lot of fascinating ideas in your life. You built a
 startup, worked on physics, worked in computer science. What advice would you give to young
 people today in high school, maybe early college about life, about career, about science and
 mathematics? I remember reading that Pongkario was once asked by a young man,
 a French journal about his advice for young people and what was his teaching philosophy.
 And he said that one of the most important things that parents should teach their kids is how to be
 enthusiastic in regards to the mysteries of the world. And that he said striking that balance was
 actually one of the most important things in education. You want to have your kids be
 enthusiastic about the mysteries of the world, but you also don't want to traumatize them if you
 really force them into something. And I think especially if you're young, I think you should
 be curious and I think you should explore that curiosity to the fullest, to the point where you
 even become almost as an expert on that topic. And you might start with something that is small.
 You might start with you're interested in numbers and how to factor numbers into primes. And then
 all of a sudden you go and you're lost in number theory and you discover cryptography and then all
 of a sudden you're buying Bitcoin. And I think you should do this. You should really try to
 fulfill this curiosity and you should live in a society that allows you to fulfill this curiosity,
 which is also important. And I think you should do this not to get to some sort of status or fame
 or money, but I think this is the way, this iterative process, I think this is the way to
 find happiness. And I think this also allows you to find the meaning for your life. I think it's
 all about being curious and being able to fulfill that curiosity and that path to fulfilling your
 curiosity. Yeah, the start small and let the fire build is kind of an interesting way to think about
 it. And you never know where you're going to end up. For Maaz, it's just a really good example.
 We started by doing this as an internal thing that we did in the company and then we started
 putting out there and now a lot of people follow it and know about it. And you still don't know
 where for Maaz's library is going to end up actually. True, exactly. So yeah, I think that
 would be my piece of advice with very limited experience, of course. Yeah, I agree. I mean,
 is there something in particular, Joao, from the computer science versus physics perspective?
 Do you regret not doing physics? Do you regret not doing computer science? Which one is the
 wiser, the better human being? This is Messi versus Ronaldo. Those are very, I don't know if you would
 agree, but they're kind of different disciplines. True. Yeah, very much so. I actually, I was, I
 had that question in my mind. I took physics classes as an undergrad or like besides what I
 had to take. And it's definitely something that I considered at some point. And that I do feel like
 later in life, that might be something that, I'm not sure if regret is the right word, but it's
 kind of something that I can imagine in an alternative universe, what would have happened
 if I had gone into physics. I try to think that like, well, it depends on what your
 path ends up being, but that it's not super important, right? Like exactly what you decide
 to major on. Like I think there's, I think Tim Urban, like the blogger had a good visualization
 of this where it's like, he has a picture where you have all sorts of paths that you could pursue
 in your life. And then maybe you're in the middle of it. And so there's maybe some paths that are
 not accessible to you, but like the tree that is still in front of you gives you a lot of optionality.
 And so there's two lessons to learn from that. Like we have a huge number of options now,
 and probably you're just one to reflect, like to try to derive wisdom from the one little path
 you've taken so far may be flawed because there's all these other paths you could have taken.
 So it's like, so one, it's inspiring that you can take any path now. And two, it's like the
 path you've taken so far is just one of many possible ones, but it does seem that like
 physics and computer science both open a lot of doors and a lot of different doors.
 True.
 It's very interesting.
 It is. Like in this case, like, and especially in our case, because I could see the difference
 I studied, I went to college in Europe and Joao went to college here in the US. So I could see
 the difference in like in the European system is more rigid in the sense that when you decide to
 study physics, you don't have a lot, especially in the early years, you don't have a lot of,
 you can't choose to take like a class from like computer science course or something like that.
 You don't have a lot of freedom to explore in that sense in university, as opposed to here
 in the US where you have more freedom. And I think that's important. I think that's what
 constitutes a good kind of educational system is one that gravitates towards the interests of
 a student as you progress. But I think in order for you to do that, you need to explore different
 areas. And I felt like if I had a chance to take, say more computer science class when I was in
 college, I would have probably have taken those classes. But yeah, but I ended up like focusing
 maybe too much in physics. And I think here at least my perception is that you can explore more
 fields. But there is a kind of it's funny, but physics can be difficult. So I don't see too many
 computer science people than exploring into physics. It's like the one, not the one, but one of the
 beneficial things of physics, it feels like it was it Rutherford that said like, basically that
 physics is the hard thing and everything is easy. So like, there's a certain sense once you figured
 out some basic like physics, that it's not that you need the tools of physics to understand
 the other disciplines, it's that you're empowered by having done difficult shit. I mean, the ultimate,
 I think, is probably mathematics there. Yeah, true. So maybe just doing difficult things,
 and proving to yourself that you can do difficult things, whatever those are.
 That's net positive, I believe that positive. Yeah. And I think like, I before I started
 a company, I had like, I worked in the financial sector for a bit. And like, I think having a
 physics background, I was I felt I was not afraid of like learning like finance things. And I think
 like when you come from those backgrounds, you are generally not afraid of stepping into other
 fields and learning about those because yeah, I feel they've learned a lot of difficult things.
 And yeah, that's an added benefit, I believe. This was an incredible conversation, Luis, Joao.
 We started with who we start with Feynman ended up with Messi and Ronaldo. So this is like the
 perfect conversation is really an honor that you guys would waste all this time with me today.
 It was really fun. Thanks for talking. Thank you so much for having us. Yeah, thank you so much.
 Thanks for listening to this conversation with Luis and Joao Batalla. And thank you to Skiff,
 Simply Safe, Indeed, NetSuite, and 4Sigmatic. Check them out in the description to support
 this podcast. And now let me leave you with some words from Richard Feynman.
 Nobody ever figures out what life is all about. And it doesn't matter. Explore the world.
 Nearly everything is really interesting if you go into it deeply enough.
 Thank you for listening. I hope to see you next time.