BERTology Meets Biology: Interpreting Attention in Protein Language Models (Paper Explained)
Proteins are the workhorses of almost all cellular functions and a core component of life. But despite their versatility, all proteins are built as sequences of the same 20 amino acids. These sequences can be analyzed with tools from NLP. This paper investigates the attention mechanism of a BERT model that has been trained on protein sequence data and discovers that the language model has implicitly learned non-trivial higher-order biological properties of proteins.

OUTLINE:
0:00 - Intro & Overview
1:40 - From DNA to Proteins
5:20 - BERT for Amino Acid Sequences
8:50 - The Structure of Proteins
12:40 - Investigating Biological Properties by Inspecting BERT
17:45 - Amino Acid Substitution
24:55 - Contact Maps
30:15 - Binding Sites
33:45 - Linear Probes
35:25 - Conclusion & Comments

Paper: https://arxiv.org/abs/2006.15222
Code: https://github.com/salesforce/provis

My Video on BERT: https://youtu.be/-9evrZnBorM
My Video on Attention: https://youtu.be/iDulhoQ2pro

Abstract:
Transformer architectures have proven to learn useful representations for protein classification and generation tasks. However, these representations present challenges in interpretability. Through the lens of attention, we analyze the inner workings of the Transformer and explore how the model discerns structural and functional properties of proteins. We show that attention (1) captures the folding structure of proteins, connecting amino acids that are far apart in the underlying sequence, but spatially close in the three-dimensional structure, (2) targets binding sites, a key functional component of proteins, and (3) focuses on progressively more complex biophysical properties with increasing layer depth. We also present a three-dimensional visualization of the interaction between attention and protein structure. Our findings align with known biological processes and provide a tool to aid discovery in protein engineering and synthetic biology. The code for visualization and analysis is available at this https URL.

Authors: Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher, Nazneen Fatema Rajani

Links:
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
 Hi there. Today we'll look at Bertology meets Biology, Interpreting Attention in Protein Language Models by Jesse Wigg, Ali Madani, Lav R. Varshney, Kaiming Hsiang, Richard Zucker and Nazneen Fatima Rajani. This paper is a investigative paper into models that are trained on biological data, specifically into BERT models, actually into a one specific BERT model that is trained on protein sequences. Now it is trained to simply perform language modeling on these protein sequences, but out of this language model you can then inspect this BERT model and read important biological data of these proteins, higher order data from the attention heads of the BERT model, which is pretty interesting. Basically means that the information of these higher order functions is at some point encoded in the structure of the language of the protein sequence. So we're going to go through what this means and how this comes about and what they did in order to investigate. I think this is a pretty cool investigative work and probably very promising for future research. Yeah, as always if you like content like this, consider sharing it out and leaving a like. Also tell me what you think in the comments. So biology. Really quick for people who maybe have never heard this. In your every cell you have this thing called DNA, which basically is a an encoding of all of your biological functions. Now usually biological functions are realized through proteins. So DNA is basically a building plan for all of your proteins. This happens in the following two steps. First there is this transcription step where RNA is built. This is basically a copy of your DNA but is only single strand as you can see right here. And then there is a translation step that finally translates the RNA into the protein. What will end up is just a sequence of these beads right here. Now these beads are what are called amino acids. So a protein is simply a chain of these amino acids. There are 20 different amino acids and the order of these amino acids in the chain makes the function of the protein. Now specifically we know about these proteins that it seems to be very important how their three-dimensional shape is. So a lot of these different amino acids have different chemical properties. Some are sort of I think negatively charged, some are neutral, some are acids and so on. So they have very different chemical properties. So once you build this protein and you kind of release it into the cell it will curl up into a three-dimensional structure. So this one might be you know doing something like this and sort of form a circle or something like this. Just because these proteins here they kind of attract each other maybe electrically and thus the protein forms a circle and the function of the protein is very much related to its shape. So if it is a circle it could maybe you know trap something else in here. So you really have to think of these things like kind of tools. There are proteins that cut other proteins and they are really shaped sort of like a scissor that exactly fits these other proteins such that you can effectively cut them. So sometimes you can substitute an amino acid for a different amino acid like this here. If it doesn't change the shape very often you're fine. The protein function isn't changed. But if you change a different amino acid that is sort of vital to the shape and the shape changes then your protein very often loses function. So mutations in DNA sometimes lead to mutations in protein, not always because there is some redundancy in this translation step from RNA. But if they do lead to a different amino acid it doesn't actually mean that the function changes. So there is sort of value in analyzing the sequence of the structure of proteins rather than the structure of DNA. Of course it's also important to analyze the structure of DNA but it is also it is equally important to analyze the structure of proteins because they not all the information is in the sequence, not all the obvious information is in the sequence. So what does this paper do? This paper goes and takes a model that has been trained on protein data. So if you look at this protein it is simply a sequence of amino acids and these amino acids they all have names. I think I have a table somewhere here. Yes, so these are the different amino acids that exist and you can see a protein is simply a sequence of these names. So usually they're abbreviated by like a three letter abbreviation or just a one letter abbreviation. So a protein might be A, V, M, M, V, A, G and so on. And this is just a string of text, right? So what I can do is I can train a language model on this. A language model is simply a model that takes a piece of text and tells you what's the next piece of text. So what's the next letter, what's the next word, in this case what's the next amino acid. And we can use tools from NLP for that. Specifically we can train a BERT model. Now BERT works a bit differently than a standard language model. BERT does what is called masked language modeling. So you would take this string, you would feed it into a BERT model right here. And I've made an entire video on BERT if you want to check that out. And what you'll do by inputting that you'll mask out some of the tokens. So you'll maybe mask out this one, mask out this one and then you ask the model to reconstruct those. You say that here's an M and here is an A without seeing them. So the model somehow has to learn from the surrounding amino acids what this amino acid could be, right? So it has to reconstruct this sequence. So the hope here is in natural language is that BERT somehow learns something about language itself. By being able to reconstruct these things it has learned something about language, about which words appear together and when. It might even learn very long distance relationships between words just because it has to predict those. And the idea carries over to biology. So we might hope that a BERT trained on an amino acid sequence will learn something about the language of proteins, about the amino acid sequence. And our goal here is to ask can we somehow infer the 3D shape of a protein, which is the important part, from its sequence right here. So given its sequence, can we infer the 3D shape? Now as I understand it usually this has to be done in like a simulation. So you would you would build this in a simulator and then you do like some sort of a molecule simulation to see how this ends up in a 3D shape. You could train a model to just predict the 3D shape but in this case we're just interested what does the BERT model learn about the 3D shape while only ever having been trained on predicting the next or predicting the sequence of amino acids. So it's never been trained to look at the 3D shape. And that's our goal here. So specifically we'll look at two different things. So here you can see examples of proteins and their high level structure. So in these proteins what you call the primary structure is this sequence of amino acid. This is simply which amino acids are in which order. There is a thing called the secondary structures and we often observe that spans of these amino acids like substrings form these what are called these helixes as you can see here or these sheets. I don't know how they're strands in English. We call them sheets or I think these are the alpha helixes and these are the beta sheets and there is also a turns. I think this here might be a turn. So there are these kind of secondary structures and then the tertiary structure is how these this is still one this is one protein. This is one unbroken chain of amino acid and you can see this here kind of forms this double ring which would be its tertiary structure. Very important for predicting the tertiary structure is to predict when two amino acids are close to each other. So if we have a chain right here and the chain as we saw before kind of turns and bends on itself then these two amino acids here are very close in close contact and to predict which amino acids are in close contact to each other helps you determine the the tertiary structure. So that's a consequence of it. So we wonder does BERT know intrinsically which of these amino acids are going to end up being in contact with each other without ever having been trained to do it. The second thing we're interested in are binding sites. So here, you might not be able to see, but we made this example before where this sort of forms a loop and then I say can trap something here right like another molecule and this is what what we would call a binding site. A binding site is a one amino acid that maybe through the structure of the surrounding amino acid as well but also through its properties and how it is exposed in 3D shape acts as sort of a receptor for other molecules. It binds to other things. It can, so think of your hemoglobin that that traps the the oxygen in your blood or something like this. It is where a chemical reaction or a reaction with something else will happen. That's a binding site and we are interested does BERT, the BERT that is only trained on a language modeling objective know which ones are the binding sites because you know that would be very interesting and not something BERT was trained on. By the way, I particularly liked Richard Soccer's tweet on this. I think he tweeted out, oh BERT trained only on language model can predict binding sites and biological properties and formulated it like it was you know like GPT-3 was formulated like if we train on Wikipedia our model can do math. I thought it was kind of a satire headline. If we train on Wikipedia our model can predict biology and also it can tie your shoes and cook your dinner. Yeah but so it's trained on language modeling on biological data and now that makes sense. So they're going to look at two different things or actually more than two different things but they formulate this in an abstract way right here. So what they'll look at is the so-called properties. The property F and this property F can be for example that a amino acid is a binding site. The property F can also be that two amino acids are in contact with each other. So F always takes I and J. If in the case for example where this is the contact property then it simply is the indicator function for when I and J are in contact. And if it is a just a binding site then I think we are looking at J. At the token level property we define to be an indicator that returns 1 if the property is present in token J. Okay so whenever J is a binding site then that holds. So what we're looking at are these attention heads in BERT. If you don't know BERT has an attention mechanism which basically means from layer to layer each token can attend to all other tokens. So here the amino acid sequence I've drawn it twice and the next layer representation of this amino acid will be able to gather information from all of the other amino acid through an attention mechanism through a dynamic routing algorithm. I've made a video on attention is all you need if you want to find out more how this works. Now what we're interested in is the strength of these connections. So the hypothesis is if molecule here is 1, 2, 3, 4, 5, and 6. If molecule 1 and 3 are contact sites then maybe we will find a layer where this connection between 1 and 3 is very strong. That would indicate that there is a connection site or that would indicate that BERT has learned something about the connection sites. Okay if we find this repeatedly so if we look at many many proteins and whenever we know that there is a contact between two things and then we observe that the corresponding attention is very high then we can be pretty sure that BERT has learned something about contact between amino acids. Okay the same goes for binding sites. So if 4 here is a binding site and then all the connections all the attention that the higher layer gets from 4 so all the information routed away from 4 is very strong that means all these other tokens are paying special attention to the token number 4 to this amino acid. And if we find that there is a big correlation with this being a binding site then we can reasonably conclude that BERT has learned something about binding sites. Alright so we're going to do a correlative analysis for proteins where we know the binding sites where we know the contacts right we can analyze them we can run simulations therefore we can know them. So we're going to look at this quantity right here which is simply a normalized quantity. So we're going to look at the attention in a given attention head so as you know BERT has many layers with many attention heads and we're going to look at whether or not this property is active and just normalize it by the total attention in that head so that we get some kind of a percentage number. That's the first task we're basically going to look at how does the attention correlate with these properties. And the second task we're going to do is this probing task. So a probing task is like a linear probe in like a classifier. So what we're going to do is we're going to take a layer right here and even though it's an intermediate layer we're simply going to run it through a linear classifier and then decide is this a binding site for example or not. Is a given amino acid a binding site or not? Is a given pair a contact or not? So this is kind of a linear probe but this sort of takes a backseat in this paper. The analysis is really on the attention heads and what the attention heads learn. And that's already it. They take a pre-trained BERT model so there are these BERT models that are already trained on these protein databases and first they look simply can we find attention heads that correlate with a given amino acid. So here you see the attention to the amino acid. This is proline I believe and this is phenylalanine. Is that the same in English? Yes, phenylalanine and proline right here. So you can see that the plots here are there's almost no attention pretty much throughout the network that pays special attention to the amino acid proline except this head right here seems to have if you look at the scale over like a 70% of attention always goes to proline in this particular head. So this is layer 1 head number 11 focuses 78% of its attention on proline. Now this is not that special if you think about it because in language models as well in natural language models you might want to think that you have some mechanism in your neural network that's especially specialized on like a very particular word in the language because that might just be a often occurring very particular word. For example in English maybe the is very important or the word what these are like very indicative very often occurring words so it is reasonable to expect to find an attention head that pays a lot of attention to these things especially here where our vocabulary size is 20 instead of like 30,000 in natural language. And the same goes for this phenylalanine where you can see that in the last layer and in the first layer you have attention and also in the proline you have in the last layer. So why does this make sense? Because what we would expect from like single tokens these are not interactions yet these are not biological functions yet. So we know that in the lower layers of a neural network we have these kind of basic features basic feature extractors and here these basic feature extractors appear to be simply paying attention to one specific token in the vocabulary a lot. So they kind of these heads sort of specialize for single for single amino acids. And the same in the last layer so in the very last layer the task of the very last layer is to prepare for the classification task. So if you remember the BERT model you have layer layer layer layer and at the end you'll have to predict which ones are masked down here. So at the end you have to predict single amino acids again so if there's a proline masked here you'll have to predict the proline. So it also makes sense that the last layers would very much specialize to single tokens. So this does make sense. Now our question is going to be do we find the biological function where would you expect them? We would expect the let's say the tertiary, sorry the secondary structures which are sort of one level higher than the primary structures we would expect to find them maybe here and then we would expect to find the tertiary structures maybe somewhere here okay because these are most highest level. And then it goes it goes back again or maybe it's like we find the tertiary structures rather here and here again and then in the middle we'll find the the most high level the tertiary structures or yeah blue secondary. This drawing is getting too too too weird but there could be multiple scenarios but that could fit here but until now it sort of makes sense. So they do an additional investigation where as I told you sometimes you can substitute an amino acid and nothing really happens right and in fact that this probably happens in you right now you probably might have some mutation that changed some amino acid and you don't even realize because it's just it's fine. No notice. So the biologists can build these matrices of how much you can substitute proteins with each other. So here you see this BLOSUM62 substitution scores which are very I guess very high if you can substitute two protein, two amino acids with each other and the effect is negligible. And it's very low if it's the the other way around. Now this is interesting so far but you compare this to this matrix right here. This is the attention similarity. So what we'll do is for each two amino acids we take those two attention things, those two attention matrices and we'll calculate the correlation between the attention matrices. And our hypothesis is that the more correlated the attention patterns are between the two amino acids the more likely we are to substitute them because as a direct result of our language model our language model is it's reconstructing right these things. So our language model is going to treat if in natural language is like a a synonym right is our language model is going to treat synonyms very similar to each other because they're synonyms they can be exchanged. So a good language model should learn that they are almost the same and therefore the attention pattern is going to be almost the same. So a high correlation we hypothesize is a means that the function of the amino acid is similar and therefore we can substitute it easily. So this here is the matrix of the correlations between each two attention patterns of these amino acid. And if you compare the two right here they look extremely similar. Just have a look for a little while and you'll see that the patterns they do not match perfectly but they are very very similar. The dark spots are in the same places the light spots are in the same places. So this already makes a good case that the language model here has learned something about biology. Now what we want to do is we want to investigate higher order higher order functions. So here we're interested in these contact maps right. So how likely is it that two amino acids are in contact and we'll look at it through the lens of attention as we did before. So here is the percentage of each head of each head's attention that is aligned with contact maps averaged over a data set suggesting that head 12 4 is uniquely specialized for contact prediction. So look at this this this head here is just spiking. So remember before we said our analysis is whenever whenever we're basically measuring the correlation of two things being in contact because we know it from our simulator or from our data set the correlation of that with an attention connection being particularly strong. And we find it in this attention head right here. So this layer 12 head number four will always peek out whenever two things are in contact. Now you can see that it's it's not like always it's like 25 percent of its attention but significantly more than anything else right here. In fact if you group the things by this attention you can build the following plot. So you can see right here probability two amino acids are in contact as a function of attention between the amino acids in head 12 4 showing attention approximates perfectly calibrated estimator which would be the green line. So here we simply for each for each pairs two amino acids for each pair of amino acids we plot we make a histogram right here of what they're sorry not a histogram we plot the probability if they have the attention weight point nine we plot how likely is it that they are in contact. So this is this if we just look at the data and we simply take this attention weight as a measure as a predictor of being in contact we get the blue curve and the green curve would be if we could perfectly predict from this attention head what the probability of contact would be and you can see that the fit is fairly good you can't predict with super high accuracy but the fit is fairly good and you can see the general trend that as the attention in this head rises the probability of the two amino acids being in contact with each other also rises. So we can sort of confidently say that BERT has learned something about a higher level biological structure just from the language modeling objective. How can we interpret this? This must somehow mean that it is possible it is vital for reconstructing the sequence from its surrounding so if we delete this right here if these two are in contact in the 3D structure that makes probably probably means that this thing right here is a very good predictor of what was here. If we mask this out and we're asked to reconstruct which amino acid was there then it probably helps to look at its neighbors right it probably always helps to look at one's neighbors especially also in natural language but if these two are in contact they have very special connection to each other it's very you can basically read out from this one which one this was. This is sort of like if you have a sentence and you say I don't know I can't come up with one right now but if it's like and then there is a name like Mark and then and then there is him right and you would expect if I drop out okay let's do it the other way around if I drop out him then from the text right here you can probably determine that it is some sort of pronoun but then you go back and you see it's Mark okay so it's not like it's not like it or some or she it's probably he or him this is sort of the analogous structure right here in biology. The second thing we're looking at is these binding sites. Now these are single properties of different amino acids and we're simply looking at all the incoming or sorry all the other tokens that focuses their attention. Why is this important? Because these binding sites are central to the structure of the or to the function of the protein right if this here is a binding site then that's a very central important point of the protein. So a lot of these other things are going to be determined by what this binding site is. This binding site needs to have a very particular function and therefore probably needs to be a very particular amino acid and the other things here are sort of supporting this binding site because they form the 3d structure around it and so on. So you would expect a lot of attention to be put on this binding site and what do we find? We find that it's a bit more murky than before so you can see that the attention is kind of spread out. Percentage of each head's attention that focuses on binding sites especially in the deeper layers binding sites are targeted at much higher frequency than would occur by chance. Head 71 has the highest percentage with 34%. So also here you can see that it is spread out but this is because multiple heads are now focusing on these binding sites because probably binding sites come in different variations so you'll have lots of heads specializing on attending to binding sites and they say it is much higher frequency than would occur by chance and you can see here this head is the highest with 34% of its attention focused on binding sites. You can also see the general trend of the attention being rather in the later layers which we would expect from a tertiary structure. Now yeah it would be interesting here. Here you also see that actually most of the things are in the in the last layer which points to points to rather maybe lower lower level information because we reasoned before about the last layer or I was just wrong but also in a general trend you can see that the attention is rather shifted towards the later layers because this is sort of a higher order function. Okay if you look at the same calibration experiment you can see that the picture is not as clear. There is the general trend at the beginning but then it sort of flattens out so you can sort of differentiate the very probably not a binding site from the somewhat probably a binding site but it's not a perfectly calibrated classifier and that might just be because there are many things specializing in different types of binding sites so you can't just go to this one head so this is just for this one head. You can't just go to that one and expect that to classify all the binding sites because you might want to be you might want to combine all of the high ranking ones here to form a classifier. The last experiment they do is these linear probes which where they just go and they just build classifiers from different parts of the network and you can see right here that what is predicted and how well they work. So each bar here is going to be the difference of performance so this is differential performance of diagnostic classifier by layer sorted by task order in figure 8. Each plot shows the change in performance between the given layer and the previous layer. So a bar up shows it's performing better than the previous layer bar down shows it's performing worse than the previous layer. So you see right here that these are the secondary structures right here and you can see that there is a lot of performance in the earlier layers right here and sort of not that high performance in the later layers whereas for the tertiary structures the binding site and the contact you can see that there is a bit of performance in places but it sort of tends to be more towards the middle certainly more at towards the middle of the end of the network than the secondary structures which sort of makes sense with our hypothesis. You can also see this here where they show the percent of attention focused as a function of layer and the red is the center of mass and you can see that as the secondary structures this their center of mass is at a lower layer in general than the tertiary functions. All of this is not perfect of course but it's still an open question I guess whether or not it's not perfect because we haven't built a strong enough language model yet. Do I want to say GPT-4 is now for biology and not for language or is it because there is really you need you really can't very well predict these things just from a language model. I mean you should technically all the information is there but maybe the language model objective as such isn't able to capture that information. So yeah this was the paper it's pretty simple they have in the appendix they have a lot of these additional experiments or full experiments I believe for all the amino acids and so on and I invite you to check that out. In general I like this kind of work because it's very applied it's and it can you know tell us something about the nature of both these language models and the biological things that we that we care about in biology. Okay I'm just talking crap right now thanks for being here I hope you enjoyed it and bye bye.
[{"start": 0.0, "end": 5.92, "text": " Hi there. Today we'll look at Bertology meets Biology, Interpreting Attention in"}, {"start": 5.92, "end": 12.040000000000001, "text": " Protein Language Models by Jesse Wigg, Ali Madani, Lav R. Varshney, Kaiming Hsiang,"}, {"start": 12.040000000000001, "end": 20.04, "text": " Richard Zucker and Nazneen Fatima Rajani. This paper is a investigative paper into"}, {"start": 20.04, "end": 25.64, "text": " models that are trained on biological data, specifically into BERT models,"}, {"start": 25.64, "end": 32.76, "text": " actually into a one specific BERT model that is trained on protein sequences. Now"}, {"start": 32.76, "end": 39.480000000000004, "text": " it is trained to simply perform language modeling on these protein sequences, but"}, {"start": 39.480000000000004, "end": 45.8, "text": " out of this language model you can then inspect this BERT model and read"}, {"start": 45.8, "end": 51.8, "text": " important biological data of these proteins, higher order data from the"}, {"start": 51.8, "end": 56.16, "text": " attention heads of the BERT model, which is pretty interesting. Basically means"}, {"start": 56.16, "end": 62.0, "text": " that the information of these higher order functions is at some point encoded"}, {"start": 62.0, "end": 68.24, "text": " in the structure of the language of the protein sequence. So we're going to go"}, {"start": 68.24, "end": 74.2, "text": " through what this means and how this comes about and what they did in order"}, {"start": 74.2, "end": 79.56, "text": " to investigate. I think this is a pretty cool investigative work and probably"}, {"start": 79.56, "end": 87.72, "text": " very promising for future research. Yeah, as always if you like content like"}, {"start": 87.72, "end": 93.44, "text": " this, consider sharing it out and leaving a like. Also tell me what you think in"}, {"start": 93.44, "end": 101.96000000000001, "text": " the comments. So biology. Really quick for people who maybe have never heard"}, {"start": 101.96000000000001, "end": 109.04, "text": " this. In your every cell you have this thing called DNA, which basically is a"}, {"start": 109.04, "end": 114.88000000000001, "text": " an encoding of all of your biological functions. Now usually biological"}, {"start": 114.88000000000001, "end": 120.84, "text": " functions are realized through proteins. So DNA is basically a building plan for"}, {"start": 120.84, "end": 126.16000000000001, "text": " all of your proteins. This happens in the following two steps. First there is this"}, {"start": 126.16000000000001, "end": 132.8, "text": " transcription step where RNA is built. This is basically a copy of your DNA but"}, {"start": 132.8, "end": 137.04000000000002, "text": " is only single strand as you can see right here. And then there is a"}, {"start": 137.04, "end": 143.79999999999998, "text": " translation step that finally translates the RNA into the protein. What will end"}, {"start": 143.79999999999998, "end": 149.51999999999998, "text": " up is just a sequence of these beads right here. Now these beads are what are"}, {"start": 149.51999999999998, "end": 155.12, "text": " called amino acids. So a protein is simply a chain of these amino acids."}, {"start": 155.12, "end": 161.6, "text": " There are 20 different amino acids and the order of these amino acids in the"}, {"start": 161.6, "end": 166.95999999999998, "text": " chain makes the function of the protein. Now specifically we know about these"}, {"start": 166.96, "end": 172.72, "text": " proteins that it seems to be very important how their three-dimensional"}, {"start": 172.72, "end": 177.52, "text": " shape is. So a lot of these different amino acids have different chemical"}, {"start": 177.52, "end": 184.68, "text": " properties. Some are sort of I think negatively charged, some are neutral, some"}, {"start": 184.68, "end": 189.16, "text": " are acids and so on. So they have very different chemical properties. So once"}, {"start": 189.16, "end": 194.8, "text": " you build this protein and you kind of release it into the cell it will curl up"}, {"start": 194.8, "end": 198.8, "text": " into a three-dimensional structure. So this one might be you know doing"}, {"start": 198.8, "end": 205.56, "text": " something like this and sort of form a circle or something like this."}, {"start": 205.56, "end": 210.88000000000002, "text": " Just because these proteins here they kind of attract each other maybe"}, {"start": 210.88000000000002, "end": 216.60000000000002, "text": " electrically and thus the protein forms a circle and the function of the protein"}, {"start": 216.60000000000002, "end": 221.92000000000002, "text": " is very much related to its shape. So if it is a circle it could maybe you know"}, {"start": 221.92, "end": 226.0, "text": " trap something else in here. So you really have to think of these things"}, {"start": 226.0, "end": 231.07999999999998, "text": " like kind of tools. There are proteins that cut other proteins and they are"}, {"start": 231.07999999999998, "end": 237.95999999999998, "text": " really shaped sort of like a scissor that exactly fits these other proteins"}, {"start": 237.95999999999998, "end": 245.2, "text": " such that you can effectively cut them. So sometimes you can substitute an"}, {"start": 245.2, "end": 250.64, "text": " amino acid for a different amino acid like this here. If it doesn't change the"}, {"start": 250.64, "end": 257.52, "text": " shape very often you're fine. The protein function isn't changed."}, {"start": 257.52, "end": 262.88, "text": " But if you change a different amino acid that is sort of vital to the shape and"}, {"start": 262.88, "end": 270.56, "text": " the shape changes then your protein very often loses function. So mutations in DNA"}, {"start": 270.56, "end": 278.03999999999996, "text": " sometimes lead to mutations in protein, not always because there is some"}, {"start": 278.04, "end": 282.72, "text": " redundancy in this translation step from RNA. But if they do lead to a"}, {"start": 282.72, "end": 288.12, "text": " different amino acid it doesn't actually mean that the function changes. So there"}, {"start": 288.12, "end": 294.84000000000003, "text": " is sort of value in analyzing the sequence of the structure of proteins"}, {"start": 294.84000000000003, "end": 298.64000000000004, "text": " rather than the structure of DNA. Of course it's also important to analyze"}, {"start": 298.64000000000004, "end": 304.56, "text": " the structure of DNA but it is also it is equally important to analyze the"}, {"start": 304.56, "end": 311.44, "text": " structure of proteins because they not all the information is in the sequence,"}, {"start": 311.44, "end": 318.24, "text": " not all the obvious information is in the sequence. So what does this paper do?"}, {"start": 318.24, "end": 324.76, "text": " This paper goes and takes a model that has been trained on protein data. So if"}, {"start": 324.76, "end": 329.84000000000003, "text": " you look at this protein it is simply a sequence of amino acids and these amino"}, {"start": 329.84, "end": 335.76, "text": " acids they all have names. I think I have a table somewhere here. Yes, so these are"}, {"start": 335.76, "end": 343.32, "text": " the different amino acids that exist and you can see a protein is simply a"}, {"start": 343.32, "end": 348.67999999999995, "text": " sequence of these names. So usually they're abbreviated by like a three"}, {"start": 348.67999999999995, "end": 354.23999999999995, "text": " letter abbreviation or just a one letter abbreviation. So a protein might be A, V,"}, {"start": 354.24, "end": 364.0, "text": " M, M, V, A, G and so on. And this is just a string of text, right? So what I can do"}, {"start": 364.0, "end": 369.2, "text": " is I can train a language model on this. A language model is simply a model that"}, {"start": 369.2, "end": 375.08, "text": " takes a piece of text and tells you what's the next piece of text. So what's"}, {"start": 375.08, "end": 379.92, "text": " the next letter, what's the next word, in this case what's the next amino acid. And"}, {"start": 379.92, "end": 386.08000000000004, "text": " we can use tools from NLP for that. Specifically we can train a BERT model."}, {"start": 386.08000000000004, "end": 390.84000000000003, "text": " Now BERT works a bit differently than a standard language model. BERT does"}, {"start": 390.84000000000003, "end": 395.04, "text": " what is called masked language modeling. So you would take this string, you would"}, {"start": 395.04, "end": 400.72, "text": " feed it into a BERT model right here. And I've made an entire video on BERT if"}, {"start": 400.72, "end": 405.96000000000004, "text": " you want to check that out. And what you'll do by inputting that you'll mask"}, {"start": 405.96, "end": 410.2, "text": " out some of the tokens. So you'll maybe mask out this one, mask out this one and"}, {"start": 410.2, "end": 415.71999999999997, "text": " then you ask the model to reconstruct those. You say that here's an M and here"}, {"start": 415.71999999999997, "end": 419.96, "text": " is an A without seeing them. So the model somehow has to learn from the"}, {"start": 419.96, "end": 427.67999999999995, "text": " surrounding amino acids what this amino acid could be, right? So it has to"}, {"start": 427.67999999999995, "end": 434.44, "text": " reconstruct this sequence. So the hope here is in natural language is that BERT"}, {"start": 434.44, "end": 440.88, "text": " somehow learns something about language itself. By being able to reconstruct"}, {"start": 440.88, "end": 445.0, "text": " these things it has learned something about language, about which words appear"}, {"start": 445.0, "end": 449.88, "text": " together and when. It might even learn very long distance relationships between"}, {"start": 449.88, "end": 457.0, "text": " words just because it has to predict those. And the idea carries over to"}, {"start": 457.0, "end": 465.0, "text": " biology. So we might hope that a BERT trained on an amino acid sequence will"}, {"start": 465.0, "end": 471.12, "text": " learn something about the language of proteins, about the amino"}, {"start": 471.12, "end": 479.04, "text": " acid sequence. And our goal here is to ask can we somehow infer the 3D"}, {"start": 479.04, "end": 484.32, "text": " shape of a protein, which is the important part, from its sequence right"}, {"start": 484.32, "end": 491.44, "text": " here. So given its sequence, can we infer the 3D shape? Now as I understand it"}, {"start": 491.44, "end": 496.15999999999997, "text": " usually this has to be done in like a simulation. So you would you would build"}, {"start": 496.15999999999997, "end": 502.44, "text": " this in a simulator and then you do like some sort of a molecule simulation to"}, {"start": 502.44, "end": 507.84, "text": " see how this ends up in a 3D shape. You could train a model to just predict the"}, {"start": 507.84, "end": 512.36, "text": " 3D shape but in this case we're just interested what does the BERT model learn"}, {"start": 512.36, "end": 518.76, "text": " about the 3D shape while only ever having been trained on predicting the"}, {"start": 518.76, "end": 524.24, "text": " next or predicting the sequence of amino acids. So it's never been trained to"}, {"start": 524.24, "end": 530.12, "text": " look at the 3D shape. And that's our goal here. So specifically we'll look at two"}, {"start": 530.12, "end": 534.5600000000001, "text": " different things. So here you can see examples of proteins and their high"}, {"start": 534.5600000000001, "end": 540.16, "text": " level structure. So in these proteins what you call the primary structure is"}, {"start": 540.16, "end": 546.28, "text": " this sequence of amino acid. This is simply which amino acids are in which"}, {"start": 546.28, "end": 551.68, "text": " order. There is a thing called the secondary structures and we often"}, {"start": 551.68, "end": 558.1999999999999, "text": " observe that spans of these amino acids like substrings form these what are"}, {"start": 558.1999999999999, "end": 565.1999999999999, "text": " called these helixes as you can see here or these sheets. I don't know how they're"}, {"start": 565.2, "end": 571.24, "text": " strands in English. We call them sheets or I think these are the alpha helixes"}, {"start": 571.24, "end": 576.48, "text": " and these are the beta sheets and there is also a turns. I think this here might"}, {"start": 576.48, "end": 582.1600000000001, "text": " be a turn. So there are these kind of secondary structures and then the"}, {"start": 582.1600000000001, "end": 587.76, "text": " tertiary structure is how these this is still one this is one protein. This is"}, {"start": 587.76, "end": 592.2, "text": " one unbroken chain of amino acid and you can see this here kind of forms this"}, {"start": 592.2, "end": 597.9200000000001, "text": " double ring which would be its tertiary structure. Very important for"}, {"start": 597.9200000000001, "end": 604.6400000000001, "text": " predicting the tertiary structure is to predict when two amino acids are close"}, {"start": 604.6400000000001, "end": 609.96, "text": " to each other. So if we have a chain right here and the chain as we saw"}, {"start": 609.96, "end": 615.6, "text": " before kind of turns and bends on itself then these two amino acids here are very"}, {"start": 615.6, "end": 622.8000000000001, "text": " close in close contact and to predict which amino acids are in close contact"}, {"start": 622.8000000000001, "end": 628.52, "text": " to each other helps you determine the the tertiary structure. So that's a"}, {"start": 628.52, "end": 636.5600000000001, "text": " consequence of it. So we wonder does BERT know intrinsically which of these"}, {"start": 636.5600000000001, "end": 640.76, "text": " amino acids are going to end up being in contact with each other without ever"}, {"start": 640.76, "end": 646.28, "text": " having been trained to do it. The second thing we're interested in are binding"}, {"start": 646.28, "end": 652.68, "text": " sites. So here, you might not be able to see, but we made this example before"}, {"start": 652.68, "end": 657.64, "text": " where this sort of forms a loop and then I say can trap something here right like"}, {"start": 657.64, "end": 663.24, "text": " another molecule and this is what what we would call a binding site. A binding"}, {"start": 663.24, "end": 671.52, "text": " site is a one amino acid that maybe through the structure of the surrounding"}, {"start": 671.52, "end": 676.8, "text": " amino acid as well but also through its properties and how it is exposed in 3D"}, {"start": 676.8, "end": 683.8, "text": " shape acts as sort of a receptor for other molecules. It binds to other things."}, {"start": 683.8, "end": 691.04, "text": " It can, so think of your hemoglobin that that traps the the oxygen in your blood"}, {"start": 691.04, "end": 698.8399999999999, "text": " or something like this. It is where a chemical reaction or a reaction with"}, {"start": 698.8399999999999, "end": 703.7199999999999, "text": " something else will happen. That's a binding site and we are interested does"}, {"start": 703.7199999999999, "end": 710.5999999999999, "text": " BERT, the BERT that is only trained on a language modeling objective know which"}, {"start": 710.5999999999999, "end": 716.16, "text": " ones are the binding sites because you know that would be very interesting and"}, {"start": 716.16, "end": 721.1999999999999, "text": " not something BERT was trained on. By the way, I particularly liked Richard"}, {"start": 721.1999999999999, "end": 727.24, "text": " Soccer's tweet on this. I think he tweeted out, oh BERT trained only on"}, {"start": 727.24, "end": 732.48, "text": " language model can predict binding sites and biological properties and formulated"}, {"start": 732.48, "end": 737.6, "text": " it like it was you know like GPT-3 was formulated like if we train on Wikipedia"}, {"start": 737.6, "end": 743.88, "text": " our model can do math. I thought it was kind of a satire headline. If we train on"}, {"start": 743.88, "end": 749.16, "text": " Wikipedia our model can predict biology and also it can tie your shoes and cook"}, {"start": 749.16, "end": 755.64, "text": " your dinner. Yeah but so it's trained on language modeling on biological data and"}, {"start": 755.64, "end": 762.64, "text": " now that makes sense. So they're going to look at two different things or actually"}, {"start": 762.64, "end": 767.96, "text": " more than two different things but they formulate this in an abstract way right"}, {"start": 767.96, "end": 775.52, "text": " here. So what they'll look at is the so-called properties. The property F and"}, {"start": 775.52, "end": 783.36, "text": " this property F can be for example that a amino acid is a binding site. The"}, {"start": 783.36, "end": 789.2800000000001, "text": " property F can also be that two amino acids are in contact with each other. So"}, {"start": 789.2800000000001, "end": 796.1600000000001, "text": " F always takes I and J. If in the case for example where this is the contact"}, {"start": 796.16, "end": 802.92, "text": " property then it simply is the indicator function for when I and J are in"}, {"start": 802.92, "end": 814.88, "text": " contact. And if it is a just a binding site then I think we are looking at J. At"}, {"start": 814.88, "end": 819.3199999999999, "text": " the token level property we define to be an indicator that returns 1 if the"}, {"start": 819.32, "end": 826.5200000000001, "text": " property is present in token J. Okay so whenever J is a binding site then that"}, {"start": 826.5200000000001, "end": 831.96, "text": " holds. So what we're looking at are these attention heads in BERT. If you don't"}, {"start": 831.96, "end": 836.7600000000001, "text": " know BERT has an attention mechanism which basically means from layer to"}, {"start": 836.7600000000001, "end": 843.08, "text": " layer each token can attend to all other tokens. So here the amino acid sequence"}, {"start": 843.08, "end": 848.84, "text": " I've drawn it twice and the next layer representation of this amino acid will"}, {"start": 848.84, "end": 853.5600000000001, "text": " be able to gather information from all of the other amino acid through an"}, {"start": 853.5600000000001, "end": 858.2, "text": " attention mechanism through a dynamic routing algorithm. I've made a video on"}, {"start": 858.2, "end": 863.8000000000001, "text": " attention is all you need if you want to find out more how this works. Now what"}, {"start": 863.8000000000001, "end": 870.9200000000001, "text": " we're interested in is the strength of these connections. So the hypothesis is"}, {"start": 870.92, "end": 881.04, "text": " if molecule here is 1, 2, 3, 4, 5, and 6. If molecule 1 and 3 are contact sites"}, {"start": 881.04, "end": 890.12, "text": " then maybe we will find a layer where this connection between 1 and 3 is very"}, {"start": 890.12, "end": 896.12, "text": " strong. That would indicate that there is a connection site or that would"}, {"start": 896.12, "end": 901.5600000000001, "text": " indicate that BERT has learned something about the connection sites. Okay if we"}, {"start": 901.5600000000001, "end": 906.88, "text": " find this repeatedly so if we look at many many proteins and whenever we know"}, {"start": 906.88, "end": 911.8, "text": " that there is a contact between two things and then we observe that the"}, {"start": 911.8, "end": 918.28, "text": " corresponding attention is very high then we can be pretty sure that BERT has"}, {"start": 918.28, "end": 924.6, "text": " learned something about contact between amino acids. Okay the same goes for"}, {"start": 924.6, "end": 932.44, "text": " binding sites. So if 4 here is a binding site and then all the"}, {"start": 932.44, "end": 938.36, "text": " connections all the attention that the higher layer gets from 4 so all the"}, {"start": 938.36, "end": 943.32, "text": " information routed away from 4 is very strong that means all these other tokens"}, {"start": 943.32, "end": 948.4, "text": " are paying special attention to the token number 4 to this amino acid. And"}, {"start": 948.4, "end": 955.0, "text": " if we find that there is a big correlation with this being a binding site then we"}, {"start": 955.0, "end": 960.24, "text": " can reasonably conclude that BERT has learned something about binding sites."}, {"start": 960.24, "end": 965.0799999999999, "text": " Alright so we're going to do a correlative analysis for proteins where"}, {"start": 965.0799999999999, "end": 970.36, "text": " we know the binding sites where we know the contacts right we can analyze them"}, {"start": 970.36, "end": 977.6, "text": " we can run simulations therefore we can know them. So we're going to look at this"}, {"start": 977.6, "end": 981.5600000000001, "text": " quantity right here which is simply a normalized quantity. So we're going to"}, {"start": 981.5600000000001, "end": 986.9200000000001, "text": " look at the attention in a given attention head so as you know BERT has"}, {"start": 986.9200000000001, "end": 992.76, "text": " many layers with many attention heads and we're going to look at whether or"}, {"start": 992.76, "end": 997.6800000000001, "text": " not this property is active and just normalize it by the total attention in"}, {"start": 997.6800000000001, "end": 1003.36, "text": " that head so that we get some kind of a percentage number. That's the first task"}, {"start": 1003.36, "end": 1007.36, "text": " we're basically going to look at how does the attention correlate with these"}, {"start": 1007.36, "end": 1013.52, "text": " properties. And the second task we're going to do is this probing task. So a"}, {"start": 1013.52, "end": 1019.36, "text": " probing task is like a linear probe in like a classifier. So what we're going to"}, {"start": 1019.36, "end": 1026.64, "text": " do is we're going to take a layer right here and even though it's an"}, {"start": 1026.64, "end": 1031.28, "text": " intermediate layer we're simply going to run it through a linear classifier and"}, {"start": 1031.28, "end": 1037.6399999999999, "text": " then decide is this a binding site for example or not. Is a given"}, {"start": 1037.6399999999999, "end": 1044.84, "text": " amino acid a binding site or not? Is a given pair a contact or not? So this is"}, {"start": 1044.84, "end": 1049.6, "text": " kind of a linear probe but this sort of takes a backseat in this paper. The"}, {"start": 1049.6, "end": 1053.6399999999999, "text": " analysis is really on the attention heads and what the attention heads learn."}, {"start": 1053.6399999999999, "end": 1059.8, "text": " And that's already it. They take a pre-trained BERT model so there are"}, {"start": 1059.8, "end": 1064.8, "text": " these BERT models that are already trained on these protein databases and"}, {"start": 1064.8, "end": 1072.44, "text": " first they look simply can we find attention heads that correlate with a"}, {"start": 1072.44, "end": 1079.6, "text": " given amino acid. So here you see the attention to the amino acid. This is"}, {"start": 1079.6, "end": 1085.76, "text": " proline I believe and this is phenylalanine. Is that the same in"}, {"start": 1085.76, "end": 1095.48, "text": " English? Yes, phenylalanine and proline right here. So you can see that"}, {"start": 1095.48, "end": 1104.68, "text": " the plots here are there's almost no attention pretty much throughout the"}, {"start": 1104.68, "end": 1111.2, "text": " network that pays special attention to the amino acid proline except this head"}, {"start": 1111.2, "end": 1117.16, "text": " right here seems to have if you look at the scale over like a 70% of attention"}, {"start": 1117.16, "end": 1122.3600000000001, "text": " always goes to proline in this particular head. So this is layer 1 head"}, {"start": 1122.3600000000001, "end": 1133.88, "text": " number 11 focuses 78% of its attention on proline. Now this is not that special"}, {"start": 1133.88, "end": 1137.72, "text": " if you think about it because in language models as well in natural"}, {"start": 1137.72, "end": 1142.48, "text": " language models you might want to think that you have some mechanism in your"}, {"start": 1142.48, "end": 1146.24, "text": " neural network that's especially specialized on like a very particular"}, {"start": 1146.24, "end": 1151.32, "text": " word in the language because that might just be a often occurring very"}, {"start": 1151.32, "end": 1158.28, "text": " particular word. For example in English maybe the is very important or the word"}, {"start": 1158.28, "end": 1164.84, "text": " what these are like very indicative very often occurring words so it is"}, {"start": 1164.84, "end": 1168.56, "text": " reasonable to expect to find an attention head that pays a lot of"}, {"start": 1168.56, "end": 1173.1999999999998, "text": " attention to these things especially here where our vocabulary size is 20"}, {"start": 1173.1999999999998, "end": 1179.9599999999998, "text": " instead of like 30,000 in natural language. And the same goes for this"}, {"start": 1179.9599999999998, "end": 1186.52, "text": " phenylalanine where you can see that in the last layer and in the"}, {"start": 1186.52, "end": 1189.9599999999998, "text": " first layer you have attention and also in the proline you have in the last"}, {"start": 1189.9599999999998, "end": 1194.3999999999999, "text": " layer. So why does this make sense? Because what we would expect from like"}, {"start": 1194.4, "end": 1198.68, "text": " single tokens these are not interactions yet these are not biological functions"}, {"start": 1198.68, "end": 1204.64, "text": " yet. So we know that in the lower layers of a neural network we have these kind"}, {"start": 1204.64, "end": 1209.52, "text": " of basic features basic feature extractors and here these basic feature"}, {"start": 1209.52, "end": 1216.52, "text": " extractors appear to be simply paying attention to one specific token in the"}, {"start": 1216.52, "end": 1222.16, "text": " vocabulary a lot. So they kind of these heads sort of specialize for single"}, {"start": 1222.16, "end": 1228.16, "text": " for single amino acids. And the same in the last layer so in the very last layer"}, {"start": 1228.16, "end": 1235.88, "text": " the task of the very last layer is to prepare for the classification task. So"}, {"start": 1235.88, "end": 1241.2, "text": " if you remember the BERT model you have layer layer layer layer and at the"}, {"start": 1241.2, "end": 1246.16, "text": " end you'll have to predict which ones are masked down here. So at the end you"}, {"start": 1246.16, "end": 1252.0800000000002, "text": " have to predict single amino acids again so if there's a proline masked here"}, {"start": 1252.08, "end": 1256.84, "text": " you'll have to predict the proline. So it also makes sense that the last layers"}, {"start": 1256.84, "end": 1266.6399999999999, "text": " would very much specialize to single tokens. So this does make sense. Now our"}, {"start": 1266.6399999999999, "end": 1271.84, "text": " question is going to be do we find the biological function where would you"}, {"start": 1271.84, "end": 1276.8, "text": " expect them? We would expect the let's say the tertiary, sorry the secondary"}, {"start": 1276.8, "end": 1281.28, "text": " structures which are sort of one level higher than the primary structures we"}, {"start": 1281.28, "end": 1285.48, "text": " would expect to find them maybe here and then we would expect to find the"}, {"start": 1285.48, "end": 1291.04, "text": " tertiary structures maybe somewhere here okay because these are most highest"}, {"start": 1291.04, "end": 1297.36, "text": " level. And then it goes it goes back again or maybe it's like we find the"}, {"start": 1297.36, "end": 1304.68, "text": " tertiary structures rather here and here again and then in the middle we'll find"}, {"start": 1304.68, "end": 1310.2, "text": " the the most high level the tertiary structures or yeah blue secondary. This"}, {"start": 1310.2, "end": 1317.28, "text": " drawing is getting too too too weird but there could be multiple scenarios but"}, {"start": 1317.28, "end": 1323.6000000000001, "text": " that could fit here but until now it sort of makes sense. So they do an"}, {"start": 1323.6000000000001, "end": 1328.72, "text": " additional investigation where as I told you sometimes you can substitute an"}, {"start": 1328.72, "end": 1334.64, "text": " amino acid and nothing really happens right and in fact that this probably"}, {"start": 1334.64, "end": 1340.04, "text": " happens in you right now you probably might have some mutation that changed"}, {"start": 1340.04, "end": 1347.1599999999999, "text": " some amino acid and you don't even realize because it's just it's fine. No"}, {"start": 1347.1599999999999, "end": 1354.1599999999999, "text": " notice. So the biologists can build these matrices of how much you can substitute"}, {"start": 1354.1599999999999, "end": 1360.92, "text": " proteins with each other. So here you see this BLOSUM62 substitution scores which"}, {"start": 1360.92, "end": 1367.76, "text": " are very I guess very high if you can substitute two protein, two amino acids"}, {"start": 1367.76, "end": 1376.32, "text": " with each other and the effect is negligible. And it's very low if it's the"}, {"start": 1376.32, "end": 1381.6, "text": " the other way around. Now this is interesting so far but you compare this"}, {"start": 1381.6, "end": 1387.2, "text": " to this matrix right here. This is the attention similarity. So what we'll do is"}, {"start": 1387.2, "end": 1393.76, "text": " for each two amino acids we take those two attention things, those two attention"}, {"start": 1393.76, "end": 1397.8799999999999, "text": " matrices and we'll calculate the correlation between the attention"}, {"start": 1397.8799999999999, "end": 1404.32, "text": " matrices. And our hypothesis is that the more correlated the attention patterns"}, {"start": 1404.32, "end": 1409.8, "text": " are between the two amino acids the more likely we are to substitute them"}, {"start": 1409.8, "end": 1416.2, "text": " because as a direct result of our language model our language model is"}, {"start": 1416.2, "end": 1424.8400000000001, "text": " it's reconstructing right these things. So our language model is going to treat"}, {"start": 1424.8400000000001, "end": 1431.16, "text": " if in natural language is like a a synonym right is our language model is"}, {"start": 1431.16, "end": 1435.68, "text": " going to treat synonyms very similar to each other because they're synonyms they"}, {"start": 1435.68, "end": 1440.76, "text": " can be exchanged. So a good language model should learn that they are almost"}, {"start": 1440.76, "end": 1444.8400000000001, "text": " the same and therefore the attention pattern is going to be almost the same."}, {"start": 1444.84, "end": 1453.56, "text": " So a high correlation we hypothesize is a means that the function of the amino"}, {"start": 1453.56, "end": 1459.4399999999998, "text": " acid is similar and therefore we can substitute it easily. So this here is the"}, {"start": 1459.4399999999998, "end": 1465.4399999999998, "text": " matrix of the correlations between each two attention patterns of these amino"}, {"start": 1465.4399999999998, "end": 1473.08, "text": " acid. And if you compare the two right here they look extremely similar. Just"}, {"start": 1473.08, "end": 1479.12, "text": " have a look for a little while and you'll see that the patterns they do not"}, {"start": 1479.12, "end": 1485.28, "text": " match perfectly but they are very very similar. The dark spots are in the same"}, {"start": 1485.28, "end": 1491.28, "text": " places the light spots are in the same places. So this already makes a good case"}, {"start": 1491.28, "end": 1498.04, "text": " that the language model here has learned something about biology. Now what we want"}, {"start": 1498.04, "end": 1505.96, "text": " to do is we want to investigate higher order higher order functions. So here"}, {"start": 1506.24, "end": 1513.76, "text": " we're interested in these contact maps right. So how likely is it that two"}, {"start": 1513.76, "end": 1518.6399999999999, "text": " amino acids are in contact and we'll look at it through the lens of attention"}, {"start": 1518.6399999999999, "end": 1523.96, "text": " as we did before. So here is the percentage of each head of each head's"}, {"start": 1523.96, "end": 1530.0, "text": " attention that is aligned with contact maps averaged over a data set suggesting"}, {"start": 1530.0, "end": 1533.8400000000001, "text": " that head 12 4 is uniquely specialized for contact prediction. So look at this"}, {"start": 1533.8400000000001, "end": 1544.16, "text": " this this head here is just spiking. So remember before we said our analysis is"}, {"start": 1544.16, "end": 1551.44, "text": " whenever whenever we're basically measuring the correlation of two things"}, {"start": 1551.44, "end": 1556.96, "text": " being in contact because we know it from our simulator or from our data set the"}, {"start": 1556.96, "end": 1562.72, "text": " correlation of that with an attention connection being particularly strong. And"}, {"start": 1562.72, "end": 1571.24, "text": " we find it in this attention head right here. So this layer 12 head number four"}, {"start": 1571.24, "end": 1577.0, "text": " will always peek out whenever two things are in contact. Now you can see that it's"}, {"start": 1577.0, "end": 1582.84, "text": " it's not like always it's like 25 percent of its attention but significantly more"}, {"start": 1582.84, "end": 1589.92, "text": " than anything else right here. In fact if you group the things by this attention"}, {"start": 1589.92, "end": 1594.92, "text": " you can build the following plot. So you can see right here probability two"}, {"start": 1594.92, "end": 1600.72, "text": " amino acids are in contact as a function of attention between the amino acids in"}, {"start": 1600.72, "end": 1606.76, "text": " head 12 4 showing attention approximates perfectly calibrated estimator which"}, {"start": 1606.76, "end": 1612.64, "text": " would be the green line. So here we simply for each for each pairs two"}, {"start": 1612.64, "end": 1620.24, "text": " amino acids for each pair of amino acids we plot we make a histogram right here"}, {"start": 1620.24, "end": 1630.08, "text": " of what they're sorry not a histogram we plot the probability if they have the"}, {"start": 1630.08, "end": 1637.72, "text": " attention weight point nine we plot how likely is it that they are in contact. So"}, {"start": 1637.72, "end": 1643.04, "text": " this is this if we just look at the data and we simply take this attention weight"}, {"start": 1643.04, "end": 1648.36, "text": " as a measure as a predictor of being in contact we get the blue curve and the"}, {"start": 1648.36, "end": 1654.24, "text": " green curve would be if we could perfectly predict from this attention"}, {"start": 1654.24, "end": 1658.6399999999999, "text": " head what the probability of contact would be and you can see that the fit is"}, {"start": 1658.64, "end": 1665.3600000000001, "text": " fairly good you can't predict with super high accuracy but the fit is fairly good"}, {"start": 1665.3600000000001, "end": 1672.5600000000002, "text": " and you can see the general trend that as the attention in this head rises the"}, {"start": 1672.5600000000002, "end": 1680.76, "text": " probability of the two amino acids being in contact with each other also rises. So"}, {"start": 1680.76, "end": 1687.1200000000001, "text": " we can sort of confidently say that BERT has learned something about a higher"}, {"start": 1687.12, "end": 1692.2399999999998, "text": " level biological structure just from the language modeling"}, {"start": 1692.2399999999998, "end": 1699.84, "text": " objective. How can we interpret this? This must somehow mean that it is"}, {"start": 1699.84, "end": 1708.0, "text": " possible it is vital for reconstructing the sequence from its"}, {"start": 1708.0, "end": 1715.84, "text": " surrounding so if we delete this right here if these two are in"}, {"start": 1715.84, "end": 1723.52, "text": " contact in the 3D structure that makes probably probably means that this thing"}, {"start": 1723.52, "end": 1728.72, "text": " right here is a very good predictor of what was here. If we mask this out"}, {"start": 1728.72, "end": 1732.36, "text": " and we're asked to reconstruct which amino acid was there then it probably"}, {"start": 1732.36, "end": 1736.4399999999998, "text": " helps to look at its neighbors right it probably always helps to look at one's"}, {"start": 1736.4399999999998, "end": 1742.8, "text": " neighbors especially also in natural language but if these two are in contact"}, {"start": 1742.8, "end": 1748.96, "text": " they have very special connection to each other it's very"}, {"start": 1748.96, "end": 1756.12, "text": " you can basically read out from this one which one this was. This is sort of like"}, {"start": 1756.12, "end": 1762.0, "text": " if you have a sentence and you say"}, {"start": 1762.0, "end": 1773.16, "text": " I don't know I can't come up with one right now but if it's like"}, {"start": 1773.16, "end": 1780.12, "text": " and then there is a name like Mark and then and then there is him"}, {"start": 1780.12, "end": 1786.68, "text": " right and you would expect if I drop out okay let's do it the other way around if"}, {"start": 1786.68, "end": 1793.5600000000002, "text": " I drop out him then from the text right here you can probably determine that it"}, {"start": 1793.5600000000002, "end": 1798.48, "text": " is some sort of pronoun but then you go back and you see it's Mark okay so it's"}, {"start": 1798.48, "end": 1809.16, "text": " not like it's not like it or some or she it's probably he or him this is sort of"}, {"start": 1809.16, "end": 1816.96, "text": " the analogous structure right here in biology. The second thing we're looking"}, {"start": 1816.96, "end": 1824.44, "text": " at is these binding sites. Now these are single properties of different"}, {"start": 1824.44, "end": 1829.3600000000001, "text": " amino acids and we're simply looking at all the incoming or sorry all the"}, {"start": 1829.3600000000001, "end": 1834.0800000000002, "text": " other tokens that focuses their attention. Why is this important? Because"}, {"start": 1834.08, "end": 1840.24, "text": " these binding sites are central to the structure of the or to the function of"}, {"start": 1840.24, "end": 1844.48, "text": " the protein right if this here is a binding site then that's a very central"}, {"start": 1844.48, "end": 1851.8799999999999, "text": " important point of the protein. So a lot of these other things are going to be"}, {"start": 1851.8799999999999, "end": 1857.1599999999999, "text": " determined by what this binding site is. This binding site needs to have a very"}, {"start": 1857.1599999999999, "end": 1860.4399999999998, "text": " particular function and therefore probably needs to be a very"}, {"start": 1860.44, "end": 1866.0, "text": " particular amino acid and the other things here are sort of supporting this"}, {"start": 1866.0, "end": 1870.52, "text": " binding site because they form the 3d structure around it and so on. So you"}, {"start": 1870.52, "end": 1877.2, "text": " would expect a lot of attention to be put on this binding site and what do we"}, {"start": 1877.2, "end": 1885.0, "text": " find? We find that it's a bit more murky than before so you can see that the"}, {"start": 1885.0, "end": 1889.1200000000001, "text": " attention is kind of spread out. Percentage of each head's attention that"}, {"start": 1889.12, "end": 1893.2399999999998, "text": " focuses on binding sites especially in the deeper layers binding sites are"}, {"start": 1893.2399999999998, "end": 1898.84, "text": " targeted at much higher frequency than would occur by chance. Head 71 has the"}, {"start": 1898.84, "end": 1906.6399999999999, "text": " highest percentage with 34%. So also here you can see that it is spread out but"}, {"start": 1906.6399999999999, "end": 1911.8, "text": " this is because multiple heads are now focusing on these binding sites because"}, {"start": 1911.8, "end": 1916.76, "text": " probably binding sites come in different variations so you'll have lots of heads"}, {"start": 1916.76, "end": 1921.84, "text": " specializing on attending to binding sites and they say it is much higher"}, {"start": 1921.84, "end": 1926.8, "text": " frequency than would occur by chance and you can see here this head is the"}, {"start": 1926.8, "end": 1932.12, "text": " highest with 34% of its attention focused on binding sites. You can also"}, {"start": 1932.12, "end": 1937.64, "text": " see the general trend of the attention being rather in the later layers which"}, {"start": 1937.64, "end": 1943.8, "text": " we would expect from a tertiary structure. Now yeah it would be"}, {"start": 1943.8, "end": 1948.96, "text": " interesting here. Here you also see that actually most of the things are in the"}, {"start": 1948.96, "end": 1956.12, "text": " in the last layer which points to points to rather maybe lower lower level"}, {"start": 1956.12, "end": 1960.0, "text": " information because we reasoned before about the last layer or I was just wrong"}, {"start": 1960.0, "end": 1964.9199999999998, "text": " but also in a general trend you can see that the attention is rather shifted"}, {"start": 1964.9199999999998, "end": 1973.3999999999999, "text": " towards the later layers because this is sort of a higher order function. Okay if"}, {"start": 1973.4, "end": 1979.6000000000001, "text": " you look at the same calibration experiment you can see that the"}, {"start": 1979.6000000000001, "end": 1982.96, "text": " picture is not as clear. There is the general trend at the beginning but then"}, {"start": 1982.96, "end": 1990.0800000000002, "text": " it sort of flattens out so you can sort of differentiate the very probably not"}, {"start": 1990.0800000000002, "end": 1994.7, "text": " a binding site from the somewhat probably a binding site but it's not a"}, {"start": 1994.7, "end": 1999.72, "text": " perfectly calibrated classifier and that might just be because there are many"}, {"start": 1999.72, "end": 2004.8, "text": " things specializing in different types of binding sites so you can't just go"}, {"start": 2004.8, "end": 2010.04, "text": " to this one head so this is just for this one head. You can't just go to that"}, {"start": 2010.04, "end": 2015.68, "text": " one and expect that to classify all the binding sites because you might want to"}, {"start": 2015.68, "end": 2023.44, "text": " be you might want to combine all of the high ranking ones here to form a"}, {"start": 2023.44, "end": 2029.3600000000001, "text": " classifier. The last experiment they do is these linear probes which where they"}, {"start": 2029.36, "end": 2034.1999999999998, "text": " just go and they just build classifiers from different parts of the network and"}, {"start": 2034.1999999999998, "end": 2040.24, "text": " you can see right here that what is predicted and how well they work. So each"}, {"start": 2040.24, "end": 2045.08, "text": " bar here is going to be the difference of performance so this is differential"}, {"start": 2045.08, "end": 2050.44, "text": " performance of diagnostic classifier by layer sorted by task order in figure 8."}, {"start": 2050.44, "end": 2054.8399999999997, "text": " Each plot shows the change in performance between the given layer and"}, {"start": 2054.84, "end": 2061.56, "text": " the previous layer. So a bar up shows it's performing better than the"}, {"start": 2061.56, "end": 2065.08, "text": " previous layer bar down shows it's performing worse than the previous layer."}, {"start": 2065.08, "end": 2071.04, "text": " So you see right here that these are the secondary structures right here and"}, {"start": 2071.04, "end": 2075.6800000000003, "text": " you can see that there is a lot of performance in the earlier layers right"}, {"start": 2075.6800000000003, "end": 2081.0, "text": " here and sort of not that high performance in the later layers whereas"}, {"start": 2081.0, "end": 2085.48, "text": " for the tertiary structures the binding site and the contact you can see that"}, {"start": 2085.48, "end": 2092.6, "text": " there is a bit of performance in places but it sort of tends to be more"}, {"start": 2092.6, "end": 2097.2, "text": " towards the middle certainly more at towards the middle of the end of the"}, {"start": 2097.2, "end": 2103.32, "text": " network than the secondary structures which sort of makes sense with our"}, {"start": 2103.32, "end": 2108.8, "text": " hypothesis. You can also see this here where they show the percent of attention"}, {"start": 2108.8, "end": 2115.0, "text": " focused as a function of layer and the red is the center of mass and you can"}, {"start": 2115.0, "end": 2121.92, "text": " see that as the secondary structures this their center of mass is at a lower"}, {"start": 2121.92, "end": 2129.2400000000002, "text": " layer in general than the tertiary functions. All of this is not perfect of"}, {"start": 2129.2400000000002, "end": 2135.0800000000004, "text": " course but it's still an open question I guess whether or not it's not perfect"}, {"start": 2135.08, "end": 2141.92, "text": " because we haven't built a strong enough language model yet. Do I want to say GPT-4"}, {"start": 2141.92, "end": 2147.84, "text": " is now for biology and not for language or is it because there is really"}, {"start": 2147.84, "end": 2155.72, "text": " you need you really can't very well predict these things just from a"}, {"start": 2155.72, "end": 2159.2799999999997, "text": " language model. I mean you should technically all the information is there"}, {"start": 2159.28, "end": 2166.0, "text": " but maybe the language model objective as such isn't able to capture that"}, {"start": 2166.0, "end": 2172.1200000000003, "text": " information. So yeah this was the paper it's pretty simple they have in the"}, {"start": 2172.1200000000003, "end": 2176.2000000000003, "text": " appendix they have a lot of these additional experiments or full"}, {"start": 2176.2000000000003, "end": 2180.96, "text": " experiments I believe for all the amino acids and so on and I invite you to"}, {"start": 2180.96, "end": 2187.0800000000004, "text": " check that out. In general I like this kind of work because it's very applied"}, {"start": 2187.08, "end": 2192.64, "text": " it's and it can you know tell us something about the nature of both these"}, {"start": 2192.64, "end": 2200.16, "text": " language models and the biological things that we that we care about in"}, {"start": 2200.16, "end": 2207.16, "text": " biology. Okay I'm just talking crap right now thanks for being here I hope you"}, {"start": 2207.16, "end": 2217.72, "text": " enjoyed it and bye bye."}]