Towards an Evolutionary Architecture
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Subramaniam, Venkat
Big up front design is discouraged in agile development. However, we know that architecture plays a significant part in software systems. Evolving architecture during the development of an application seems to be a risky business. In this presentation we will discuss the reasons to evolve the architecture, some of the core principles that can help us develop in such a manner, and the ways to minimize the risk and succeed in creating a practical and useful architecture.
 All right, welcome to the session on two words and evolutionary architecture. My name is Venkat Subramanyam. We're gonna talk about evolving architecture. We'll talk a little bit about architecture. We'll talk about some of the pros and cons of evolving, and then how do we actually approach evolving? I do appreciate questions along the way, so don't wait till the end, and anytime you do have a question, please say the words like question or make a noise to draw my attention. I'm being blinded by those lights, but if you do have a question, I'll be able to see you by blocking the lights away. So please don't hesitate. Do ask me questions, comments. Anytime you have something to interact, love to hear from you. So let's talk about architecture. The first question I would like to start with is, what's architecture? Anyone? That's about the right answer I get every time I ask that question. Because architecture is something we all can feel, but quite cannot explain, right? Well, let's make an effort. What do you think is architecture? Anyone? Don't worry about being wrong. I'm wrong most of the time. It's a design of the system. Well, we use the word design in it. Then how is it different from design versus architecture? And I'm not saying you're wrong, but that's the question that pops up. It's part of the design which is expensive. Repeat that for me, please. It's part of the design which is expensive. Aha, it is part of a design that is expensive to change. So I got an idea. I will design it in a way it's expensive. Now I got an architecture. Right? So it's a play of the words, right? Yes, I agree with you, but it's got to be more than that, isn't it? It's a set of decisions. That's pretty good, but also it's pretty general also. So what is architecture? Well, we have this notion of what an architecture is, but it's really hard to put words together. And mine is as wrong as anybody else's, but my vision of architecture is, it is a design of the system, but I would say it is a very high level design of a system with interconnectivity of components that make the system, where we are dealing more with the interfaces of what works together. It is also things where we make very high level set of decisions. I would call them as strategic decisions rather than tactical decisions. What are strategic decisions? Strategic decisions are things that set us on a course in a path. Tactical decisions are low level decisions that are really significant about a certain area or a small portion, but strategic decisions. Now let's think about what kind of decisions would be considered an architectural decision. Can we think of a few? Please. A little louder, please. Choice of storage. What else? Speak at will, speak at will. Speak at will. Speak at will. Say that again. Speak at will. Being independent of storage. Absolutely, what else? Implementation technology. Implementation technology. Distribution. Languages you use, is that architectural decision? Maybe, should be. What about the frameworks, the libraries that we use? That takes a part as well. What about the storage or what to use? What kind of database are we gonna use? Is it gonna be no SQL? Is it gonna be a relational database? Those are architectural decisions too, isn't it? Now then the question is, okay, we got a lot of these decisions we're gonna make, but why should we evolve an architecture? Now let's think about this for a little bit. When you consider the timeline of how things are, when do you normally create the architecture? In the starting, that's when you create the architecture. Everybody agrees to it? That's what we do, right? Or that's what we're being told to do. And if your job is an architect, what would you do in the beginning? Create an architecture. Would you go to the boss and say, well, I'm gonna create an architecture, but all of that could be wrong. That's the easiest way to get fired, right? So what do we do? You know what? You want architecture, I'm gonna give you the architecture. And we create the architecture in the beginning. But let's revisit this for a minute, ignore that, erase that. Would you agree that this is the progression of knowledge we have over our application? Is that reasonable to say? In the beginning, we know very little. We may think we know a lot, but that's called fallacy, right? We know very little in the beginning. Right about 40 to 50% of the time into the project, things begin to fall in place. We kind of understand what's going on. And then we know a little bit about as we go through. Is that, is it reasonable to say? Now put these two together. We will create the architecture when we have no clue. How does it sound to you? Pretty stupid, isn't it? But that's what we've been doing, right? We've been told you better create the architecture because we gotta work with the architecture. So what do we do? We create the architecture when we really have the least clue about what we are actually building. And this is a very high risk. Now we all know that upfront design or a big upfront design is a bad idea. Why is a big upfront design a bad idea? Because we are trying to create something when we really don't know a lot about what we are creating. So all the decision we make in the beginning really become obsolete, but there's something worse than that. If I'm an architect and I got hired to build an architecture and if I build it when I have the least clue about what I'm building, what am I gonna do? I'm gonna hold my fort and say, this is the architecture, we better build it because I don't wanna get fired. And so we're gonna hold this architecture and very quickly what we are building doesn't quite fit into the architecture and there's a tension between what we wanna build and what should we be really building. And so there's a very big risk of creating the architecture in the beginning. But there is also a big risk of creating the architecture later on. What's gonna happen if we evolve the architecture? Let's say we decide to evolve the architecture and we are nicely marching along in time and then we come over here and suddenly there's a feature that doesn't fit into the architecture. And we're sitting there scratching over our head, what do we do now? Oh, we gotta make all these changes to fit these bunch of features, which means a significant part of the architecture is broken and we have to deal with that mess. And that is a big concern we have. Most of the time, why do we wanna do things early because we are fearful of the future? But Ken Beck said, courage is postponing the decisions of tomorrow to tomorrow. Well, sure, it's a courage to postpone the decisions of tomorrow to tomorrow, but how do we get that courage? Especially we know that we may be burned by this tomorrow. What can we do about it? Well, here's something to think about. If you are worried about what you will face here, why would something suddenly creep up there with a bunch of features that don't fit into your architecture? How can that possibly happen? Well, of course, because we've been dealing with features and these sort of features don't fit in. But let's revisit the problem for a second. If you have a bunch of features, let's say feature one and feature two, and it goes on for several features that you have. So you got a lot of features you wanna build and you are in this mode of agile development and you pick these features and you start implementing and sure enough along the way, right in the middle, you get a bunch of features, features K and features K plus one and so on that quite don't fit in. Well, what can we do about it? Well, what if we do a little bit of a planning? Wait a minute, planning, I thought we are an agile development. We don't do any planning, right? Well, that's a fallacy. If you plan everything ahead, that's a disaster. We know that. That's one extreme on this side. The other extreme, which is equally a problem, probably more of a problem, the other extreme is when we don't do any planning at all and agile development doesn't say don't do any planning. In fact, planning is so important, we call it adaptive planning. We constantly adapt our plan in agile development. But one thing we need to think about is, how do we minimize this kind of a risk? Well, here's an idea. What if we don't take features arbitrarily and start plugging through them? What if instead we take every feature and we put down, to the extent we can at this time, two different values? What is the value this feature offers? And what is the architectural significance of this particular feature, or architectural impact of this feature? If we can start putting together a value a feature offers and architectural significance of a feature, and then we start tabulating based on the value and the architectural significance. If a feature has a lot of value and has very high architectural impact, we wanna do that in the beginning, and a feature that's not very valuable and has the least architectural impact, we wanna do it later. So if we go with this approach, what we end up doing is, we start doing the most valuable and the most architecturally impactful things in the beginning, and we tend to do the least valuable and the least impactful towards the very end. And now, if we stick with that for a minute, as you are progressing, what you end up in the middle has the less impact on architecture anyways. So it's very unlikely that would blow up your architecture based on what you've built so far, because you started with the one that has the most impact on what you're doing. There is a disadvantage to this approach. And the disadvantage is, if you take the most architecturally impact and the most valuable, your velocity is gonna be slow in the beginning. Again, if we are under corporate pressure to really look good in the beginning and show good velocity, we're not gonna show that. It is not gonna show the velocity, it's gonna show the reality. And you gotta ask yourself, what do you wanna see? Do you wanna see a perceived velocity or the reality? And if you take this approach, you may be a little slow towards implementing the feature in the beginning, but you're gonna gain speed, the velocity, as you progress because things become easier and things begin to fit into what you've built already. So that is something to think about, is to prioritize the features based on architectural significance and the value, and then tally them up and start working on them. Now, there's one small problem still. We may not have perceived this at all, and a customer may walk into the office one morning on a weird Wednesday and say, hey, I just got off the phone, we really need to implement a feature that we never ever thought about. And nobody thought of that feature, and that feature just came by just now, this is going to happen, all of a sudden, what do we do? Well, that could happen in any possible situation. It didn't happen because we had evolutionary architecture. And so when that happens, we gotta have a discussion, no matter what approach we take, and the discussion is how valuable is that feature? Is that feature is a lot more valuable than what we have done so far? If that feature is a lot more valuable than what we are planning ahead, then are you willing to pay the price to include that feature because there's an impact of the feature? This is purely something unanticipated. And so that is something we have to deal with no matter what approach we take, and we have to deal with it here as well. So one way to reduce the risk of evolution is not to just implement features. And that is to really start organizing the features based on architectural impact and the business value, and then start implementing them based on that. So that is something to think about when it comes to implementing these features. But how do we really evolve the features and how do we make it really possible to do that? Now this is where we have to really ask ourselves, what is our goal? Is our goal to create an architecture, or is our goal to deliver a product that provides value to the customers in a way it is easier for us to maintain and build? If our goal is to build the architecture, I don't think we have anything to talk about. But if our goal is to build a product that provides value and easier to maintain, then the architecture is there to support the product, not the other way around. In a way, I wanna think about it this way. I would say architecture is like salt to food, right? You can't have a good meal without it. You can't either with a lot of it. So the point is, it's got to be the right proportion. And you can say, well, salt, bring it on, and you don't put a lot of salt on your food, it becomes hard to taste, and it's unhealthy after that. You can't really have it without salt either. So to me, an architecture is like a salt. You need it, but you gotta know what proportion to have it, and it is there to support it. You don't make a meal out of it, it is there to support the meal you're having. So the way to think about architecture is, architecture is there to serve your application, not the other way around, then it becomes easier to discuss about what our goal is, what we are trying to do. So one of the things to ask ourselves is, if architecture is there to support us, how could we build a product that provides value and how could we build enough of it? There is this one idea that an architecture is very difficult to change. Now, if you have not had a chance, I would like to encourage you to read this article. In my opinion, this is a must read article. I think everybody should have read it or reread it. It's called Who Needs an Architect. This is a very small article by Martin Fowler, and it is a golden article in my opinion. You Google for it, Who Needs an Architect. It's a three and a half page article, very easy to read. And he talks about Who Needs an Architect. What is an architect's role in a project? And in that, he also talks about what Ralph Johnson says about what architecture really is. And Ralph Johnson says, architecture is something that is very hard to change in a system, and usually we tend to put more effort creating it because we are fearful of this change. But we need to really remove that from our shoulders as much as we can, and we'll talk about some of those moving forward here. So one of the things is, how do we really create an architecture and deal with it? So let's understand some of the ways we can move towards creating such an architecture. The very first thing is keep it simple. Now, here's the problem. Most of us would agree, keep it simple, but most of us don't know what simple really is, because how do we know something is really simple? We look at a solution. If I can understand that solution, hey, it's got to be simple, isn't it? Well, but the question is, are we solving the problems that we really need to solve? Or are we solving problems that we really don't need to solve? So simplicity is where we understand the problem and we remove solutions that don't have to exist. I'll give you an example. I got a call from a company, and they said we got a huge issue with concurrency. And we got problems everywhere in the code with concurrency. We got deadlocks everywhere. We got problems everywhere. And we got a lot of code to deal with concurrency, and yet we are having problems. Well, I wrote a few books on concurrency and on things like Scala, so they said you got to be a guy to come and help us. So come over and help us and recommend a solution for us. And I'll be honest about this. Before I went there, I was excited. I said, this is awesome. I'm going to go find what their concurrency problem is, and I've got these set of answers. I'm going to recommend one of them to them. And this is going to be a fun project, right? And I told them, send me your code, and at least a few weeks before I come over, I had an NDA, so there was not a problem. And so send me the code, and I'll look at your code before I come over. So they promptly send the code. And I was on a long flight, so this was a nice way to keep yourself entertained by reading code, obviously. So I read through all the code they sent me on the flight, and I went there to talk to them. And I was pretty hyped up because this is going to be awesome. I understand a concurrency problem proposal solution. So once they went there, they started describing what their application is, how they are solving it, and how concurrency has been such a problem for them. Here is all the code to deal with concurrency, and yet they're having so much problem. After listening quietly for a few hours, that is, and I asked them one question, I said, could you please help me understand one thing from the user's point of view? Where is the concurrency problem? Where is concurrency, first of all? How do you see concurrency in this particular case? And they said, this is obvious, isn't it, that we have a concurrency issue? I said, I'm sorry, but I failed to see it. Could you explain it? So the architect, nice guy, very, very nice, bright guy, spent the time explaining one more time how concurrency is important. At the end of his description, I said, please bear with me, don't get angry at me. Could you please explain to me where concurrency is needed from the use case point of view? And he kind of stayed at me and went on with it for another 15 minutes. When he finished, I said, please, I beg for your politeness here, but can you explain to me where concurrency is? He kind of scratched and said, how do I explain this? And then he thought for a minute and he looked at me and said, damn, are you telling me we don't need concurrency? And I said, can you help me see otherwise? And he kind of laughed for a minute and said, you just got rid of 60% of the code from our application. And he looked at me and said, thanks for coming. We don't need you to be here. We know how to remove the code now. And I kind of left a little heart broken because I didn't have a solution that I can propose to them. I just removed the problem from their hands. And I see this quite often. This is just one example of what I saw. But we are eager and we create architecture. And what do we do in the architecture? You got to solve concurrency problem. And the architecture involves a solution to a problem that may not even exist. So I've been working with companies for the past several years. And if you ask me what I've been doing actively for the past five years, I go to companies. And when I leave, their code has a weight loss in place. So they tend to remove code because I fear complexity. I ask them questions. Why do you do this? Why do you do this? Why do you need this? And we end up finding we actually don't need a lot of stuff. And so the program becomes a lot simpler because we're not doing stuff we don't need to do. So one way to make things simple is to recognize what are we really solving? This brings back another experience in my mind. I was working on a project. And one of these guys had written a piece of code. And literally, this is what I saw. There was 200 lines of code to take data from the database and XML being created. And then I look at this XML being created. And that is called into another function. And this XML is being passed. And I then go to this function. And this XML is being reparsed. And the data is being sent into the database. And I'm scratching my head. Why did we go from database to XML to XML to the database? So I went to the developer and said, hey, I'm looking at this code. You're going from data to XML, XML to data. And he's like, isn't that cool? I'm like, no. Why are you doing this? And he said, Venkat, you need to have a vision. I said, I failed to have that vision. Why would you do this? And he said, look at how extensible this code is. I said, I'm sorry. I don't see extensibility. Tell me about it. He said, the data could come from anywhere in the world. And the data could go to anywhere in the world. And I said, according to what we are doing right now, where does the data come from, from that table? Where does it go to here? Then why are we solving this problem? He said, Venkat, you really need to understand extensibility. We need to design this in a way it will be so extensible. So while he was saying this, I kind of took my phone and was dialing. He said, excuse me. And he took his phone and said, hello. I said, hello. He said, Venkat, is that you? I said, yes. And he said, why are you calling me? I said, how extensible this is? Why am I talking to you in person? I could be anywhere in the world talking to you, right? And the point is this. What are we really solving? And if we don't want to solve the real problems, we could be building complex things around us. And who is going to maintain all that code when we build complexity in place? So first thing is, keep it simple. And how do you keep it simple? The one way to keep it simple is really to have this vision where you remove things that are not really related and then get to the very core of the problem, what am I really solving? Now, once we do that, we can then build on on top of those things to provide the solution. Refactor willfully. Now, I don't know of a way to create a good software in one setting. I really don't know that. There is no way, in my opinion, to actually do this. So I'm a huge fan of the slogan. And the slogan is, make it work and make it better real soon. So if I make it work, if I make it do what I want it to do, as soon as it works, but not before it, as soon as it works, what are the little feature, little component, little thing we try to work, little test even. As soon as it works, we want to turn around and make it better real soon. And if we can make it work better real soon, then we are refactoring, making adjustments. And if you really think about it, if you read about a few interesting things, there was this fine gentleman, there is this fine gentleman called Frederick Brooks. And Frederick Brooks wrote some fantastic articles and books back in the 70s and early 80s. And his words are golden. They still stay true. And one of the things that Frederick Brooks said is build a software and throw it away. Now you have a way to build the real software. And I'm going to really emphasize, I am paraphrasing this or something like this. So this is not a direct quote. I'm just paraphrasing that. So he said that, go ahead and build a software. And when you are done, throw it away and start again. And you have a chance of building the right software, a real software, whatever that is. But the point really is this. Is he wrong? Well, he is absolutely right. But there's one problem. You don't go to work tomorrow and say, or Monday and say, boss, I exactly figured this out. We're going to build a software. When it's done, we're going to throw it away. And we're going to start over. Don't do that and get fired and call me back and say, you said this, right? I've won you that. So that's not practical, right? But does it mean that Frederick Brooks is wrong? If it's not practical. Well, what if we write small amount of software and throw it away almost continuously? And then we rebuild that part. This is where refactoring can help. This is what test-driven development is something I like. This is where I like a little active prototyping experiments so that there are a number of things we can do to build in small pieces and throw it away and rebuild it. So refactoring is a very valuable tool. And if we can refactor along the way, not huge refactoring, but small amount of refactoring. And this can help us a great deal. In fact, we can learn quite a bit from a lot of other industries and other experiences. If you look at masterpieces of paintings, and if you look at people who have done wonderful job in painting, and you look at a masterpiece and say, wow, what a great painting by this particular artist of the past. Well, you will notice one thing if you read a little bit deeper. Those masterpieces that we look at, they were not done as the masterpiece in one sitting. Often, they built this for a particular king or somebody who is rich, and they wanted something to be painted or built. And what they did was they created several different prototypes. And they created not just one or two, sometimes dozens of prototypes they created. And each of the prototype starts out by showing one feature versus another. And then there are things that they quite didn't like in those prototype. And they changed it in the second prototype, and the third one, and the fourth one. And then there is a prototype that is a lot closer to that masterpiece we all admire, maybe with one feature that is different. So even the brightest of the people in the world never created things in one sitting. They created things by creating prototypes. And here we come along as programmers, and we sit down, and we expect to write code once. And all of a sudden, it should be done. What a foolish way of thinking about it. So it is very important for us to be able to evolve this that becomes extremely important. The other feature to think about is reversibility. And to me, this feature is something that is very critical because if you forget reversibility, it can be very expensive to evolve software. Now what does reversibility really mean? Reversibility means how can I back out of a certain design decision that I make? So if I made a decision, and if I have to be stuck with the decision, then I should think a lot more about the decision than a decision where I can easily change. Now let's think of a few decisions that are not as reversible. A language that you use is a fairly hard decision you cannot reverse out of because the code, as we know it right now, is created in a certain programming language. About a year into the project, you come along and say, yeah, we don't like this language, we're gonna switch over, that is expensive to make the change. So languages are, for example, very hard to reverse. But the question is, are there things we can make easier to reverse? Now just because something are easier to reverse doesn't mean they actually are easier to reverse. So we gotta make sure that the things that we targeted as easy to reverse really are easy to reverse. I'll give you an example of this. I was working on a system about seven months ago, and I was going to put this in production. It was about two months of development effort, give or take, it was a fairly rapid development to get into the market. And one of the things I said to myself is, I wanna focus on the features right now. I wanna postpone any single decision that I can postpone. I don't wanna deal with it today, I wanna postpone it. So one of the decisions I postponed on the project is the database I can use in deployment. Now you may call me crazy, because that may have a major impact, but I purposefully said, I am not gonna worry about the database I'm gonna use in production. And so I ignored that and I went on to develop the application, absolutely, I'm about two weeks away from going into production. And my little brain is telling me, Venkat, you better pay attention right now. And this is the last responsible moment, I cannot ignore this any further. So two weeks before going into production, I decide to go look into what database I could use in production. By now, I got a better understanding of which cloud I'm gonna run this on, what kind of environment they have available to me, what are the options, so I'm looking at all of these, and I figured out what database I'm gonna actually use. So I got a copy of the database, installed on my machine, modified my code to go use that other database using a configuration setting, and I run all my tests, and to my fear, two, actually three tests actually failed. And immediately what strikes you is a panic, uh-oh, wait a minute, three tests are broken, what is it? And I look into these three tests, and of course, Google for the error you get, within two minutes I find out what the problem is, there was a data type compatibility between the database I was using and the database I'm gonna use moving forward. So I read a little bit about it, I make a change to the data model in the code, rerun the code, all the tests are passing, flip back to the old database I was using, all the tests are still passing, and you get a feedback within seconds that everything is actually working still after this change. And so in this particular example, I postponed the decision of what database I'm gonna use. Now the application has been in production for about seven months, it is running on the database that I did not develop with, it's running on the database that I switched over to about two weeks before, within two weeks before deployment, and you can see that that's working just fine. So the point really is, what is the benefit of doing this? The benefit of doing this is, it gave me the ability to focus on the real essential features in the beginning, and I designed it in a way that I can flip over the database I wanna use with minor change. So the cost of changing the database two weeks before was installing it, running all the tests, seeing some of them fail, learn about it, and fix them a total of about 10 minutes of effort. And, but that is not the way it's worked all the time. I got some scars on my body because of bad decisions that have been done in the past. I worked in a company where we used a certain database. I will not mention the name of the database here. And, oh, every single class, every single class in the application, and there were about 500 of them, had to inherit from a common base class that was provided by this database vendor. And there were several classes in our system that were coupled to the data structure provided by this database. And we didn't know better. I was young back then. I needed the money, so I worked on this project. And I would also defensively say, I didn't create the initial code. I inherited it from other people, but I should have known better, right? Because when you come to work on a project, these are smells you should know. I know it today, but I didn't know it back then. You start kind of working on the code that's given to you. And one day, the database vendor decided to make a very significant upgrade on the database. And this was a breaking change on their part. Just that one change with very little feature really ended up costing the company I worked for a good eight months of recoding. And eight months of recoding the application just to switch the database version. And a few years later, I was looking through this, and I was asking myself, what really went wrong? And what went wrong in that case is very tight coupling. So if you couple very tightly to something, it is very hard to reverse out of it. If you loosely couple to something, it is easier to back out. But oftentimes we would convince ourselves that a database is something you have to deeply couple with. And the answer is no, we don't. And if such a deep coupling is prompted, that is a smell we should bring out and talk about and say, we don't wanna do that. What are the alternatives we can consider to remove that kind of tight coupling? Because changing the database that we rely upon is something we have to do potentially. We got asked this question, how likely is it for us to have to change this database? I was just talking to a client two days ago, and I was talking to them and they said, we got tons of stored procedures with a particular relational database. And I asked them, what would you do if you have to switch the database? They just laughed and said, that will never have to happen. We are married to this database, and if we have to change it, pretty much we are dead in the water, right? So that is a very irreversible decision. But my question really is this, did they make this decision consciously or did they make this decision accidentally? So what if we can create an architectural diagram, an architectural document, a very lightweight one, and highlight reversible versus irreversible, and start putting things on it as, what is reversible and what is irreversible? And when you start putting things very explicitly on it, it brings the team together to say, do you really want that to be irreversible? Do you want this really to be reversible? Do we understand the cost of this irreversibility and consequences? Do we understand the cost of this reversibility and the consequences? So we need to be able to, there's a cost for both, and we need to ask ourselves, are we making the right decision on what is reversible and irreversible? I don't want to find out what is irreversible later on as, oh, this is how it turned out to be. I want this to be a conscious decision we have architected towards reversibility. So I would like to see two sections in the lightweight document, reversible items, irreversible items, what are those and why they are such, and what are the consequences and costs of doing it. That can help us to decide whether we want reversibility or irreversibility in our application in various areas. So postponing things to the last responsible moment. So how do we know what's the last responsible moment and why should we really do this? This really ties in with another thing called the Yagney feature. Now what is Yagney? Yagney is for you aren't going to need it. Lot of times we build things that we really don't need with the perception that how could we live without it? We really have to build it, of course, right? So we tend to really build things that we really don't need. And if we can ask ourselves, do we really need this feature? Do we really need this particular component? We could ask ourselves to do that. And last responsible moment is to really find a time when we no longer can postpone this particular decision. And in my example, the last responsible moment to pick the database was when I have committed to my deployment. I didn't know where I'm going to deploy yet. I was considering multiple different cloud services. And I wanted to wait until I make the decision before I decided the database I wanted to use. In my case, that was the last or some other moment for that specific decision on the database. But what language did I use? That was selected for early, early on. And how did I choose the language? When did I choose the language? I didn't choose the language before I started the project. But the very first day I sat down on the project, I asked myself a few questions. I said I can comfortably program in about eight different languages. So here are the pool of languages available on my hand. I'm willing to learn a language if I have to do. That's perfectly fine as well. But what are the things I wanted to consider? I wanted to consider languages that will not only solve the problem for me, but I also wanted to make sure the set of tools I selected made testability extremely easy. I wanted a complete round trip of testability, both integration, functional and unit test. And so I looked for a solution that will help me with that kind of testability. That was a decision I made into it. So these are the decisions I made. Of course, that had to be made fairly early on. The last responsible moment for that decision was the very first day of the project. But selecting a database could be done two weeks before in my case. And there were decisions that were postponed along the way. And having this willingness to postpone decisions, this is very different from procrastination. Procrastination is where we postpone things beyond the last responsible moment. So this is really knowing what is important now and what can wait until a later time. That is something for us to decide. And postponing things to a later time gives us a certain advantages. What are the advantages? First of all, if the cost of doing something now is the same as cost of doing something later, you would rather do it later than now because you can argue you are smarter tomorrow than you are today. This goes back to the picture we started with. And the picture is, you know more in this time frame than you do here. So if it can wait until here, you rather make it wait than do it up here because you have more information as you move forward. If the cost of developing it is going to be higher now compared to later, now why is it going to be higher now? Because a lot of features don't exist. But this will fit in really well when some other code has been written. Then I would rather wait to do it later than now and not do it right now. But if the cost is reversed, if it's more expensive to do it later than now, I would try to find out, A, why it is more expensive later, and B, what is the certainty to which I would definitely need this? And if the certainty is very high and the cost is high, I would do it now rather than doing it later. If the certainty is very low, I would rather postpone it and take the cost of it because there's a probability I wouldn't need it, then I would not have to spend money on it at all. So in other words, one of the things we need to think about in developing software is the economic impact of it. What is the economic impact of what we do? And unfortunately, we cannot develop software in isolation of economic knowledge. We need to definitely do that. But there's another holy thing we often get across. What about extensibility? We all want our code to be extensible. Who doesn't want it? Absolutely. But here's the problem of extensibility. We often think our code has to be extensible, but what we want it to be extended is not what we actually face eventually. Now, let's think about this for a minute. There are three kinds of people, kinds of people, on a project. The first kind knows a lot about the domain. So there are people on your team who know a lot about the domain. You can go ask them any question about the domain. They can answer it for you. They are part of your development team. They are available. The second type of people knows a lot about software development. These guys rock. They can sit there and write code all day, but don't know anything about the domain, right? Or not ask much. The third kind of people who are very few in number knows a lot about both domain and software development. I will politely ignore the fourth category here that we don't want to talk about, right? So the point really here is the people who can make the right set of decision are the people in the third category who understand the domain and understand the software development practices. Because these people have a well-rounded knowledge to look at both. But I'm sorry, that is foreign, few in general. So what can we do instead? What we can do instead is we can have these two set of people collaborate with each other. And this is something very important to think about. So when I used to work in areas that I had no knowledge at all in the domain, that happens a lot, because I'm a software developer by practice. I'm a software developer by education. I'm a software developer by what I do every day. I love coding. That's what I like. But I get hired into projects where I don't know anything about the domain. It could be a financial industry. It could be chemical industry. It could be an industry that's focused on medicine. I don't know. People call me and say, can you come and work with us? There are two things I normally do. I try to get some education on the domain as much as my little brain can hold. I would read up some very important material. I would come to terms with vocabulary. I would shamelessly ask questions. What does that mean? But I will also immediately find the domain expert and say, I'm designing the software for you. I'm going to ask you a lot of stupid questions. And I want you to be very kind and polite and answer these questions for me. And I would go to them and say, I'm designing this. I'm wondering if I should worry about these kinds of data also. Hence, they will usually give me three kinds of answers. Sometimes they will say, oh, of course, that we have to worry about too. We just didn't mention it, but absolutely, we have to worry about it. And then I'll ask them, do we have to worry about it now? Is it something we'll worry about later on? What are the probability I would ask the kind of questions? Or they would laugh and say, no, we definitely don't want to do that. This kind of problem will never have that kind of data or things to work with. But a third answer they would give is, we are not sure. We've got to go ask other people and find out. But that is my job as a designer is to ask these extensibility questions. Are we going to face these problems? But here's the thing. If you're going to plan for extensibility, that requires the knowledge of the domain and the application. If we ignore these two and build extensibility, what's going to happen is we make it extensible for things that it won't be really meaningful. I'll give you an example of this. I was invited to a company. I was sitting down talking to them. And we had group people sitting and having lunch. And the entire team was there along with the manager. And the manager asked me, hey Venkat, I got a question for you. Every single day, my team said they are building an extensible product. And every single day, they guaranteed me, this product is extensible. And we went into production. And two weeks after we went into production, a customer called us and suggested a change. And they said the product is not usable without this change for their situation. And I looked at the request. And it was very clear how simple and more really integral this change was. So I promised to my company, we'll get this back quickly. Because my team has created extensible code. I came to my team and said, can you make it extensible? And the team said, they need a few weeks to make the change. And then so he said, I don't understand this. Where things went wrong? I said that to the manager, it is your fault. And the manager said, what's my fault? What did I do wrong? And I said, if you're not as them for the change, the product will still be extensible. So the point is extensibility is what a product has until you ask it to be extended. And the minute you say extended, it is not going to handle it. So where things go wrong? Well, Ralph Johnson said this in a wonderful way. He said that we build hierarchies, interfaces, all of these we build. Why do we build? We build it for the sake of extensibility. But in doing so, we build enough complexity into the product that it becomes really hard to extend it when the time comes to extend it. So one of the ways to make things extensible is to keep it minimal and very lean and solve the problem we know today. And when we really see a real problem tomorrow, we should be able to make it extensible based on that rather than building a hierarchy of things when we don't really need that kind of extensibility. That becomes very important. So one of the ways to do that is to use what is called triangulation. Ken Beck talks about triangulation. And I think this is a wonderful approach. Now you talk to people and they will tell you duplication is bad, don't do it. And one of those guys too, I say this often. However, there is one way to accommodate this. Remember to make it work and make it better real soon. And this is something we can use here. So when you know a particular feature, just implement it. So we have one implementation of a feature. It is working. We have the code in place. You come across yet another particular feature. And it's got a little variation from what you have, but a lot of similarities. Well, go ahead and implement that. And that may have a little bit of duplication at this point. That's perfectly okay. Make it work. And now that you have both of those working, now is your refactoring opportunity to pull the commonality out into a common base. So extract the commonality. Extract the common. There are two ways to extract the common. One is to use inheritance to pull it up. And the other way is to use delegation and pull it to the site. And you can make the decision which makes more sense, depending on the context. But you wait until this evolves in front of you to make this decision. But once you have done this, now the code is ready to handle a third type of situation that arises, which is similar to this. So you cannot just magically make a code extensible. You have to really know what extensibility is in the given context and make that affordable. Otherwise, we end up creating something overly complex and we don't get extensibility and we get a lot of code we have to deal with. So it is actually a double JPD. What about reuse? Well, reuse is the holy grail in our field. You go to people and say, what's good about opting to programming? Oh, we can make everything reusable. Grady Butch said this really nicely. He said, don't make code reusable. First, make it usable. Now, this is one thing you see quite often. You go to programmers and they say, why should I create code when I can create frameworks? So everyone wants to create framework. I was actually on a project I was reviewing their code and I noticed something really peculiar. I was looking at interfaces and I love interfaces. I said, these guys are good. They got interfaces. I looked around and there was reflection being used. I said to myself, I love these guys. I love reflection. They have used it. I looked around a little bit further and I saw what they were doing and they had factories. I mean, who doesn't love factories, right? I said, this is awesome. These guys have used factory. Then I went through and reviewed one more time and I found something very, very strange. They had interfaces and there were two kinds of interfaces in their system. The first kind of interface, they only had one class implementing it and there were a few interfaces with no class at all implementing it. So I was a little puzzled. I went back to the team and said, help me out with this. How come you have interfaces that only have one classes? And they said, oh, we know what happened. The guys who wrote it left, that's the first thing they said and those guys wanted to build a framework so they created all of these things and that quite didn't materialize. So when they left, we have all these interfaces hanging around with no real implementations for the second type. So this urge of creating something very reusable is in all of us and we have to avoid the temptation. We got to create something that is usable. I was working with a company where they decided that several of their products had duplication in it and one of the higher up guys said, who was pretty technical said, that said, I want to create a reusable component. I don't want this to be done in every product. It took 12 years, I'm not kidding, 12 years for them to extract the reusable component from it and the reason is sure everything is doing the same but when you get into the details, you quickly realize it's not really the same and that's where it gets very tricky. Now, if you look at successful reusable framework, not a single one of them were created by a committee sitting in a room and as a words of Alexander, he said, designed by committee creates a horse when they are designing a camel, right? So the point really is you cannot just walk into a room and say, I want to create something reusable. A better way to create reusability is to use it and use it and use it and eventually realize this is something reusable and start refactoring and extracting the component, then you create something very, very reusable. If you look at some of the most popular, truly reusable framework, they all went through that path of creation and anything that's really sucked is what was created in a design room and then they said this is reusable because we told you so and the last thing I want to mention here is about libraries and frameworks. I have to honestly tell you, I fear this and the reason I fear is twofold. Libraries and frameworks are the shiny toys for us as programmers. This is like taking kids into a candy store and what do kids say? I want this, this, this, this and this and I want more of it, right? And we programmers are like that. You do a quick web search, I want that, that and that and that, right? I was talking to a team recently and they said, oh, we used to run all these tests. I said, you're talking a past tense. Why are you using a past tense? Why can't you run those tests? And they said because we upgraded some of the plugins on our JavaScript side and all these tests quit working. And I asked them the question I shouldn't have asked. I said, how many plugins do you have? And they all went quiet because it was not in dozens. It was a lot more than in dozens. They had like a hundred framework they were using. And I asked them, what does your product do? And they said, we are still figuring it out, but we have 150 frameworks we are using, right? So why is this infatuation to use frameworks and tools and libraries? The more you use, the more you have to maintain, the more you have to struggle, think about the simplicity they talked about in the beginning. There is another problem. I'll share with you another experience, which was very bitter. This was back in time when the economy was really bad and I was consulting for a company. And they were interviewing a candidate who was extremely good and he had built a framework. So the gentleman who was the hiring authority came to me and said, hey, we're gonna hire this person. What do you think? I said, I love it. This person is the best person we have ever interviewed. Hire him. He's actually really good. I really wanna learn a few things from him. So please don't talk to me. Go and hire him now. We can't lose the time. And then he said, what do you think of the framework he's been developing? I said, that's an awesome framework. It's lovely. And then he has the third question. What do you think of us using the framework? I said, oh no, don't do that. He said, Venkat, you need to have a vision. Do you notice he's got distributed persistence? I said, you haven't talked about distributed persistence to me, not a single day. How did that become your feature right now? He said, look, it can do distributed messaging. I said, you didn't even mention the word messaging to me. Where did that come from? Please focus on building the application. Release it in the timeframe we have for the six months. Then you do whatever you wanna do. I don't care. So hire the guy. Don't commit to the framework. So we clearly comes over, sits on my table and says, Venkat, I got a news for you. I remember this very clear Wednesday afternoon. He said, I got a news for you. I said, what's up? We have hired this person. I leaned back on my chair and said, I cannot tell you how happy I am to hear this news. I'm really excited. This is gonna be fun. This guy is awesome. I'm gonna learn a lot from him as well. We're gonna just rock. This is great. And then he said, well, as part of the hiring contract, we have agreed to use this framework. I put my head on the table for a second. I said, you know what? I'm sorry, but I quit. I'm resigning. He said, oh, you are so predictable. I said, yes, that's good. And I said, do me a favor. If you have any project, please call me. I wanna come and work with you, but not on this project anymore. And I picked up my stuff and left. I went home. This was one of those few days I actually go home on time. I knocked on the door, walked in. My wife said, you are early, way too early. I said, because I resigned. And my wife looked at me and said, really, did you? Well, you can come and help me with the dishes. That's how she is. So I was jobless pretty much at home, but it felt right. And about six months later, they were supposed to finish the project. Well, I got a call 18 months later. 18 months later, I got a call. I said, hello. And he said, go ahead and say it. I said, I'm sorry to say what? Well, I told you so. I said, I'm sorry to hear. I don't wanna tell you I told you so, but what happened? Well, in the 18 months, they put a lot of money and time building the framework. Because the framework they never knew about that they decided to use did almost everything they wanted, but not quiet. And the product has not been released in 18 months. And I said, why are you calling? He said, can you come and help us? I said, there's a reason I ran away. Right? I'm sorry, I cannot. Well, that project never got done. And it was a major failure. And it's very disheartening to see that. There's a name for this. It is called RDD. And I have seen this in many companies, not one. That is called resume-driven development. People want to use technology so their curriculum vitae or resume can look really shiny. Hey, bring all these. In fact, when you hire people, if their resume has 20 frameworks or libraries listed, you know it's a bad sign, right? So that is a tendency to say, I wanna use all the shiny tools, but let's ask ourselves, what is the problem we are trying to really solve? So to summarize, if you wanna create extensible, evolutionary architecture, we have to consider some of these factors. We got to keep things simple. We got to refactor the code. Think about reversibility, but identify them very clearly. What is reversible? What is not reversible? Why? What are the consequences? And think about postponing things that you can actually postpone. And think about extensibility, not in terms of how can I arbitrarily make things extensible, but more meaningfully based on the problem domain that you are trying to work with. And then use a triangulation and extract the reusable components from it, or extensible component from it. Reusability is something that comes from active use and not by upfront design. And also think about what minimum set of libraries and frameworks you can end up using, and that can help us to create evolutionary architecture along the way. I really hope you found that useful. Thank you very much. Thank you.
 All right, welcome to the session on  two words and evolutionary architecture.  My name is Venkat Subramanyam.  We're gonna talk about evolving architecture.  We'll talk a little bit about architecture.  We'll talk about some of the pros and cons of evolving,  and then how do we actually approach evolving?  I do appreciate questions along the way,  so don't wait till the end,  and anytime you do have a question,  please say the words like question  or make a noise to draw my attention.  I'm being blinded by those lights,  but if you do have a question,  I'll be able to see you by blocking the lights away.  So please don't hesitate.  Do ask me questions, comments.  Anytime you have something to interact,  love to hear from you.  So let's talk about architecture.  The first question I would like to start with is,  what's architecture?  Anyone?  That's about the right answer I get  every time I ask that question.  Because architecture is something we all can feel,  but quite cannot explain, right?  Well, let's make an effort.  What do you think is architecture?  Anyone?  Don't worry about being wrong.  I'm wrong most of the time.  It's a design of the system.  Well, we use the word design in it.  Then how is it different from design versus architecture?  And I'm not saying you're wrong,  but that's the question that pops up.  It's part of the design which is expensive.  Repeat that for me, please.  It's part of the design which is expensive.  Aha, it is part of a design that is expensive to change.  So I got an idea.  I will design it in a way it's expensive.  Now I got an architecture.  Right?  So it's a play of the words, right?  Yes, I agree with you,  but it's got to be more than that, isn't it?  It's a set of decisions.  That's pretty good, but also it's pretty general also.  So what is architecture?  Well, we have this notion of what an architecture is,  but it's really hard to put words together.  And mine is as wrong as anybody else's,  but my vision of architecture is,  it is a design of the system,  but I would say it is a very high level design of a system  with interconnectivity of components that make the system,  where we are dealing more with the interfaces  of what works together.  It is also things where we make very high level  set of decisions.  I would call them as strategic decisions  rather than tactical decisions.  What are strategic decisions?  Strategic decisions are things  that set us on a course in a path.  Tactical decisions are low level decisions  that are really significant about a certain area  or a small portion, but strategic decisions.  Now let's think about what kind of decisions  would be considered an architectural decision.  Can we think of a few?  Please.  A little louder, please.  Choice of storage.  What else?  Speak at will, speak at will.  Speak at will.  Speak at will.  Say that again.  Speak at will.  Being independent of storage.  Absolutely, what else?  Implementation technology.  Implementation technology.  Distribution.  Languages you use, is that architectural decision?  Maybe, should be.  What about the frameworks, the libraries that we use?  That takes a part as well.  What about the storage or what to use?  What kind of database are we gonna use?  Is it gonna be no SQL?  Is it gonna be a relational database?  Those are architectural decisions too, isn't it?  Now then the question is, okay,  we got a lot of these decisions we're gonna make,  but why should we evolve an architecture?  Now let's think about this for a little bit.  When you consider the timeline of how things are,  when do you normally create the architecture?  In the starting, that's when you create the architecture.  Everybody agrees to it?  That's what we do, right?  Or that's what we're being told to do.  And if your job is an architect,  what would you do in the beginning?  Create an architecture.  Would you go to the boss and say,  well, I'm gonna create an architecture,  but all of that could be wrong.  That's the easiest way to get fired, right?  So what do we do?  You know what?  You want architecture, I'm gonna give you the architecture.  And we create the architecture in the beginning.  But let's revisit this for a minute,  ignore that, erase that.  Would you agree that this is the progression of knowledge  we have over our application?  Is that reasonable to say?  In the beginning, we know very little.  We may think we know a lot, but that's called fallacy, right?  We know very little in the beginning.  Right about 40 to 50% of the time into the project,  things begin to fall in place.  We kind of understand what's going on.  And then we know a little bit about as we go through.  Is that, is it reasonable to say?  Now put these two together.  We will create the architecture when we have no clue.  How does it sound to you?  Pretty stupid, isn't it?  But that's what we've been doing, right?  We've been told you better create the architecture  because we gotta work with the architecture.  So what do we do?  We create the architecture when we really have  the least clue about what we are actually building.  And this is a very high risk.  Now we all know that upfront design  or a big upfront design is a bad idea.  Why is a big upfront design a bad idea?  Because we are trying to create something  when we really don't know a lot about what we are creating.  So all the decision we make in the beginning  really become obsolete, but there's something worse than that.  If I'm an architect and I got hired to build an architecture  and if I build it when I have the least clue  about what I'm building, what am I gonna do?  I'm gonna hold my fort and say,  this is the architecture, we better build it  because I don't wanna get fired.  And so we're gonna hold this architecture  and very quickly what we are building  doesn't quite fit into the architecture  and there's a tension between what we wanna build  and what should we be really building.  And so there's a very big risk  of creating the architecture in the beginning.  But there is also a big risk  of creating the architecture later on.  What's gonna happen if we evolve the architecture?  Let's say we decide to evolve the architecture  and we are nicely marching along in time  and then we come over here and suddenly there's a feature  that doesn't fit into the architecture.  And we're sitting there scratching over our head,  what do we do now?  Oh, we gotta make all these changes  to fit these bunch of features,  which means a significant part of the architecture  is broken and we have to deal with that mess.  And that is a big concern we have.  Most of the time, why do we wanna do things early  because we are fearful of the future?  But Ken Beck said,  courage is postponing the decisions of tomorrow to tomorrow.  Well, sure, it's a courage to postpone  the decisions of tomorrow to tomorrow,  but how do we get that courage?  Especially we know that we may be burned by this tomorrow.  What can we do about it?  Well, here's something to think about.  If you are worried about what you will face here,  why would something suddenly creep up there  with a bunch of features that don't fit into your architecture?  How can that possibly happen?  Well, of course, because we've been dealing with features  and these sort of features don't fit in.  But let's revisit the problem for a second.  If you have a bunch of features,  let's say feature one and feature two,  and it goes on for several features that you have.  So you got a lot of features you wanna build  and you are in this mode of agile development  and you pick these features and you start implementing  and sure enough along the way, right in the middle,  you get a bunch of features,  features K and features K plus one and so on  that quite don't fit in.  Well, what can we do about it?  Well, what if we do a little bit of a planning?  Wait a minute, planning,  I thought we are an agile development.  We don't do any planning, right?  Well, that's a fallacy.  If you plan everything ahead, that's a disaster.  We know that.  That's one extreme on this side.  The other extreme, which is equally a problem,  probably more of a problem,  the other extreme is when we don't do any planning at all  and agile development doesn't say don't do any planning.  In fact, planning is so important,  we call it adaptive planning.  We constantly adapt our plan in agile development.  But one thing we need to think about is,  how do we minimize this kind of a risk?  Well, here's an idea.  What if we don't take features arbitrarily  and start plugging through them?  What if instead we take every feature  and we put down, to the extent we can at this time,  two different values?  What is the value this feature offers?  And what is the architectural significance  of this particular feature,  or architectural impact of this feature?  If we can start putting together  a value a feature offers  and architectural significance of a feature,  and then we start tabulating based on  the value and the architectural significance.  If a feature has a lot of value  and has very high architectural impact,  we wanna do that in the beginning,  and a feature that's not very valuable  and has the least architectural impact,  we wanna do it later.  So if we go with this approach,  what we end up doing is,  we start doing the most valuable  and the most architecturally impactful things  in the beginning,  and we tend to do the least valuable  and the least impactful towards the very end.  And now, if we stick with that for a minute,  as you are progressing,  what you end up in the middle  has the less impact on architecture anyways.  So it's very unlikely that would blow up your architecture  based on what you've built so far,  because you started with the one  that has the most impact on what you're doing.  There is a disadvantage to this approach.  And the disadvantage is,  if you take the most architecturally impact  and the most valuable,  your velocity is gonna be slow in the beginning.  Again, if we are under corporate pressure  to really look good in the beginning  and show good velocity,  we're not gonna show that.  It is not gonna show the velocity,  it's gonna show the reality.  And you gotta ask yourself, what do you wanna see?  Do you wanna see a perceived velocity or the reality?  And if you take this approach,  you may be a little slow towards implementing the feature  in the beginning,  but you're gonna gain speed, the velocity,  as you progress because things become easier  and things begin to fit into what you've built already.  So that is something to think about,  is to prioritize the features  based on architectural significance and the value,  and then tally them up and start working on them.  Now, there's one small problem still.  We may not have perceived this at all,  and a customer may walk into the office one morning  on a weird Wednesday and say,  hey, I just got off the phone,  we really need to implement a feature  that we never ever thought about.  And nobody thought of that feature,  and that feature just came by just now,  this is going to happen,  all of a sudden, what do we do?  Well, that could happen in any possible situation.  It didn't happen because we had evolutionary architecture.  And so when that happens,  we gotta have a discussion,  no matter what approach we take,  and the discussion is how valuable is that feature?  Is that feature is a lot more valuable  than what we have done so far?  If that feature is a lot more valuable  than what we are planning ahead,  then are you willing to pay the price  to include that feature  because there's an impact of the feature?  This is purely something unanticipated.  And so that is something we have to deal with  no matter what approach we take,  and we have to deal with it here as well.  So one way to reduce the risk of evolution  is not to just implement features.  And that is to really start organizing the features  based on architectural impact and the business value,  and then start implementing them based on that.  So that is something to think about  when it comes to implementing these features.  But how do we really evolve the features  and how do we make it really possible to do that?  Now this is where we have to really ask ourselves,  what is our goal?  Is our goal to create an architecture,  or is our goal to deliver a product  that provides value to the customers  in a way it is easier for us to maintain and build?  If our goal is to build the architecture,  I don't think we have anything to talk about.  But if our goal is to build a product  that provides value and easier to maintain,  then the architecture is there to support the product,  not the other way around.  In a way, I wanna think about it this way.  I would say architecture is like salt to food, right?  You can't have a good meal without it.  You can't either with a lot of it.  So the point is, it's got to be the right proportion.  And you can say, well, salt, bring it on,  and you don't put a lot of salt on your food,  it becomes hard to taste, and it's unhealthy after that.  You can't really have it without salt either.  So to me, an architecture is like a salt.  You need it, but you gotta know what proportion to have it,  and it is there to support it.  You don't make a meal out of it,  it is there to support the meal you're having.  So the way to think about architecture is,  architecture is there to serve your application,  not the other way around,  then it becomes easier to discuss about  what our goal is, what we are trying to do.  So one of the things to ask ourselves is,  if architecture is there to support us,  how could we build a product that provides value  and how could we build enough of it?  There is this one idea that an architecture  is very difficult to change.  Now, if you have not had a chance,  I would like to encourage you to read this article.  In my opinion, this is a must read article.  I think everybody should have read it or reread it.  It's called Who Needs an Architect.  This is a very small article by Martin Fowler,  and it is a golden article in my opinion.  You Google for it, Who Needs an Architect.  It's a three and a half page article,  very easy to read.  And he talks about Who Needs an Architect.  What is an architect's role in a project?  And in that, he also talks about what Ralph Johnson says  about what architecture really is.  And Ralph Johnson says,  architecture is something that is very hard to change  in a system,  and usually we tend to put more effort creating it  because we are fearful of this change.  But we need to really remove that from our shoulders  as much as we can,  and we'll talk about some of those moving forward here.  So one of the things is,  how do we really create an architecture and deal with it?  So let's understand some of the ways we can move towards  creating such an architecture.  The very first thing is keep it simple.  Now, here's the problem.  Most of us would agree, keep it simple,  but most of us don't know what simple really is,  because how do we know something is really simple?  We look at a solution.  If I can understand that solution,  hey, it's got to be simple, isn't it?  Well, but the question is,  are we solving the problems  that we really need to solve?  Or are we solving problems that we really don't need to solve?  So simplicity is where we understand the problem  and we remove solutions that don't have to exist.  I'll give you an example.  I got a call from a company,  and they said we got a huge issue with concurrency.  And we got problems everywhere in the code with concurrency.  We got deadlocks everywhere.  We got problems everywhere.  And we got a lot of code to deal with concurrency,  and yet we are having problems.  Well, I wrote a few books on concurrency  and on things like Scala,  so they said you got to be a guy to come and help us.  So come over and help us and recommend a solution for us.  And I'll be honest about this.  Before I went there, I was excited.  I said, this is awesome.  I'm going to go find what their concurrency problem is,  and I've got these set of answers.  I'm going to recommend one of them to them.  And this is going to be a fun project, right?  And I told them, send me your code,  and at least a few weeks before I come over,  I had an NDA, so there was not a problem.  And so send me the code,  and I'll look at your code before I come over.  So they promptly send the code.  And I was on a long flight,  so this was a nice way to keep yourself entertained  by reading code, obviously.  So I read through all the code they sent me on the flight,  and I went there to talk to them.  And I was pretty hyped up  because this is going to be awesome.  I understand a concurrency problem proposal solution.  So once they went there,  they started describing what their application is,  how they are solving it,  and how concurrency has been such a problem for them.  Here is all the code to deal with concurrency,  and yet they're having so much problem.  After listening quietly for a few hours, that is,  and I asked them one question,  I said, could you please help me understand one thing  from the user's point of view?  Where is the concurrency problem?  Where is concurrency, first of all?  How do you see concurrency in this particular case?  And they said, this is obvious, isn't it,  that we have a concurrency issue?  I said, I'm sorry, but I failed to see it.  Could you explain it?  So the architect, nice guy, very, very nice, bright guy,  spent the time explaining one more time  how concurrency is important.  At the end of his description, I said,  please bear with me, don't get angry at me.  Could you please explain to me  where concurrency is needed from the use case point of view?  And he kind of stayed at me and went on with it  for another 15 minutes.  When he finished, I said, please, I beg for your politeness  here, but can you explain to me where concurrency is?  He kind of scratched and said, how do I explain this?  And then he thought for a minute and he looked at me and said,  damn, are you telling me we don't need concurrency?  And I said, can you help me see otherwise?  And he kind of laughed for a minute and said,  you just got rid of 60% of the code from our application.  And he looked at me and said, thanks for coming.  We don't need you to be here.  We know how to remove the code now.  And I kind of left a little heart broken  because I didn't have a solution that I can propose to them.  I just removed the problem from their hands.  And I see this quite often.  This is just one example of what I saw.  But we are eager and we create architecture.  And what do we do in the architecture?  You got to solve concurrency problem.  And the architecture involves a solution to a problem that  may not even exist.  So I've been working with companies for the past several  years.  And if you ask me what I've been doing actively for the past five  years, I go to companies.  And when I leave, their code has a weight loss in place.  So they tend to remove code because I fear complexity.  I ask them questions.  Why do you do this?  Why do you do this?  Why do you need this?  And we end up finding we actually don't need a lot of stuff.  And so the program becomes a lot simpler  because we're not doing stuff we don't need to do.  So one way to make things simple is to recognize what are we  really solving?  This brings back another experience in my mind.  I was working on a project.  And one of these guys had written a piece of code.  And literally, this is what I saw.  There was 200 lines of code to take data from the database  and XML being created.  And then I look at this XML being created.  And that is called into another function.  And this XML is being passed.  And I then go to this function.  And this XML is being reparsed.  And the data is being sent into the database.  And I'm scratching my head.  Why did we go from database to XML to XML to the database?  So I went to the developer and said, hey, I'm looking at this code.  You're going from data to XML, XML to data.  And he's like, isn't that cool?  I'm like, no.  Why are you doing this?  And he said, Venkat, you need to have a vision.  I said, I failed to have that vision.  Why would you do this?  And he said, look at how extensible this code is.  I said, I'm sorry.  I don't see extensibility.  Tell me about it.  He said, the data could come from anywhere in the world.  And the data could go to anywhere in the world.  And I said, according to what we are doing right now,  where does the data come from, from that table?  Where does it go to here?  Then why are we solving this problem?  He said, Venkat, you really need to understand extensibility.  We need to design this in a way it will be so extensible.  So while he was saying this, I kind of took my phone  and was dialing.  He said, excuse me.  And he took his phone and said, hello.  I said, hello.  He said, Venkat, is that you?  I said, yes.  And he said, why are you calling me?  I said, how extensible this is?  Why am I talking to you in person?  I could be anywhere in the world talking to you, right?  And the point is this.  What are we really solving?  And if we don't want to solve the real problems,  we could be building complex things around us.  And who is going to maintain all that code  when we build complexity in place?  So first thing is, keep it simple.  And how do you keep it simple?  The one way to keep it simple is really  to have this vision where you remove things that  are not really related and then get  to the very core of the problem, what am I really solving?  Now, once we do that, we can then build on on top of those things  to provide the solution.  Refactor willfully.  Now, I don't know of a way to create a good software  in one setting.  I really don't know that.  There is no way, in my opinion, to actually do this.  So I'm a huge fan of the slogan.  And the slogan is, make it work and make it better real soon.  So if I make it work, if I make it do what I want it to do,  as soon as it works, but not before it, as soon as it works,  what are the little feature, little component, little thing  we try to work, little test even.  As soon as it works, we want to turn around  and make it better real soon.  And if we can make it work better real soon,  then we are refactoring, making adjustments.  And if you really think about it,  if you read about a few interesting things,  there was this fine gentleman, there  is this fine gentleman called Frederick Brooks.  And Frederick Brooks wrote some fantastic articles and books  back in the 70s and early 80s.  And his words are golden.  They still stay true.  And one of the things that Frederick Brooks said  is build a software and throw it away.  Now you have a way to build the real software.  And I'm going to really emphasize,  I am paraphrasing this or something like this.  So this is not a direct quote.  I'm just paraphrasing that.  So he said that, go ahead and build a software.  And when you are done, throw it away and start again.  And you have a chance of building the right software,  a real software, whatever that is.  But the point really is this.  Is he wrong?  Well, he is absolutely right.  But there's one problem.  You don't go to work tomorrow and say, or Monday and say,  boss, I exactly figured this out.  We're going to build a software.  When it's done, we're going to throw it away.  And we're going to start over.  Don't do that and get fired and call me back and say,  you said this, right?  I've won you that.  So that's not practical, right?  But does it mean that Frederick Brooks is wrong?  If it's not practical.  Well, what if we write small amount of software  and throw it away almost continuously?  And then we rebuild that part.  This is where refactoring can help.  This is what test-driven development is something I like.  This is where I like a little active prototyping  experiments so that there are a number of things we can do  to build in small pieces and throw it away and rebuild it.  So refactoring is a very valuable tool.  And if we can refactor along the way, not huge refactoring,  but small amount of refactoring.  And this can help us a great deal.  In fact, we can learn quite a bit  from a lot of other industries and other experiences.  If you look at masterpieces of paintings,  and if you look at people who have done wonderful job  in painting, and you look at a masterpiece and say, wow,  what a great painting by this particular artist of the past.  Well, you will notice one thing if you  read a little bit deeper.  Those masterpieces that we look at,  they were not done as the masterpiece in one sitting.  Often, they built this for a particular king  or somebody who is rich, and they wanted something  to be painted or built.  And what they did was they created  several different prototypes.  And they created not just one or two, sometimes dozens  of prototypes they created.  And each of the prototype starts out  by showing one feature versus another.  And then there are things that they quite didn't like  in those prototype.  And they changed it in the second prototype,  and the third one, and the fourth one.  And then there is a prototype that is a lot closer  to that masterpiece we all admire, maybe with one feature  that is different.  So even the brightest of the people in the world  never created things in one sitting.  They created things by creating prototypes.  And here we come along as programmers, and we sit down,  and we expect to write code once.  And all of a sudden, it should be done.  What a foolish way of thinking about it.  So it is very important for us to be able to evolve this  that becomes extremely important.  The other feature to think about is reversibility.  And to me, this feature is something that is very critical  because if you forget reversibility,  it can be very expensive to evolve software.  Now what does reversibility really mean?  Reversibility means how can I back out of a certain design  decision that I make?  So if I made a decision, and if I have to be stuck  with the decision, then I should think a lot more  about the decision than a decision  where I can easily change.  Now let's think of a few decisions  that are not as reversible.  A language that you use is a fairly hard decision  you cannot reverse out of because the code,  as we know it right now, is created  in a certain programming language.  About a year into the project, you come along and say,  yeah, we don't like this language, we're gonna switch over,  that is expensive to make the change.  So languages are, for example, very hard to reverse.  But the question is, are there things we can  make easier to reverse?  Now just because something are easier to reverse  doesn't mean they actually are easier to reverse.  So we gotta make sure that the things that we targeted  as easy to reverse really are easy to reverse.  I'll give you an example of this.  I was working on a system about seven months ago,  and I was going to put this in production.  It was about two months of development effort,  give or take, it was a fairly rapid development  to get into the market.  And one of the things I said to myself is,  I wanna focus on the features right now.  I wanna postpone any single decision that I can postpone.  I don't wanna deal with it today, I wanna postpone it.  So one of the decisions I postponed on the project  is the database I can use in deployment.  Now you may call me crazy,  because that may have a major impact,  but I purposefully said,  I am not gonna worry about the database I'm gonna use  in production.  And so I ignored that and I went on to develop the application,  absolutely, I'm about two weeks away  from going into production.  And my little brain is telling me,  Venkat, you better pay attention right now.  And this is the last responsible moment,  I cannot ignore this any further.  So two weeks before going into production,  I decide to go look into what database  I could use in production.  By now, I got a better understanding of  which cloud I'm gonna run this on,  what kind of environment they have available to me,  what are the options, so I'm looking at all of these,  and I figured out what database I'm gonna actually use.  So I got a copy of the database, installed on my machine,  modified my code to go use that other database  using a configuration setting, and I run all my tests,  and to my fear, two, actually three tests actually failed.  And immediately what strikes you is a panic,  uh-oh, wait a minute, three tests are broken, what is it?  And I look into these three tests,  and of course, Google for the error you get,  within two minutes I find out what the problem is,  there was a data type compatibility  between the database I was using  and the database I'm gonna use moving forward.  So I read a little bit about it,  I make a change to the data model in the code,  rerun the code, all the tests are passing,  flip back to the old database I was using,  all the tests are still passing,  and you get a feedback within seconds  that everything is actually working still after this change.  And so in this particular example,  I postponed the decision of what database I'm gonna use.  Now the application has been in production  for about seven months, it is running on the database  that I did not develop with,  it's running on the database that I switched over to  about two weeks before, within two weeks before deployment,  and you can see that that's working just fine.  So the point really is, what is the benefit of doing this?  The benefit of doing this is,  it gave me the ability to focus on  the real essential features in the beginning,  and I designed it in a way that I can flip over  the database I wanna use with minor change.  So the cost of changing the database two weeks before  was installing it, running all the tests,  seeing some of them fail, learn about it,  and fix them a total of about 10 minutes of effort.  And, but that is not the way it's worked all the time.  I got some scars on my body  because of bad decisions that have been done in the past.  I worked in a company where we used a certain database.  I will not mention the name of the database here.  And, oh, every single class, every single class  in the application, and there were about 500 of them,  had to inherit from a common base class  that was provided by this database vendor.  And there were several classes in our system  that were coupled to the data structure  provided by this database.  And we didn't know better.  I was young back then.  I needed the money, so I worked on this project.  And I would also defensively say,  I didn't create the initial code.  I inherited it from other people,  but I should have known better, right?  Because when you come to work on a project,  these are smells you should know.  I know it today, but I didn't know it back then.  You start kind of working on the code that's given to you.  And one day, the database vendor decided to make  a very significant upgrade on the database.  And this was a breaking change on their part.  Just that one change with very little feature really  ended up costing the company I worked for  a good eight months of recoding.  And eight months of recoding the application  just to switch the database version.  And a few years later, I was looking through this,  and I was asking myself, what really went wrong?  And what went wrong in that case is very tight coupling.  So if you couple very tightly to something,  it is very hard to reverse out of it.  If you loosely couple to something,  it is easier to back out.  But oftentimes we would convince ourselves  that a database is something you have to deeply couple with.  And the answer is no, we don't.  And if such a deep coupling is prompted,  that is a smell we should bring out and talk about  and say, we don't wanna do that.  What are the alternatives we can consider  to remove that kind of tight coupling?  Because changing the database that we rely upon  is something we have to do potentially.  We got asked this question,  how likely is it for us to have to change this database?  I was just talking to a client two days ago,  and I was talking to them and they said,  we got tons of stored procedures  with a particular relational database.  And I asked them, what would you do  if you have to switch the database?  They just laughed and said, that will never have to happen.  We are married to this database,  and if we have to change it,  pretty much we are dead in the water, right?  So that is a very irreversible decision.  But my question really is this,  did they make this decision consciously  or did they make this decision accidentally?  So what if we can create an architectural diagram,  an architectural document, a very lightweight one,  and highlight reversible versus irreversible,  and start putting things on it as,  what is reversible and what is irreversible?  And when you start putting things very explicitly on it,  it brings the team together to say,  do you really want that to be irreversible?  Do you want this really to be reversible?  Do we understand the cost of this irreversibility  and consequences?  Do we understand the cost of this reversibility  and the consequences?  So we need to be able to, there's a cost for both,  and we need to ask ourselves,  are we making the right decision  on what is reversible and irreversible?  I don't want to find out what is irreversible later on  as, oh, this is how it turned out to be.  I want this to be a conscious decision  we have architected towards reversibility.  So I would like to see two sections  in the lightweight document, reversible items,  irreversible items, what are those and why they are such,  and what are the consequences and costs of doing it.  That can help us to decide whether we want reversibility  or irreversibility in our application in various areas.  So postponing things to the last responsible moment.  So how do we know what's the last responsible moment  and why should we really do this?  This really ties in with another thing  called the Yagney feature.  Now what is Yagney?  Yagney is for you aren't going to need it.  Lot of times we build things that we really don't need  with the perception that how could we live without it?  We really have to build it, of course, right?  So we tend to really build things  that we really don't need.  And if we can ask ourselves,  do we really need this feature?  Do we really need this particular component?  We could ask ourselves to do that.  And last responsible moment is to really find a time  when we no longer can postpone this particular decision.  And in my example, the last responsible moment  to pick the database was when I have committed  to my deployment.  I didn't know where I'm going to deploy yet.  I was considering multiple different cloud services.  And I wanted to wait until I make the decision  before I decided the database  I wanted to use.  In my case, that was the last or some other moment  for that specific decision on the database.  But what language did I use?  That was selected for early, early on.  And how did I choose the language?  When did I choose the language?  I didn't choose the language before I started the project.  But the very first day I sat down on the project,  I asked myself a few questions.  I said I can comfortably program  in about eight different languages.  So here are the pool of languages available on my hand.  I'm willing to learn a language if I have to do.  That's perfectly fine as well.  But what are the things I wanted to consider?  I wanted to consider languages  that will not only solve the problem for me,  but I also wanted to make sure the set of tools I selected  made testability extremely easy.  I wanted a complete round trip of testability,  both integration, functional and unit test.  And so I looked for a solution that will help me  with that kind of testability.  That was a decision I made into it.  So these are the decisions I made.  Of course, that had to be made fairly early on.  The last responsible moment for that decision  was the very first day of the project.  But selecting a database could be done two weeks  before in my case.  And there were decisions that were postponed along the way.  And having this willingness to postpone decisions,  this is very different from procrastination.  Procrastination is where we postpone things  beyond the last responsible moment.  So this is really knowing what is important now  and what can wait until a later time.  That is something for us to decide.  And postponing things to a later time  gives us a certain advantages.  What are the advantages?  First of all, if the cost of doing something now  is the same as cost of doing something later,  you would rather do it later than now  because you can argue you are smarter tomorrow  than you are today.  This goes back to the picture we started with.  And the picture is, you know more in this time frame  than you do here.  So if it can wait until here, you  rather make it wait than do it up here  because you have more information as you move forward.  If the cost of developing it is going to be higher now  compared to later, now why is it going to be higher now?  Because a lot of features don't exist.  But this will fit in really well when some other code has  been written.  Then I would rather wait to do it later than now  and not do it right now.  But if the cost is reversed, if it's more expensive to do it  later than now, I would try to find out, A, why it is more  expensive later, and B, what is the certainty to which I  would definitely need this?  And if the certainty is very high and the cost is high,  I would do it now rather than doing it later.  If the certainty is very low, I would rather postpone it  and take the cost of it because there's a probability  I wouldn't need it, then I would not  have to spend money on it at all.  So in other words, one of the things  we need to think about in developing software  is the economic impact of it.  What is the economic impact of what we do?  And unfortunately, we cannot develop  software in isolation of economic knowledge.  We need to definitely do that.  But there's another holy thing we often get across.  What about extensibility?  We all want our code to be extensible.  Who doesn't want it?  Absolutely.  But here's the problem of extensibility.  We often think our code has to be extensible,  but what we want it to be extended  is not what we actually face eventually.  Now, let's think about this for a minute.  There are three kinds of people, kinds of people,  on a project.  The first kind knows a lot about the domain.  So there are people on your team who  know a lot about the domain.  You can go ask them any question about the domain.  They can answer it for you.  They are part of your development team.  They are available.  The second type of people knows a lot about software  development.  These guys rock.  They can sit there and write code all day,  but don't know anything about the domain, right?  Or not ask much.  The third kind of people who are very few in number  knows a lot about both domain and software development.  I will politely ignore the fourth category here  that we don't want to talk about, right?  So the point really here is the people  who can make the right set of decision  are the people in the third category who  understand the domain and understand  the software development practices.  Because these people have a well-rounded knowledge  to look at both.  But I'm sorry, that is foreign, few in general.  So what can we do instead?  What we can do instead is we can have these two set of people  collaborate with each other.  And this is something very important to think about.  So when I used to work in areas that I had no knowledge  at all in the domain, that happens a lot,  because I'm a software developer by practice.  I'm a software developer by education.  I'm a software developer by what I do every day.  I love coding.  That's what I like.  But I get hired into projects where  I don't know anything about the domain.  It could be a financial industry.  It could be chemical industry.  It could be an industry that's focused on medicine.  I don't know.  People call me and say, can you come and work with us?  There are two things I normally do.  I try to get some education on the domain  as much as my little brain can hold.  I would read up some very important material.  I would come to terms with vocabulary.  I would shamelessly ask questions.  What does that mean?  But I will also immediately find the domain expert  and say, I'm designing the software for you.  I'm going to ask you a lot of stupid questions.  And I want you to be very kind and polite and answer  these questions for me.  And I would go to them and say, I'm designing this.  I'm wondering if I should worry about these kinds of data  also.  Hence, they will usually give me three kinds of answers.  Sometimes they will say, oh, of course,  that we have to worry about too.  We just didn't mention it, but absolutely,  we have to worry about it.  And then I'll ask them, do we have to worry about it now?  Is it something we'll worry about later on?  What are the probability I would ask the kind of questions?  Or they would laugh and say, no, we definitely  don't want to do that.  This kind of problem will never have that kind of data  or things to work with.  But a third answer they would give is, we are not sure.  We've got to go ask other people and find out.  But that is my job as a designer is  to ask these extensibility questions.  Are we going to face these problems?  But here's the thing.  If you're going to plan for extensibility,  that requires the knowledge of the domain and the application.  If we ignore these two and build extensibility,  what's going to happen is we make it  extensible for things that it won't be really meaningful.  I'll give you an example of this.  I was invited to a company.  I was sitting down talking to them.  And we had group people sitting and having lunch.  And the entire team was there along with the manager.  And the manager asked me, hey Venkat,  I got a question for you.  Every single day, my team said they are building  an extensible product.  And every single day, they guaranteed me,  this product is extensible.  And we went into production.  And two weeks after we went into production,  a customer called us and suggested a change.  And they said the product is not  usable without this change for their situation.  And I looked at the request.  And it was very clear how simple and more really  integral this change was.  So I promised to my company, we'll get this back quickly.  Because my team has created extensible code.  I came to my team and said, can you make it extensible?  And the team said, they need a few weeks to make the change.  And then so he said, I don't understand this.  Where things went wrong?  I said that to the manager, it is your fault.  And the manager said, what's my fault?  What did I do wrong?  And I said, if you're not as them for the change,  the product will still be extensible.  So the point is extensibility is what a product has  until you ask it to be extended.  And the minute you say extended,  it is not going to handle it.  So where things go wrong?  Well, Ralph Johnson said this in a wonderful way.  He said that we build hierarchies, interfaces,  all of these we build.  Why do we build?  We build it for the sake of extensibility.  But in doing so, we build enough complexity into the product  that it becomes really hard to extend it  when the time comes to extend it.  So one of the ways to make things extensible  is to keep it minimal and very lean  and solve the problem we know today.  And when we really see a real problem tomorrow,  we should be able to make it extensible based on that  rather than building a hierarchy of things  when we don't really need that kind of extensibility.  That becomes very important.  So one of the ways to do that is to use  what is called triangulation.  Ken Beck talks about triangulation.  And I think this is a wonderful approach.  Now you talk to people and they will tell you  duplication is bad, don't do it.  And one of those guys too, I say this often.  However, there is one way to accommodate this.  Remember to make it work and make it better real soon.  And this is something we can use here.  So when you know a particular feature, just implement it.  So we have one implementation of a feature.  It is working.  We have the code in place.  You come across yet another particular feature.  And it's got a little variation from what you have,  but a lot of similarities.  Well, go ahead and implement that.  And that may have a little bit of duplication at this point.  That's perfectly okay.  Make it work.  And now that you have both of those working,  now is your refactoring opportunity  to pull the commonality out into a common base.  So extract the commonality.  Extract the common.  There are two ways to extract the common.  One is to use inheritance to pull it up.  And the other way is to use delegation and pull it to the site.  And you can make the decision which makes more sense,  depending on the context.  But you wait until this evolves in front of you  to make this decision.  But once you have done this,  now the code is ready to handle  a third type of situation that arises,  which is similar to this.  So you cannot just magically make a code extensible.  You have to really know what extensibility is  in the given context and make that affordable.  Otherwise, we end up creating something overly complex  and we don't get extensibility  and we get a lot of code we have to deal with.  So it is actually a double JPD.  What about reuse?  Well, reuse is the holy grail in our field.  You go to people and say,  what's good about opting to programming?  Oh, we can make everything reusable.  Grady Butch said this really nicely.  He said, don't make code reusable.  First, make it usable.  Now, this is one thing you see quite often.  You go to programmers and they say,  why should I create code when I can create frameworks?  So everyone wants to create framework.  I was actually on a project I was reviewing their code  and I noticed something really peculiar.  I was looking at interfaces and I love interfaces.  I said, these guys are good.  They got interfaces.  I looked around and there was reflection being used.  I said to myself, I love these guys.  I love reflection.  They have used it.  I looked around a little bit further  and I saw what they were doing and they had factories.  I mean, who doesn't love factories, right?  I said, this is awesome.  These guys have used factory.  Then I went through and reviewed one more time  and I found something very, very strange.  They had interfaces and there were two kinds of interfaces  in their system.  The first kind of interface,  they only had one class implementing it  and there were a few interfaces  with no class at all implementing it.  So I was a little puzzled.  I went back to the team and said,  help me out with this.  How come you have interfaces that only have one classes?  And they said, oh, we know what happened.  The guys who wrote it left, that's the first thing they said  and those guys wanted to build a framework  so they created all of these things  and that quite didn't materialize.  So when they left, we have all these interfaces hanging around  with no real implementations for the second type.  So this urge of creating something very reusable  is in all of us and we have to avoid the temptation.  We got to create something that is usable.  I was working with a company where they decided  that several of their products had duplication in it  and one of the higher up guys said,  who was pretty technical said,  that said, I want to create a reusable component.  I don't want this to be done in every product.  It took 12 years, I'm not kidding,  12 years for them to extract the reusable component from it  and the reason is sure everything is doing the same  but when you get into the details,  you quickly realize it's not really the same  and that's where it gets very tricky.  Now, if you look at successful reusable framework,  not a single one of them were created by a committee sitting  in a room and as a words of Alexander, he said,  designed by committee creates a horse  when they are designing a camel, right?  So the point really is you cannot just walk into a room  and say, I want to create something reusable.  A better way to create reusability is to use it  and use it and use it and eventually realize  this is something reusable and start refactoring  and extracting the component,  then you create something very, very reusable.  If you look at some of the most popular,  truly reusable framework,  they all went through that path of creation  and anything that's really sucked  is what was created in a design room  and then they said this is reusable because we told you so  and the last thing I want to mention here is  about libraries and frameworks.  I have to honestly tell you, I fear this  and the reason I fear is twofold.  Libraries and frameworks are the shiny toys  for us as programmers.  This is like taking kids into a candy store  and what do kids say?  I want this, this, this, this and this  and I want more of it, right?  And we programmers are like that.  You do a quick web search,  I want that, that and that and that, right?  I was talking to a team recently and they said,  oh, we used to run all these tests.  I said, you're talking a past tense.  Why are you using a past tense?  Why can't you run those tests?  And they said because we upgraded some of the plugins  on our JavaScript side and all these tests quit working.  And I asked them the question I shouldn't have asked.  I said, how many plugins do you have?  And they all went quiet because it was not in dozens.  It was a lot more than in dozens.  They had like a hundred framework they were using.  And I asked them, what does your product do?  And they said, we are still figuring it out,  but we have 150 frameworks we are using, right?  So why is this infatuation to use frameworks  and tools and libraries?  The more you use, the more you have to maintain,  the more you have to struggle,  think about the simplicity they talked about  in the beginning.  There is another problem.  I'll share with you another experience,  which was very bitter.  This was back in time when the economy was really bad  and I was consulting for a company.  And they were interviewing a candidate  who was extremely good and he had built a framework.  So the gentleman who was the hiring authority came to me  and said, hey, we're gonna hire this person.  What do you think?  I said, I love it.  This person is the best person we have ever interviewed.  Hire him.  He's actually really good.  I really wanna learn a few things from him.  So please don't talk to me.  Go and hire him now.  We can't lose the time.  And then he said, what do you think  of the framework he's been developing?  I said, that's an awesome framework.  It's lovely.  And then he has the third question.  What do you think of us using the framework?  I said, oh no, don't do that.  He said, Venkat, you need to have a vision.  Do you notice he's got distributed persistence?  I said, you haven't talked about distributed persistence  to me, not a single day.  How did that become your feature right now?  He said, look, it can do distributed messaging.  I said, you didn't even mention the word messaging to me.  Where did that come from?  Please focus on building the application.  Release it in the timeframe we have for the six months.  Then you do whatever you wanna do.  I don't care.  So hire the guy.  Don't commit to the framework.  So we clearly comes over, sits on my table and says,  Venkat, I got a news for you.  I remember this very clear Wednesday afternoon.  He said, I got a news for you.  I said, what's up?  We have hired this person.  I leaned back on my chair and said,  I cannot tell you how happy I am to hear this news.  I'm really excited.  This is gonna be fun.  This guy is awesome.  I'm gonna learn a lot from him as well.  We're gonna just rock.  This is great.  And then he said, well, as part of the hiring contract,  we have agreed to use this framework.  I put my head on the table for a second.  I said, you know what?  I'm sorry, but I quit.  I'm resigning.  He said, oh, you are so predictable.  I said, yes, that's good.  And I said, do me a favor.  If you have any project, please call me.  I wanna come and work with you,  but not on this project anymore.  And I picked up my stuff and left.  I went home.  This was one of those few days I actually go home on time.  I knocked on the door, walked in.  My wife said, you are early, way too early.  I said, because I resigned.  And my wife looked at me and said, really, did you?  Well, you can come and help me with the dishes.  That's how she is.  So I was jobless pretty much at home, but it felt right.  And about six months later,  they were supposed to finish the project.  Well, I got a call 18 months later.  18 months later, I got a call.  I said, hello.  And he said, go ahead and say it.  I said, I'm sorry to say what?  Well, I told you so.  I said, I'm sorry to hear.  I don't wanna tell you I told you so, but what happened?  Well, in the 18 months,  they put a lot of money and time building the framework.  Because the framework they never knew about  that they decided to use did almost everything they wanted,  but not quiet.  And the product has not been released in 18 months.  And I said, why are you calling?  He said, can you come and help us?  I said, there's a reason I ran away.  Right? I'm sorry, I cannot.  Well, that project never got done.  And it was a major failure.  And it's very disheartening to see that.  There's a name for this.  It is called RDD.  And I have seen this in many companies, not one.  That is called resume-driven development.  People want to use technology  so their curriculum vitae or resume can look really shiny.  Hey, bring all these.  In fact, when you hire people,  if their resume has 20 frameworks or libraries listed,  you know it's a bad sign, right?  So that is a tendency to say,  I wanna use all the shiny tools,  but let's ask ourselves,  what is the problem we are trying to really solve?  So to summarize, if you wanna create  extensible, evolutionary architecture,  we have to consider some of these factors.  We got to keep things simple.  We got to refactor the code.  Think about reversibility, but identify them very clearly.  What is reversible?  What is not reversible?  Why? What are the consequences?  And think about postponing things  that you can actually postpone.  And think about extensibility,  not in terms of how can I arbitrarily make things extensible,  but more meaningfully based on the problem domain  that you are trying to work with.  And then use a triangulation  and extract the reusable components from it,  or extensible component from it.  Reusability is something that comes from active use  and not by upfront design.  And also think about what minimum set of libraries  and frameworks you can end up using,  and that can help us to create  evolutionary architecture along the way.  I really hope you found that useful.  Thank you very much.  Thank you.
Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_0_6659.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_6660_6707.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_6708_7019.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_7020_7235.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_7236_7403.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_7404_7967.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_7968_9371.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_9372_10919.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_10920_11279.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_11280_13139.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_13140_13403.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_13404_13835.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_13836_14795.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_14796_15011.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_15012_15143.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_15144_16451.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_16452_16535.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_16536_17195.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_17196_17651.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_17652_20459.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_20460_21995.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_21996_22043.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_22044_22187.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_22188_22295.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_22296_22343.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_22344_22403.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_22404_23387.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_23388_23879.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_23880_24035.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_24036_24251.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_24252_24311.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_24312_25331.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_25332_25427.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_25428_25475.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_25476_29795.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_29796_36479.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_36480_36527.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_36528_36599.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_36600_36659.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_36660_37295.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_37296_38267.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_38268_38339.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_38340_38387.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_38388_38519.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_38520_38567.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_38568_40259.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_40260_44795.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_44796_52295.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_52296_59795.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_59796_61475.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_61476_61535.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_61536_62735.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_62736_63419.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_63420_64115.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64116_64163.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64164_64223.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64224_64343.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64344_64415.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64416_64739.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64740_64787.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64788_64835.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_64836_65123.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_65124_65171.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_65172_65975.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_65976_67295.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_67296_72323.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_72324_72755.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_72756_72839.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_72840_72887.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_72888_72935.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_72936_73115.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_73116_73487.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_73488_73907.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_73908_74795.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_74796_74879.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_74880_82295.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_82296_86987.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_86988_87035.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_87036_87659.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_87660_88043.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_88044_88091.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_88092_88607.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_88608_89363.png Venkat_Subramaniam_-_Towards_an_Evolutionary_Architecture_89364_89447.png