Professional Software Development
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Martin, Robert Cecil
We’ve come a long way in the last 20 years. We start our journey in the late 80s and our "discovery" of design principles such as The Open Closed Principle and the Liskov Substitution Principle. In the middle 90s, we discovered that these principle led to repeating patterns of design. We gave those patterns names such as Visitor and Decorator. At the turn of the millennium we found that the benefits gained from the principles and patterns could be amplified by combining them with practices such as Test Driven Development and Continuous Integration. And now, as the decade is coming to a close, we have found that these principles, patterns, and practices have driven us to define a true profession. What will that profession require of us, and who among us can truly claim to be professional?
 Hello, hello. Oh, I think I've got sound now. Good morning. Okay, fine. So, it's a piece of green candy. My laser. Can you see the green sparkles in there? This light doesn't look smooth. It looks grainy. Those of you who have young eyes could try to defocus those sparkles, change the focal plane of your eyes. You will note that you cannot defocus the sparkles. Those of you with glasses on, remove your glasses. You will find that the sparkles are still in focus. The reason for that is that these sparkles are not here in the candy. They are on the back of your retina. That's an interference pattern. The light coming from the laser is coherent. All one color, all in the same phase. So the wave fronts interfere with each other, creating this nice sparkly effect. If you move your head back and forth, you will note that the sparkles move with your head. This is because your eyes are moving through the interference pattern. Isn't that cool? Our discussion today is about professionalism in software development. This is a talk I've given many times before, although there are bits of it that change from time to time. The purpose of the talk is to impress upon all of us the need for professional behavior. Our industry doesn't really know what it means to behave professionally. The definition of a software developer is someone who sits in a room, you throw meat in, and code comes out. No one really understands how or why that happens. We get very confused about deadlines and dates and estimates and all of the things that we're supposed to be doing, and we do them rather badly. Now that's not unusual. Our industry is young. How many of you have been programmers for more than ten years? Look at that. We've got quite a crowd here. More than twenty. Quite a few of you, I'd say about five percent. More than thirty. Okay, we've got maybe half a dozen. The next question is a much harder one. More than forty leaves me. In most professions, you would find that there would be far more hands raised in the older crowd. Oh, there'd be plenty of hands raised in the younger crowd as well, but if you went to a convention of plumbers, the number of people who had been plumbers for thirty years would still be rather large. If you went to a convention of hardware engineers, electronics engineers, the number of people who would be over thirty years would still be fairly large. But in our industry, we see this cut-off at about twenty years. Now that's growing, and if we came back in another ten years, we'd find that the number of people who were thirty-year veterans at programming would have gotten larger. There is a force in our industry that discourages older programmers. The idea is that you should have moved into management after ten years or so, or even after five years. If you're not a manager after five or six or seven years, something's wrong with you. And that's probably true. At least there's certainly something wrong with me. I have tried to be a manager a number of times. I have managed teams, I've managed groups, I've managed divisions, I've managed whole companies, but I'm still a programmer. It's what I want to do. It's what I like to do. It's what I'm happiest doing when I'm writing code, or if I'm on stage yelling at a bunch of programmers, that makes me happy too. And I happen to enjoy doing my videos, which is just yelling at a bunch of programmers through an indirect means. The idea that someone would remain a programmer for decades is rather foreign to our industry, but it's something that we're going to have to get used to, because there are a lot of us who would just rather stay programmers for a very long time and enjoy the process. And those folks will gather a great deal of experience and will help us define what professionalism is. The slide says that our craft is defined. Ten years ago, I would not have been able to make that statement. Ten years ago, we did not know exactly what it was that a programmer did, other than write code. But we couldn't define on a day-by-day basis what the process was, what the set of disciplines was, what the professional behavior was. Now we can. And that's because of something that happened about ten years ago. How many of you are now doing agile development? Now, this is a remarkable number. That's probably over half of you, maybe over two-thirds of you. Most of you are lying without knowing it. But that doesn't matter. The notion that most of you believe you are doing agile development is fascinating. Agile development is a set of practices. It began, oh gosh, it began a very long time ago, but in earnest it began around 1998 to 1999, when Kent Beck published his rather controversial book, Extreme Programming Explained. Did anybody read that book? It caused a firestorm. The industry at the time was immersed in waterfall. Who's still doing waterfall? Look at that, right? Hardly anybody, a couple of unfortunate people raised their hands without realizing that they're not supposed to do that. Hardly anybody does waterfall anymore, but ten years ago, twelve years ago, that was what everybody was doing. That's what everybody thought they were supposed to do. We have managed to turn the industry around in ten years, 180 degrees. That's a fantastic result. And in the midst of all of that, we defined a group of disciplines and behaviors, which I'm going to outline here, that define to a large degree what it means to be a professional software developer. But professionalism goes beyond simply following disciplines. There's an entire ethical topic involved here as well. What does it mean to be an ethical programmer? How do you know if you are being right? How do you know if you are defending your profession? That's an interesting topic as well, which we're going to discuss here along with the disciplines. I wear this green band. I've had this band or a band like it on my hand now for six or seven years. I don't know exactly when it was. The woman who put it on me is named Elizabeth Hendrickson. She just walked up to me and handed me a green band and said, I hear wear this, and at the time the band said Test Obsessed, which is kind of her trademark. And it was a marketing gimmick for her company. I put it on and I found that it took on a meaning, an ethical meaning. I couldn't take it off, not because it was glued on, but because there was an ethical impact to having this band on my hand. The band said, I will follow the disciplines that I know. I will behave professionally. I will not do those things that I know are wrong. Even though I'm put under pressure by managers and other people who have influence over me, I will not do those things I know I should not do. A remarkable effect that it had on me. If you want a band like this, you can get one at that URL. I don't take money for them. I do ask that you make a donation to a charity. I figure that if you're going to put something like this on, you should pay for it. But don't pay me. Give the money to some worthy cause and then I will send you a band. One of the obvious disciplines that we learned, starting around 1999 and why we didn't learn it before that I don't know, is that things are best done in short iterations. Now everybody here is doing agile, almost everybody. So what is your iteration size? Who is doing four weeks? So a few of you are still following the old-fashioned scrum thing that said four weeks. Almost nobody does this anymore because it was found to be too long. How many of you are doing three weeks? A few more. Two. Look at that. One. Okay, I think we found the norm of the curve. The norm of the curve seems to be around two. But if we had asked that question five years ago, it would have been closer to three. If I had asked that question eight years ago, it would have been closer to four. The mean of that number is moving to the small side. Come back in five years, there will be far more of you doing one-week iterations. I know that seems impossible, but iteration sizes are shrinking every year. People have found that four weeks is too long. Too much can go wrong. You can get way off the rails in four weeks, and then it takes too long to recover. Three weeks is too long. Two weeks, okay. All right, we can do two weeks. One week. Not a lot can go wrong in one week. You can't make a big mess in one week. And if you close the feedback loop once a week, you can stay pretty close to the rails, not Ruby rails. Rails. You can stay pretty close to your goal. You can do quick course corrections. Now, of course, everybody says, yeah, but holy cow, if we're doing one-week iterations, all we'll ever be doing is really planning and retrospecting and demoing, and we'll never write any code. No. All those meetings, the planning meetings, retrospective meetings, and the demos, they all get shorter too. You do a week iteration. The planning meeting for a week doesn't take more than an hour. The demo, well, you're not going to get a lot done in a week, so the demo's not going to take more than an hour. Retrospective, what can you talk about after a week? Yeah. So, everything shortens up, and you get very nice cycle time. So, I'd like you to think about that. Short iterations are a moral issue. The shorter we make them, the more control we have over our process. The more accuracy we can get in our process. The longer we make them, the more can go wrong, but more importantly, the longer you can hide. If your iteration is a month long, you can hide for three and a half weeks. Three and a half weeks can go by, and no one will know that you've been surfing the web, doing some day trading that you're just behind. No one will know that you're depending on someone else to finish the job for you. Has anybody ever worked at a company where you knew there were programmers who simply weren't doing anything, or worse, doing stuff and you wished they wouldn't? Stop writing that code, because you're going to have to come in on Saturday and fix the mess they've made. Has anybody been the guy who comes in on Saturday and fixes the mess that everybody else made during the week? Okay, nobody wants to raise their hands on that either. I consulted for a company like that. They had about 40 guys working on code. Five of them knew what they were doing. The other 35 made horrible messes during the week, and then those five would come in on Saturday and Sunday, and they'd make everything work. One day, the manager, this is several years into this project, the manager finally said, this project's going too slowly. We need everybody working on Saturdays. And then nothing got done. Short iterations go beyond just the iterations that we see in Agile. We do everything in short iterations. We write our code in tiny little iterations. We will write our code in periods that last perhaps 30 seconds if you're following a test-driven development cycle, something I'll be talking about a little bit later. We do everything. We can in tiny little iterations because, frankly, we're not good at doing anything. So the more we check it, the more we close that feedback loop, the better we are at keeping on course. Has anybody ever had this lovely dream that you would be given a specification? You would take that specification and study it and internalize it, and then in a burst of insight and great creativity, the code would pour out of your fingers, and it would be perfect. You would hit Control S for one and only one time to save that source file. You wouldn't even bother to compile it because you knew it would work. You'd check it into source code control, and everyone else would stare in awe at your mastery. Huh! That person does not exist. No such person has ever been able to do that. The best way to get anything done is to iterate in tiny little cycles. By the way, we're taught this from about first grade on. Who remembers the teacher who first told them that in order to write a story, you need to first write a rough draft? This might have been in second grade or possibly third grade, and the teacher says this, and I remember the teacher saying that to me, and my thought in my head was, what? I'm not doing that. What a waste. I'm just going to write the damn story. And I did. I just wrote the damn story, and in second grade, no one could tell. But you get into high school, you get into the upper grades, and all of a sudden the teacher starts kicking it back at you, saying, what's this crap? They don't use those terms. They actually just use numbers. You know what the numbers mean. There's some numbers that mean really good, and there's some numbers that mean crap. And you start getting these bad numbers, and you think, oh, what am I doing wrong? And maybe you go to the teacher and ask, what are you doing wrong? And the teacher looks at you and says, well, did you do a rough draft? Well, no. Well, that's what you did wrong. And so then you start doing the rough draft, and then you do another draft, and you read it over. That's a little thing we learned after a while. You write something, it's a really good idea to read it. So you read it over, and then as you read it, you realize, oh, this is awful, and you make some changes to it, and you do another draft. And as you get into college, that gets worse and worse. So you're doing draft after draft after draft. You're handing it to other people. Would you read this for me, please? Because I don't know if I'm doing any good. You get this feedback, and changes and changes, and you finally hand it in, and people are going, oh, man, you did a very good job on that. And you realize that it requires this kind of constant iteration and feedback to do anything creative. Then you get your job as a programmer, and the rules completely change. You're expected to write it correctly the first time. You must now abandon all of that stuff that you learned about creative writing and iteration and rough drafts, and then you just have to write that code the first time. And we are taught, not taught, we are made to feel that any change to existing code is rework, and it's evil, and it's wasteful. It's not. It's part of the natural process. You cannot write the code correctly the first time. You have to iterate on it. Does anybody write their code and then read it? It's very depressing. You write the code, you actually get it working, then you read it and go, oh, my God, I wrote that. Some of you will have that feeling. You'll go away to lunch. You'll come back and there's this code on the screen and you look at the code on the screen and say, I didn't write that. That's not my code. And then eventually you learn, OK. Another practice, never be blocked. This is an ethical issue. We sometimes get caught in this notion that this is my job, that's their job. And I'm waiting for something they're supposed to do. That means I can't do anything because it's their job and I'm not going to do it. It's their job. It doesn't work that way. You can try to make it work that way, but that is a huge pathology, a disease of dysfunctional organizations. If you're waiting for something because some poor guy couldn't get it to you, do it. Go do it. Find some way not to be blocked. Or if you don't want to do it, find some way to isolate yourself from it so that you don't have to wait. Do not be blocked. The worst thing you can do as a professional is nothing. What you can do is to sit and go, well, I can't work right now because Bob over there hasn't finished what he's supposed to do. Or that group over there, they haven't delivered their part of the bargain. They haven't delivered their module to us. So we can't work. I once was consulting for a team. The team had a set of QA people, three QA people who were supposed to be testing the system. And they were just sitting there. I said, you're not testing the system. They said, no, our QA system is not working at the moment and the computer is not working, the server is not working. And we've called the people who are supposed to fix it, but they're busy so we can't work. And I looked around the room and there were all these laptops sitting on desks because all the developers were at lunch. And I said, well, why don't you use one of those laptops? Oh, no. We can't use those laptops because they're not bonafide production environments. We can only do our QA tests on real production environments. I said, programmers are testing their stuff all the time on those machines. Why don't you open up one of those machines and run some tests? If you find a bug, you can write it down. Oh, no. We have to wait. I was amazed at that attitude. That is an attitude that accepts blockage as though it were normal. Being blocked is immoral. It is wrong. It is unprofessional. It is irresponsible. It is something that you as a creative engineer ought to find some solution to. When you open up the top level directory of your application, what do you see? Do you see directories that are named models, views, and controllers? Do you see top level directories that tell you that you are in a Rails framework or a Spring framework or a.NET framework? If you were to look at that top level directory, what would it tell you about the application that you were writing? Would it tell you what that application did or would it tell you what technology that application was using? I'm going to be doing a talk a little bit later tomorrow, I believe, called clean architecture. The idea behind our architectures is that a good architecture tells you what it's for, not what it's made out of. Many of us, if asked, what is the architecture of your system, we would say something like, oh, yeah, it's Spring, hibernate, Tomcat, with a lot of dependency injection. Or we might say, oh, well, it's based on.NET with an ADO back end, and we're doing a lot of link, and we got this view model thing that we're doing. Yeah, but that's not an architecture. That's a set of tools. The frameworks you're using, the technology you're using are a bunch of tools. They're not an architecture. An architecture tells you what the application is for. If I were to show you the blueprints of this building, you would look at those blueprints and you'd say, well, that's some kind of sports arena or convention center or something. You could see where all the bleachers were. You could see that big open area out there. You could see the area around the outside. And you would go, ah, that's some kind of arena. You would not say, hmm, looks like they used hammers and saws and bricks. You would know exactly what that building was for. The architecture of your system should scream at you. I am an accounting application. I am a trading application. I am a shipping application. The architecture should yell at you what it's for. First time you open up those directories, you should look at that and go, ah, yeah, this is some kind of accounting application. Look, there's transactions. There's ledgers. There's credits. There's dabbits. This is an accounting application. You shouldn't have to hunt for that. What should you have to hunt for? You should have to hunt for the fact that it's delivered over the web. The initial architecture should not say, ah, I am a web application. That's a detail. The fact that it's a web application is some nasty little detail. I don't want to know that it's a web application. You can tell me about that later. I'll hunt for how the damn system is delivered. I don't want to know web, web, web, web. I want to know that it's an accounting system. I don't want to know what database you're using. I don't want to know that there is a database. The database is a detail. Get it out of my way. I want to know what this system does. Do you know what the goal of a good architecture is? Other than to announce to the world the purpose of the application, the goal of a good architecture, the goal of a good architect is to delay as many decisions as possible. Differ decisions about frameworks and databases, web servers, and all those frameworks you're going to use. Differ those decisions. Get them out of the way. Make them late. The later you make a decision, the more information you have about how good that decision is. One of the big mistakes we make as software developers is we make a whole raft of decisions far too early. We say, oh gosh, we've got to have a database. What database? Well, this is.NET. We don't have a lot of choice, do we? Well, we'll use one. And we've got to have a framework. And we've got to have a language. And we've got to have all these things. And we make all those decisions, and then we start to write the code. Now you've got to pick your language early. But other than that, you want to be able to write your business rules early and defer everything else for as long as possible. I'll have more to say about that later in my clean architecture talk. Anybody here got a mess? Bunch of code that you think is crap? Look around the room here. This is a large fraction of the room. Why is it a large fraction of the room? Because we've been writing a lot of crap. Why? Why have we been writing so much crap? Well, there's a reason. And it's not just down to carelessness. And it's not just down to rushing. Oh, rushing is certainly part of it. We get into the mode where we've got a deadline, and we have to get this thing working, and we just forget all about the rules, and we spit this code out on the screen, and we think it works, and we hand it to QA, and we run home. And we've made a mess. But then later, something more interesting happens. Later we come back, and we open up a module on our screen, and we look at that module, and we say, ah, that's awful. I should clean it. And this, for a moment, for a moment in your mind, you think, I'm going to clean this code. But then something else takes over. And the next thought in your mind is, I'm not touching it. Because if I touch it, I'll break it. And if I break it, it will become mine. Walk away from the code. And so we allow the code to sit there, dirty, rotten. Our opportunity to clean it is gone. And we allow the entire system to rot. I will talk a little bit later today in this talk about a means to get around that problem. However, how do you solve it? Has anybody here ever told their boss, when the boss has come to them and said, how come it takes so long for you guys to do anything? And you say, well, the code is a mess. The boss says, well, what are we going to do about it? And you see your opportunity. We could redesign it. Let's just rewrite it. If we rewrite it, we can do it clean and neat. We can throw away all this old crap. Let's just rewrite the whole thing. This is the great seduction. Everybody wants to rewrite the application. Everybody wants to walk away from the mess they made, leave it in the room where it's messy, close the door on that room, and never open up the room again. That's what we like to do. This is incredibly irresponsible. If you did that in your house, eventually you'd be out of rooms. All right? But don't go in that room. There's another reason why we don't do this. This is horrifically expensive. And what we have found is that these big redesigns just create another messy room. They don't actually clean anything up. You spend an immense amount of time and effort to get to the exact same place you were a year ago. What we need to do is face the fact that we've made messes and clean them. But you cannot clean them all in one great shot. You cannot clean things up with one great cleaning project. Do not, please do not dedicate a month of your time to nothing but refactoring. All right? Instead, you just every day look at some code and you clean it up a little bit. You do some random act of kindness to the code. If you follow a basic rule, always check the code in a little cleaner than when you checked it out. Always do some random act of kindness. Check out a module. Do whatever you have to do in that module. Then make sure it's just a little bit cleaner than when you checked it out. Check it back in in a slightly cleaner state. If everybody did just that, the code base would just get gradually better and better and better. I'll talk a little later about means to do that. But for now, no grand redesigns. Resist the temptation to redesign everything. The project is almost certainly going to fail. Oh, not a month long. When a month long will probably succeed. But a year long redesign, man years of effort into a redesign, probably going to fail. And it's fundamentally a flawed technique. It's irresponsible. Clean the damn mess up. You made it. Clean it up. And keep your code clean. Write clean code. Clean the code that you've written. Emphasize in your mind that it is unethical to write code that is messy. Learn the simple rules of clean code and then follow them because in the end, the only way to get things done in a hurry is to do them right. Your grandparents taught you this. Your parents taught you this. It's an old saw. Everybody knows it. Anything worth doing is worth doing well. It is certainly true. And although we are tempted to rush and make a mess to meet some stupid deadline, we will actually meet that deadline better if we keep everything clean and do the best job we can. Has anybody here been slowed down by a big batch of messy code? That's everybody. What if the code were always clean? We could go faster. The lines would be shorter. We'd actually have fewer programmers. How many of you are practicing test-driven development? Hands up, test-driven developers. So this looks like about a third of you. Maybe a quarter of you. What's the matter with the rest of you? This is a discipline where the jury is in and was in five years ago, where the technique is well-known to be very effective, that you can produce much better code by following it, much better performance. You get a lot of benefits out of test-driven development. So why wouldn't you practice this? Now let me describe it for you, just in case you weren't aware of what this is. Test-driven development is composed of three laws. The three laws sound stupid. Anybody who hears them goes, you can't possibly mean that. So here's the first law. You are not allowed to write any production code until you have first written a unit test that fails. Now right away, that sounds dumb. I mean, what test are you supposed to write? There's no code to test. So how can you write a test if there's no code to test? There's a whole bunch of arguments that you could go on about this. It just sounds stupid to write a test first. But the second law is worse. The second law says you're not allowed to write more of a test than is sufficient to fail and not compiling is failing. Which means that you've got to start by writing a test, but you can't write more than a line or two. Because instantly you're going to call some function that doesn't exist or mention some class that doesn't exist. So you have to stop writing the test before you're even begun writing it almost. And you have to start writing production code. And then the third law kicks in. The third law says you're not allowed to write more production code than is sufficient to pass the currently failing test. Which means that you write a little line of test code and oh, that doesn't compile. And then you write a line of production code and oh, that makes it compile. And then you have to go back to the test. Add another line to the test. Oh, that doesn't pass. You write some more production code. Oh, that makes it pass. And you're going back and forth between these two streams of code every 30 seconds. If you're a programmer of some years and most of us in the room are, that just sounds stupid. I mean, why would you do that? Why would you interrupt your flow of thought by bouncing back and forth between these two streams of tests? Why would anybody do that? It would be boring. It would be tedious. It would be slow. You'd never be able to write a whole algorithm without bouncing back and forth. You'd never be able to focus on anything. It's all good points. All true. But imagine a group of people following these three laws. Pick one of them. It doesn't matter who. It doesn't matter when. Everything they were working on executed and passed all its tests a minute or so ago. And it doesn't matter who you pick and it doesn't matter when you pick them. Everything they were working on executed and passed its tests a minute or so ago. Maybe two, maybe five. But some very short time ago, everything worked. What would your life be like if you were never more than a minute or so away from seeing everything work? How much debugging do you think you would do? The answer to that is, well, there's not a lot to debug if it worked a minute ago. If it all worked a minute ago. And now it doesn't, it's something in the last two lines of code that I wrote. The most common debugging technique is to hit control Z enough times to back away and then just retype what you wrote. Because typically you just typed it wrong. Does that mean I don't use a debugger? No, I use a debugger. Every once in a while there are bugs that escape. It's just still software. It's still hard. I still use a debugger. I don't live in the debugger. How many of you are really good at the debugger? You know all the hotkeys, right? You are debugging God. You know how to do a step into and a step over. You can set 20 break points here. Wait for that one to get to it 10 times. Then you get to this one three times. Now you watch that variable. Wait for it to be a 37. And now you can debug. This is not a skill to be desired. You don't want to be good at the debugger. If you are good at the debugger, it means you spend a lot of time debugging. I don't want you spending a lot of time debugging. The debugger should not be your tool. It should be something that you use every once in a while because you know you screwed up. Not because it's, oh, I'm going to debug this now and I'm really good at debugging. That's a shame. I'm sorry you're good at debugging. I believe that if you were to practice those three laws, you could cut your debugging time by half, maybe more. And maybe you could think, well, that's a good reason to practice those three laws, or maybe not. Maybe that's not enough. So never mind that. How many of you have integrated a third party package? So you can everybody, everybody. OK, so you get this zip file from somebody and you unpack the zip file and there's a PDF in there along with all of the DLLs and stuff that you need. The PDF has got a nice little manual that describes the third party package. At the end of the PDF, there's a little appendix area with all the code examples. Where's the first place you go? You go to the code examples. Why? Because your programmers, you don't want to read the manual that some tech writer wrote. You want to go to the code. The code will tell you the truth. You'll read the code and go, oh, I see how this damn thing works. And if you're lucky, you can copy and paste the code out of the manual and put it in your application and fiddle with it to get it to work. When you are writing unit tests, you are writing the code examples for the whole system. If you're following those three laws, every part of the system will have a unit test. And that unit test will be a short little snippet of code, an example for how that part of the system works. You want to know how to call some API function. There is a test that calls that API function every way it can be called. You want to know how to create some object or set up some relationship. There are tests that do that every way it can be done. And those tests are short, little unambiguous snippets of code. They are written in a language you understand. They are completely unambiguous. They are so formal they execute and they cannot get out of sync with the application. They are the perfect kind of documentation. The perfect kind of low-level documentation. If you follow those three laws, you get massive, comprehensive, thorough, low-level documentation that cannot get out of sync is utterly unambiguous. It's the perfect kind of documentation. If you like documentation, Test Driven Development will give it to you for free, or at least for the cost of writing those tests. But maybe that doesn't impress you, so never mind it. How many of you have written tests after the fact? How much fun is this? It's not a lot of fun, right? You spend a lot of time writing the code and that's the fun part. And then you have to test it manually and debug it. That's kind of the fun part, too. And then you get it all working and then there's this rule that some guy wrote and says, well, you've got to write tests now. Okay, test here and all right, it says stupid rules and write a test there. That part over there is going to be hard to test. It's all kind of coupled into the code. I know it works. I'm not going to write that. I got other things to do and so you leave holes in the test suite. Writing tests after the fact is something that you do almost under protest. And under protest, you leave holes in the test suite. You don't want to test everything for crying out loud. You've already done it manually. When you write the test first, something else happens. First of all, writing the test first is fun because you're into the problem. You're specifying the problem. You've set yourself a challenge past that test, you sucker. And now you will demonstrate your prowess as a programmer by making that test pass. I got it to pass. How many of you are programmers because you got something to run once and you want that feeling again? You get that feeling all the time. You write a test. I wonder if I can make that pass and then you write the thing that makes it. Oh, I got it to pass. All right, next test. That would be hard. I got it to pass. You get this lovely charge every time. So the enthusiasm does not wane. It's not like this. Oh, crazy. I got to write a bunch of tests. There's a level of enthusiasm that stays up. And because of that, the test suite tends to be complete. Moreover, you're writing the test first and that forces you to design the code so that it can be testable. Another word for testable code is decoupled code. So you are going to decouple massively simply to get your test done. You'll have a better design. Okay. Maybe that's not enough for you. Maybe you don't want to shorten your debug times and maybe you don't want to stream a perfect documentation and maybe you don't want a really good decoupled design because you just think it's stupid to write test first. But there's one other thing you get out of it. In fact, you can throw all that other stuff away. It's one other thing you get out of this. You get a suite of tests that you trust with your life. You get a suite of tests where you can click a button and see all those tests pass and you know the system works. And when you have that and then up on your screen comes some dirty code and you think, oh, I should clean it, you've got those tests. So you try. You make a small change. Maybe I can change that variable's name. And you run the tests. Oh, they still pass. Maybe I can take that function and split it into two functions. Run the tests. Oh, they still pass. Maybe I can take that class and split it up into two classes. Run the test. Oh, I don't know if I can walk, put it back. When you have those tests and when you trust them, you can clean the code. You will clean the code because suddenly the cost of changing code approaches zero. The risk goes way down. You've got a suite of tests that tells you it's okay. Don't worry. I'll just make a little change here. Oh, the tests still pass. This is so nice. Of course, the tests have to run quickly. It doesn't help you if the tests take five hours to run. So you need the tests to run like this. That's a design challenge, by the way. I pose it to you as a design challenge. Keep your tests running fast. Your designers, you can figure out a way to do that. Software is a sensitive discipline. And by that I mean that if I have a binary executable that's maybe a megabyte long, how many bits are in that? Eight million bits, small program. Eight million bits. How many of those bits could I flip from one to zero or zero to one and crash the system? Well, there is one in there that will do it. How many of those one bits are in there that will do that? Probably thousands. There are probably thousands of bits inside that eight million, which if I just flip them from one to zero, it will crash the system. Our systems are sensitive to failure at the bit level. Hardly any systems are that sensitive. We could walk around this building and find some bolts and some girders, take a few of them out. The building wouldn't fall down. We could go out onto the highway and start removing bolts off the bridges. The bridges would stay up there for a while. We could go out drinking tonight and kill 10,000 brain cells. Probably enjoy the process. Humans are not sensitive at the one cell level. Bridges are not sensitive at the one bolt level, but software is sensitive at the one bit level. The only way to create correct software is to make sure that all eight million of those bits is correct. That's hard to do. Most engineering disciplines don't have that problem. Most engineering disciplines live inside a robust world where there's room for error. We don't have that. What other discipline is similar to ours? Well, there is one. It's accounting. The accountants, they work with spreadsheets and there are digits on those spreadsheets, not all of them, but certain digits on those spreadsheets which if they get wrong will take the company down and send the executives to jail. How do accountants prevent that from happening? Sometimes they don't. But how do they try to prevent that from happening? They have a discipline. That discipline is called double entry bookkeeping. They say everything twice. They enter every transaction once on the debit side, once on the credit side, and those two transactions follow separate mathematical pathways until there's this lovely subtraction on the balance sheet that must yield a zero. If it doesn't yield a zero, something went wrong. If it does yield a zero, we hope nothing went wrong. It's possible that you could make complementary entries that gave you a false zero. That can happen. The odds are very unlikely. No judge would throw you in jail if you destroyed the company based on that kind of an error. When accountants enter their transactions, they enter it in the debit side, they enter it on the credit side, they get the zero on the balance sheet, then they enter the next ones. They invented this 500 years ago, probably after a king started cutting off some heads. Test-driven development is double entry bookkeeping. We say everything twice, once on the test side, once on the production code side. We say them at the same time, just like they enter the transactions at the same time. We do that for exactly the same reason. We have to get all our bits right. They've got to get all their digits right. We've got to get all our bits right. How can we as professionals treat our material, our code, with less respect than accountants treat theirs? Are they better than us at the moment they are? But should they be? Is their stuff, are their spreadsheets more important to the company than our code? Why do they have a discipline like that? Do we don't, or at least we don't follow it? Why is it that we, thumb our nose and say, ah, there's test-driven development stuff. Let me write the code. Can you imagine an accountant saying, ah, there's double entry bookkeeping stuff. Let me just enter the transactions. I'll enter them right. What a lot of crap. Do accountants have a QA department that they hand their spreadsheets to? Here, we're done with our spreadsheet's QA. Now see if you can find any problems with them. Why do we have QA departments? Why has our industry invented an entire department to make sure that we don't screw up? Why? Because we screwed up. And our companies are going, ah, well, what are we going to do about these developers? They're just pushing crap out to the door. And, well, let's create another group, QA. That'll make sure it doesn't happen. And then you know what happens. The developers say, ah, we got a QA department now. That means I can meet any deadline. Give me a deadline. I'll check the code in by that deadline. It'll have some bugs, but that's their problem. I worked for a company once where everything was always done on time. Right? The schedule was perfect. Everything was always done on time. Sometimes a programmer would achieve his goal by checking in a blank source file. There was nothing in it, but he checked it in. Okay, check that off the schedule, and then QA gets it. Hey, wait, there's no code here. Well, that's a bug. Put that on the schedule to fix it on the bug. That's not on the development side of the schedule. If you are practicing test-driven development or any kind of testing discipline, the goal is to cover the code with tests to make sure that you have tested every line of code you write. Does it make sense to have any lesser goal? For example, should you be satisfied with 60% code coverage? Let's say you've got a tool that measures code coverage, and it says, oh, 60% of your code is covered. Is that good? A lot of us would look at it and think that's great. But what it means is that you don't know if 40% of the code works. Is that acceptable? No. What if an accountant said, 40% of our transactions are double entry? Yeah, they're 60%, we're not so sure about. But 40% of them we're pretty sure are right. Isn't that good enough? No, of course it's not good enough. There is no meaningful goal short of 100%. If you're looking at code coverage, that's the only number to think of. How am I going to get this up to 100%? You never will get it up to 100%. That's okay. It's an asymptotic goal. You never fall for the silly line that says, you know, 70% is pretty damn good. No, it's not. It's awful. 70% is terrible. It means there's 30% of your code that you've got no idea about. This is the end result of QA. When you write a bunch of code, you hand it to QA, what should they find? Nothing. QA should find nothing. They should wonder why the heck they have a job. And that should be your goal. You should be looking over at those QA guys and going, you're not going to find anything. Next year you won't be here. You'd like to take QA and have them do a slightly different job, actually, because having them work at the back end is an awful thing to do to people. Imagine that this is your job. You're going to, has anybody seen what the QA test plan looks like? There it is. Anybody seen the QA test plan? That's not the QA test plan. That's the table of contents for the QA test plan. The guy holding it out to me is the QA manager of a large internet travel organization. He's got 80,000 manual tests that he farms out to India every six weeks. It costs him a million dollars every six weeks to run those manual tests. He's holding it out to me because he's just come back from a meeting with his boss who just came back from a meeting with the CFO. The CFO said, what the hell is this million dollars every six weeks that we're spending? He's talking to me now saying, I just got my budget cut by 50%. Which half of these tests should I not run? I told him, it doesn't matter. You can cut the document any one of a bunch of different ways. You're not going to know if half your system works. This is the end result of manual testing. Manual tests grow and grow because the features of the system grow and grow. At some point, the effort required to execute those manual tests will become so expensive that some financial officer will spot it and say that must be cut. You will lose your tests. It will happen. The only solution to that is to automate those tests. I don't want QA finding anything and I want all the tests that QA writes. I want QA writing the tests, by the way. I want all the tests that QA writes to be executed automatically. I don't want any manual tests in the system. Manual tests are fundamentally immoral. That doesn't mean that you can't have people sitting there operating the system. I like the idea of exploratory tests, but I don't want them given a script. These are the tests you must pass. Imagine that's your job. You've got to follow that script. Has anybody looked at what that's like? Enter P9-537 into the username field. Hit return. Enter QX723 into the password field. Click login. Did the welcome screen come up? This is your job. And you're going to repeat that every six weeks. One last. One of the things a professional learns how to do, and it's very, very difficult, is to say no. If your boss comes to you and says, we've got to have this by Friday, and you know it cannot be done by Friday, you must say no. You don't understand. We must have it by Friday. No. Now, you understand that if we don't have this by Friday, we're going to lose a lot of money. I understand that. So will you get it to us by Friday? No. Why not? Because it's not possible. Well, can you try? No. You must never say that you will try. What behavior are you going to change because you said you would try? How will your strategy now be altered because you said you would try? How are the odds of succeeding made different because you said you would try? The reason you said you would try was to make him go away. And what he heard was that you would do it. And what you meant was get out of my face by saying that you would try, you told a lie. And that lie will come back, of course, because you won't get it done by Friday. And then they'll say, well, you said you'd try. Well, yeah, I tried. Well, you didn't try enough. Thank you for your attention. I'll see you all at another talk. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you.
 Hello, hello.  Oh, I think I've got sound now.  Good morning.  Okay, fine.  So, it's a piece of green candy.  My laser.  Can you see the green sparkles in there?  This light doesn't look smooth.  It looks grainy.  Those of you who have young eyes could try to defocus those sparkles,  change the focal plane of your eyes.  You will note that you cannot defocus the sparkles.  Those of you with glasses on, remove your glasses.  You will find that the sparkles are still in focus.  The reason for that is that these sparkles are not here in the candy.  They are on the back of your retina.  That's an interference pattern.  The light coming from the laser is coherent.  All one color, all in the same phase.  So the wave fronts interfere with each other,  creating this nice sparkly effect.  If you move your head back and forth,  you will note that the sparkles move with your head.  This is because your eyes are moving through the interference pattern.  Isn't that cool?  Our discussion today is about professionalism in software development.  This is a talk I've given many times before,  although there are bits of it that change from time to time.  The purpose of the talk is to impress upon all of us  the need for professional behavior.  Our industry doesn't really know what it means to behave professionally.  The definition of a software developer is someone who sits in a room,  you throw meat in, and code comes out.  No one really understands how or why that happens.  We get very confused about deadlines and dates and estimates  and all of the things that we're supposed to be doing,  and we do them rather badly.  Now that's not unusual.  Our industry is young.  How many of you have been programmers for more than ten years?  Look at that.  We've got quite a crowd here.  More than twenty.  Quite a few of you, I'd say about five percent.  More than thirty.  Okay, we've got maybe half a dozen.  The next question is a much harder one.  More than forty leaves me.  In most professions, you would find that there would be far more hands raised in the older crowd.  Oh, there'd be plenty of hands raised in the younger crowd as well,  but if you went to a convention of plumbers,  the number of people who had been plumbers for thirty years would still be rather large.  If you went to a convention of hardware engineers, electronics engineers,  the number of people who would be over thirty years would still be fairly large.  But in our industry, we see this cut-off at about twenty years.  Now that's growing, and if we came back in another ten years,  we'd find that the number of people who were thirty-year veterans at programming would have gotten larger.  There is a force in our industry  that discourages older programmers.  The idea is that you should have moved into management after ten years or so,  or even after five years.  If you're not a manager after five or six or seven years,  something's wrong with you.  And that's probably true.  At least there's certainly something wrong with me.  I have tried to be a manager a number of times.  I have managed teams, I've managed groups, I've managed divisions,  I've managed whole companies, but I'm still a programmer.  It's what I want to do.  It's what I like to do.  It's what I'm happiest doing when I'm writing code,  or if I'm on stage yelling at a bunch of programmers, that makes me happy too.  And I happen to enjoy doing my videos,  which is just yelling at a bunch of programmers through an indirect means.  The idea that someone would remain a programmer for decades  is rather foreign to our industry,  but it's something that we're going to have to get used to,  because there are a lot of us who would just rather stay programmers  for a very long time and enjoy the process.  And those folks will gather a great deal of experience  and will help us define what professionalism is.  The slide says that our craft is defined.  Ten years ago, I would not have been able to make that statement.  Ten years ago, we did not know exactly what it was that a programmer did,  other than write code.  But we couldn't define on a day-by-day basis what the process was,  what the set of disciplines was, what the professional behavior was.  Now we can.  And that's because of something that happened about ten years ago.  How many of you are now doing agile development?  Now, this is a remarkable number.  That's probably over half of you, maybe over two-thirds of you.  Most of you are lying without knowing it.  But that doesn't matter.  The notion that most of you believe you are doing agile development is fascinating.  Agile development is a set of practices.  It began, oh gosh, it began a very long time ago,  but in earnest it began around 1998 to 1999,  when Kent Beck published his rather controversial book,  Extreme Programming Explained.  Did anybody read that book?  It caused a firestorm.  The industry at the time was immersed in waterfall.  Who's still doing waterfall?  Look at that, right?  Hardly anybody, a couple of unfortunate people raised their hands without realizing  that they're not supposed to do that.  Hardly anybody does waterfall anymore,  but ten years ago, twelve years ago, that was what everybody was doing.  That's what everybody thought they were supposed to do.  We have managed to turn the industry around in ten years, 180 degrees.  That's a fantastic result.  And in the midst of all of that, we defined a group of disciplines and behaviors,  which I'm going to outline here,  that define to a large degree what it means to be a professional software developer.  But professionalism goes beyond simply following disciplines.  There's an entire ethical topic involved here as well.  What does it mean to be an ethical programmer?  How do you know if you are being right?  How do you know if you are defending your profession?  That's an interesting topic as well,  which we're going to discuss here along with the disciplines.  I wear this green band.  I've had this band or a band like it on my hand now for six or seven years.  I don't know exactly when it was.  The woman who put it on me is named Elizabeth Hendrickson.  She just walked up to me and handed me a green band and said,  I hear wear this, and at the time the band said Test Obsessed, which is kind of her trademark.  And it was a marketing gimmick for her company.  I put it on and I found that it took on a meaning, an ethical meaning.  I couldn't take it off, not because it was glued on,  but because there was an ethical impact to having this band on my hand.  The band said, I will follow the disciplines that I know.  I will behave professionally.  I will not do those things that I know are wrong.  Even though I'm put under pressure by managers and other people who have influence over me,  I will not do those things I know I should not do.  A remarkable effect that it had on me.  If you want a band like this, you can get one at that URL.  I don't take money for them.  I do ask that you make a donation to a charity.  I figure that if you're going to put something like this on, you should pay for it.  But don't pay me.  Give the money to some worthy cause and then I will send you a band.  One of the obvious disciplines that we learned,  starting around 1999 and why we didn't learn it before that I don't know,  is that things are best done in short iterations.  Now everybody here is doing agile, almost everybody.  So what is your iteration size?  Who is doing four weeks?  So a few of you are still following the old-fashioned scrum thing that said four weeks.  Almost nobody does this anymore because it was found to be too long.  How many of you are doing three weeks?  A few more.  Two.  Look at that.  One.  Okay, I think we found the norm of the curve.  The norm of the curve seems to be around two.  But if we had asked that question five years ago, it would have been closer to three.  If I had asked that question eight years ago, it would have been closer to four.  The mean of that number is moving to the small side.  Come back in five years, there will be far more of you doing one-week iterations.  I know that seems impossible, but iteration sizes are shrinking every year.  People have found that four weeks is too long.  Too much can go wrong.  You can get way off the rails in four weeks, and then it takes too long to recover.  Three weeks is too long.  Two weeks, okay.  All right, we can do two weeks.  One week.  Not a lot can go wrong in one week.  You can't make a big mess in one week.  And if you close the feedback loop once a week, you can stay pretty close to the rails,  not Ruby rails.  Rails.  You can stay pretty close to your goal.  You can do quick course corrections.  Now, of course, everybody says, yeah, but holy cow, if we're doing one-week iterations,  all we'll ever be doing is really planning and retrospecting and demoing,  and we'll never write any code.  No.  All those meetings, the planning meetings, retrospective meetings, and the demos,  they all get shorter too.  You do a week iteration.  The planning meeting for a week doesn't take more than an hour.  The demo, well, you're not going to get a lot done in a week,  so the demo's not going to take more than an hour.  Retrospective, what can you talk about after a week?  Yeah.  So, everything shortens up, and you get very nice cycle time.  So, I'd like you to think about that.  Short iterations are a moral issue.  The shorter we make them, the more control we have over our process.  The more accuracy we can get in our process.  The longer we make them, the more can go wrong, but more importantly,  the longer you can hide.  If your iteration is a month long, you can hide for three and a half weeks.  Three and a half weeks can go by, and no one will know that you've been surfing the web,  doing some day trading that you're just behind.  No one will know that you're depending on someone else to finish the job for you.  Has anybody ever worked at a company where you knew there were programmers  who simply weren't doing anything, or worse, doing stuff and you wished they wouldn't?  Stop writing that code, because you're going to have to come in on Saturday  and fix the mess they've made.  Has anybody been the guy who comes in on Saturday and fixes the mess  that everybody else made during the week?  Okay, nobody wants to raise their hands on that either.  I consulted for a company like that.  They had about 40 guys working on code.  Five of them knew what they were doing.  The other 35 made horrible messes during the week,  and then those five would come in on Saturday and Sunday, and they'd make everything work.  One day, the manager, this is several years into this project,  the manager finally said, this project's going too slowly.  We need everybody working on Saturdays.  And then nothing got done.  Short iterations go beyond just the iterations that we see in Agile.  We do everything in short iterations.  We write our code in tiny little iterations.  We will write our code in periods that last perhaps 30 seconds  if you're following a test-driven development cycle,  something I'll be talking about a little bit later.  We do everything.  We can in tiny little iterations because, frankly, we're not good at doing anything.  So the more we check it, the more we close that feedback loop,  the better we are at keeping on course.  Has anybody ever had this lovely dream that you would be given a specification?  You would take that specification and study it and internalize it,  and then in a burst of insight and great creativity,  the code would pour out of your fingers, and it would be perfect.  You would hit Control S for one and only one time to save that source file.  You wouldn't even bother to compile it because you knew it would work.  You'd check it into source code control,  and everyone else would stare in awe at your mastery.  Huh!  That person does not exist.  No such person has ever been able to do that.  The best way to get anything done is to iterate in tiny little cycles.  By the way, we're taught this from about first grade on.  Who remembers the teacher who first told them that in order to write a story,  you need to first write a rough draft?  This might have been in second grade or possibly third grade,  and the teacher says this, and I remember the teacher saying that to me,  and my thought in my head was, what?  I'm not doing that. What a waste.  I'm just going to write the damn story.  And I did. I just wrote the damn story, and in second grade, no one could tell.  But you get into high school, you get into the upper grades,  and all of a sudden the teacher starts kicking it back at you, saying,  what's this crap? They don't use those terms.  They actually just use numbers.  You know what the numbers mean.  There's some numbers that mean really good, and there's some numbers that mean crap.  And you start getting these bad numbers, and you think, oh, what am I doing wrong?  And maybe you go to the teacher and ask, what are you doing wrong?  And the teacher looks at you and says, well, did you do a rough draft?  Well, no. Well, that's what you did wrong.  And so then you start doing the rough draft, and then you do another draft,  and you read it over. That's a little thing we learned after a while.  You write something, it's a really good idea to read it.  So you read it over, and then as you read it, you realize, oh, this is awful,  and you make some changes to it, and you do another draft.  And as you get into college, that gets worse and worse.  So you're doing draft after draft after draft.  You're handing it to other people. Would you read this for me, please?  Because I don't know if I'm doing any good.  You get this feedback, and changes and changes, and you finally hand it in,  and people are going, oh, man, you did a very good job on that.  And you realize that it requires this kind of constant iteration and feedback  to do anything creative.  Then you get your job as a programmer, and the rules completely change.  You're expected to write it correctly the first time.  You must now abandon all of that stuff that you learned about creative writing  and iteration and rough drafts, and then you just have to write that code  the first time. And we are taught, not taught, we are made to feel  that any change to existing code is rework, and it's evil, and it's wasteful.  It's not. It's part of the natural process.  You cannot write the code correctly the first time.  You have to iterate on it.  Does anybody write their code and then read it?  It's very depressing.  You write the code, you actually get it working, then you read it and go,  oh, my God, I wrote that. Some of you will have that feeling.  You'll go away to lunch. You'll come back and there's this code on the screen  and you look at the code on the screen and say, I didn't write that.  That's not my code.  And then eventually you learn, OK.  Another practice, never be blocked.  This is an ethical issue.  We sometimes get caught in this notion that this is my job, that's their job.  And I'm waiting for something they're supposed to do.  That means I can't do anything because it's their job and I'm not going to do it.  It's their job.  It doesn't work that way.  You can try to make it work that way, but that is a huge pathology,  a disease of dysfunctional organizations.  If you're waiting for something because some poor guy couldn't get it to you, do it.  Go do it.  Find some way not to be blocked.  Or if you don't want to do it, find some way to isolate yourself from it  so that you don't have to wait.  Do not be blocked.  The worst thing you can do as a professional is nothing.  What you can do is to sit and go, well, I can't work right now because Bob over there  hasn't finished what he's supposed to do.  Or that group over there, they haven't delivered their part of the bargain.  They haven't delivered their module to us.  So we can't work.  I once was consulting for a team.  The team had a set of QA people, three QA people who were supposed to be testing the  system.  And they were just sitting there.  I said, you're not testing the system.  They said, no, our QA system is not working at the moment and the computer is not working,  the server is not working.  And we've called the people who are supposed to fix it, but they're busy so we can't work.  And I looked around the room and there were all these laptops sitting on desks because  all the developers were at lunch.  And I said, well, why don't you use one of those laptops?  Oh, no.  We can't use those laptops because they're not bonafide production environments.  We can only do our QA tests on real production environments.  I said, programmers are testing their stuff all the time on those machines.  Why don't you open up one of those machines and run some tests?  If you find a bug, you can write it down.  Oh, no.  We have to wait.  I was amazed at that attitude.  That is an attitude that accepts blockage as though it were normal.  Being blocked is immoral.  It is wrong.  It is unprofessional.  It is irresponsible.  It is something that you as a creative engineer ought to find some solution to.  When you open up the top level directory of your application, what do you see?  Do you see directories that are named models, views, and controllers?  Do you see top level directories that tell you that you are in a Rails framework or a  Spring framework or a.NET framework?  If you were to look at that top level directory, what would it tell you about the application  that you were writing?  Would it tell you what that application did or would it tell you what technology that  application was using?  I'm going to be doing a talk a little bit later tomorrow, I believe, called clean architecture.  The idea behind our architectures is that a good architecture tells you what it's for,  not what it's made out of.  Many of us, if asked, what is the architecture of your system, we would say something like,  oh, yeah, it's Spring, hibernate, Tomcat, with a lot of dependency injection.  Or we might say, oh, well, it's based on.NET with an ADO back end, and we're doing a lot  of link, and we got this view model thing that we're doing.  Yeah, but that's not an architecture.  That's a set of tools.  The frameworks you're using, the technology you're using are a bunch of tools.  They're not an architecture.  An architecture tells you what the application is for.  If I were to show you the blueprints of this building, you would look at those blueprints  and you'd say, well, that's some kind of sports arena or convention center or something.  You could see where all the bleachers were.  You could see that big open area out there.  You could see the area around the outside.  And you would go, ah, that's some kind of arena.  You would not say, hmm, looks like they used hammers and saws and bricks.  You would know exactly what that building was for.  The architecture of your system should scream at you.  I am an accounting application.  I am a trading application.  I am a shipping application.  The architecture should yell at you what it's for.  First time you open up those directories, you should look at that and go, ah, yeah,  this is some kind of accounting application.  Look, there's transactions.  There's ledgers.  There's credits.  There's dabbits.  This is an accounting application.  You shouldn't have to hunt for that.  What should you have to hunt for?  You should have to hunt for the fact that it's delivered over the web.  The initial architecture should not say, ah, I am a web application.  That's a detail.  The fact that it's a web application is some nasty little detail.  I don't want to know that it's a web application.  You can tell me about that later.  I'll hunt for how the damn system is delivered.  I don't want to know web, web, web, web.  I want to know that it's an accounting system.  I don't want to know what database you're using.  I don't want to know that there is a database.  The database is a detail.  Get it out of my way.  I want to know what this system does.  Do you know what the goal of a good architecture is?  Other than to announce to the world the purpose of the application, the goal of a good architecture,  the goal of a good architect is to delay as many decisions as possible.  Differ decisions about frameworks and databases, web servers, and all those frameworks you're  going to use.  Differ those decisions.  Get them out of the way.  Make them late.  The later you make a decision, the more information you have about how good that decision is.  One of the big mistakes we make as software developers is we make a whole raft of decisions  far too early.  We say, oh gosh, we've got to have a database.  What database?  Well, this is.NET.  We don't have a lot of choice, do we?  Well, we'll use one.  And we've got to have a framework.  And we've got to have a language.  And we've got to have all these things.  And we make all those decisions, and then we start to write the code.  Now you've got to pick your language early.  But other than that, you want to be able to write your business rules early and defer  everything else for as long as possible.  I'll have more to say about that later in my clean architecture talk.  Anybody here got a mess?  Bunch of code that you think is crap?  Look around the room here.  This is a large fraction of the room.  Why is it a large fraction of the room?  Because we've been writing a lot of crap.  Why?  Why have we been writing so much crap?  Well, there's a reason.  And it's not just down to carelessness.  And it's not just down to rushing.  Oh, rushing is certainly part of it.  We get into the mode where we've got a deadline, and we have to get this thing working, and  we just forget all about the rules, and we spit this code out on the screen, and we  think it works, and we hand it to QA, and we run home.  And we've made a mess.  But then later, something more interesting happens.  Later we come back, and we open up a module on our screen, and we look at that module,  and we say, ah, that's awful.  I should clean it.  And this, for a moment, for a moment in your mind, you think, I'm going to clean this code.  But then something else takes over.  And the next thought in your mind is, I'm not touching it.  Because if I touch it, I'll break it.  And if I break it, it will become mine.  Walk away from the code.  And so we allow the code to sit there, dirty, rotten.  Our opportunity to clean it is gone.  And we allow the entire system to rot.  I will talk a little bit later today in this talk about a means to get around that problem.  However, how do you solve it?  Has anybody here ever told their boss, when the boss has come to them and said, how come  it takes so long for you guys to do anything?  And you say, well, the code is a mess.  The boss says, well, what are we going to do about it?  And you see your opportunity.  We could redesign it.  Let's just rewrite it.  If we rewrite it, we can do it clean and neat.  We can throw away all this old crap.  Let's just rewrite the whole thing.  This is the great seduction.  Everybody wants to rewrite the application.  Everybody wants to walk away from the mess they made, leave it in the room where it's  messy, close the door on that room, and never open up the room again.  That's what we like to do.  This is incredibly irresponsible.  If you did that in your house, eventually you'd be out of rooms.  All right?  But don't go in that room.  There's another reason why we don't do this.  This is horrifically expensive.  And what we have found is that these big redesigns just create another messy room.  They don't actually clean anything up.  You spend an immense amount of time and effort to get to the exact same place you were a  year ago.  What we need to do is face the fact that we've made messes and clean them.  But you cannot clean them all in one great shot.  You cannot clean things up with one great cleaning project.  Do not, please do not dedicate a month of your time to nothing but refactoring.  All right?  Instead, you just every day look at some code and you clean it up a little bit.  You do some random act of kindness to the code.  If you follow a basic rule, always check the code in a little cleaner than when you checked  it out.  Always do some random act of kindness.  Check out a module.  Do whatever you have to do in that module.  Then make sure it's just a little bit cleaner than when you checked it out.  Check it back in in a slightly cleaner state.  If everybody did just that, the code base would just get gradually better and better  and better.  I'll talk a little later about means to do that.  But for now, no grand redesigns.  Resist the temptation to redesign everything.  The project is almost certainly going to fail.  Oh, not a month long.  When a month long will probably succeed.  But a year long redesign, man years of effort into a redesign, probably going to fail.  And it's fundamentally a flawed technique.  It's irresponsible.  Clean the damn mess up.  You made it.  Clean it up.  And keep your code clean.  Write clean code.  Clean the code that you've written.  Emphasize in your mind that it is unethical to write code that is messy.  Learn the simple rules of clean code and then follow them because in the end, the only way  to get things done in a hurry is to do them right.  Your grandparents taught you this.  Your parents taught you this.  It's an old saw.  Everybody knows it.  Anything worth doing is worth doing well.  It is certainly true.  And although we are tempted to rush and make a mess to meet some stupid deadline, we will  actually meet that deadline better if we keep everything clean and do the best job we can.  Has anybody here been slowed down by a big batch of messy code?  That's everybody.  What if the code were always clean?  We could go faster.  The lines would be shorter.  We'd actually have fewer programmers.  How many of you are practicing test-driven development?  Hands up, test-driven developers.  So this looks like about a third of you.  Maybe a quarter of you.  What's the matter with the rest of you?  This is a discipline where the jury is in and was in five years ago, where the technique  is well-known to be very effective, that you can produce much better code by following  it, much better performance.  You get a lot of benefits out of test-driven development.  So why wouldn't you practice this?  Now let me describe it for you, just in case you weren't aware of what this is.  Test-driven development is composed of three laws.  The three laws sound stupid.  Anybody who hears them goes, you can't possibly mean that.  So here's the first law.  You are not allowed to write any production code until you have first written a unit  test that fails.  Now right away, that sounds dumb.  I mean, what test are you supposed to write?  There's no code to test.  So how can you write a test if there's no code to test?  There's a whole bunch of arguments that you could go on about this.  It just sounds stupid to write a test first.  But the second law is worse.  The second law says you're not allowed to write more of a test than is sufficient to  fail and not compiling is failing.  Which means that you've got to start by writing a test, but you can't write more than a line  or two.  Because instantly you're going to call some function that doesn't exist or mention some  class that doesn't exist.  So you have to stop writing the test before you're even begun writing it almost.  And you have to start writing production code.  And then the third law kicks in.  The third law says you're not allowed to write more production code than is sufficient to  pass the currently failing test.  Which means that you write a little line of test code and oh, that doesn't compile.  And then you write a line of production code and oh, that makes it compile.  And then you have to go back to the test.  Add another line to the test.  Oh, that doesn't pass.  You write some more production code.  Oh, that makes it pass.  And you're going back and forth between these two streams of code every 30 seconds.  If you're a programmer of some years and most of us in the room are, that just sounds stupid.  I mean, why would you do that?  Why would you interrupt your flow of thought by bouncing back and forth between these two  streams of tests?  Why would anybody do that?  It would be boring.  It would be tedious.  It would be slow.  You'd never be able to write a whole algorithm without bouncing back and forth.  You'd never be able to focus on anything.  It's all good points.  All true.  But imagine a group of people following these three laws.  Pick one of them.  It doesn't matter who.  It doesn't matter when.  Everything they were working on executed and passed all its tests a minute or so ago.  And it doesn't matter who you pick and it doesn't matter when you pick them.  Everything they were working on executed and passed its tests a minute or so ago.  Maybe two, maybe five.  But some very short time ago, everything worked.  What would your life be like if you were never more than a minute or so away from seeing  everything work?  How much debugging do you think you would do?  The answer to that is, well, there's not a lot to debug if it worked a minute ago.  If it all worked a minute ago.  And now it doesn't, it's something in the last two lines of code that I wrote.  The most common debugging technique is to hit control Z enough times to back away and  then just retype what you wrote.  Because typically you just typed it wrong.  Does that mean I don't use a debugger?  No, I use a debugger.  Every once in a while there are bugs that escape.  It's just still software.  It's still hard.  I still use a debugger.  I don't live in the debugger.  How many of you are really good at the debugger?  You know all the hotkeys, right?  You are debugging God.  You know how to do a step into and a step over.  You can set 20 break points here.  Wait for that one to get to it 10 times.  Then you get to this one three times.  Now you watch that variable.  Wait for it to be a 37.  And now you can debug.  This is not a skill to be desired.  You don't want to be good at the debugger.  If you are good at the debugger, it means you spend a lot of time debugging.  I don't want you spending a lot of time debugging.  The debugger should not be your tool.  It should be something that you use every once in a while because you know you screwed  up.  Not because it's, oh, I'm going to debug this now and I'm really good at debugging.  That's a shame.  I'm sorry you're good at debugging.  I believe that if you were to practice those three laws, you could cut your debugging time  by half, maybe more.  And maybe you could think, well, that's a good reason to practice those three laws,  or maybe not.  Maybe that's not enough.  So never mind that.  How many of you have integrated a third party package?  So you can everybody, everybody.  OK, so you get this zip file from somebody and you unpack the zip file and there's a  PDF in there along with all of the DLLs and stuff that you need.  The PDF has got a nice little manual that describes the third party package.  At the end of the PDF, there's a little appendix area with all the code examples.  Where's the first place you go?  You go to the code examples.  Why?  Because your programmers, you don't want to read the manual that some tech writer wrote.  You want to go to the code.  The code will tell you the truth.  You'll read the code and go, oh, I see how this damn thing works.  And if you're lucky, you can copy and paste the code out of the manual and put it in your  application and fiddle with it to get it to work.  When you are writing unit tests, you are writing the code examples for the whole system.  If you're following those three laws, every part of the system will have a unit test.  And that unit test will be a short little snippet of code, an example for how that part  of the system works.  You want to know how to call some API function.  There is a test that calls that API function every way it can be called.  You want to know how to create some object or set up some relationship.  There are tests that do that every way it can be done.  And those tests are short, little unambiguous snippets of code.  They are written in a language you understand.  They are completely unambiguous.  They are so formal they execute and they cannot get out of sync with the application.  They are the perfect kind of documentation.  The perfect kind of low-level documentation.  If you follow those three laws, you get massive, comprehensive, thorough, low-level documentation  that cannot get out of sync is utterly unambiguous.  It's the perfect kind of documentation.  If you like documentation, Test Driven Development will give it to you for free, or at least for  the cost of writing those tests.  But maybe that doesn't impress you, so never mind it.  How many of you have written tests after the fact?  How much fun is this?  It's not a lot of fun, right?  You spend a lot of time writing the code and that's the fun part.  And then you have to test it manually and debug it.  That's kind of the fun part, too.  And then you get it all working and then there's this rule that some guy wrote and says, well,  you've got to write tests now.  Okay, test here and all right, it says stupid rules and write a test there.  That part over there is going to be hard to test.  It's all kind of coupled into the code.  I know it works.  I'm not going to write that.  I got other things to do and so you leave holes in the test suite.  Writing tests after the fact is something that you do almost under protest.  And under protest, you leave holes in the test suite.  You don't want to test everything for crying out loud.  You've already done it manually.  When you write the test first, something else happens.  First of all, writing the test first is fun because you're into the problem.  You're specifying the problem.  You've set yourself a challenge past that test, you sucker.  And now you will demonstrate your prowess as a programmer by making that test pass.  I got it to pass.  How many of you are programmers because you got something to run once and you want that  feeling again?  You get that feeling all the time.  You write a test.  I wonder if I can make that pass and then you write the thing that makes it.  Oh, I got it to pass.  All right, next test.  That would be hard.  I got it to pass.  You get this lovely charge every time.  So the enthusiasm does not wane.  It's not like this.  Oh, crazy.  I got to write a bunch of tests.  There's a level of enthusiasm that stays up.  And because of that, the test suite tends to be complete.  Moreover, you're writing the test first and that forces you to design the code so that  it can be testable.  Another word for testable code is decoupled code.  So you are going to decouple massively simply to get your test done.  You'll have a better design.  Okay.  Maybe that's not enough for you.  Maybe you don't want to shorten your debug times and maybe you don't want to stream a  perfect documentation and maybe you don't want a really good decoupled design because  you just think it's stupid to write test first.  But there's one other thing you get out of it.  In fact, you can throw all that other stuff away.  It's one other thing you get out of this.  You get a suite of tests that you trust with your life.  You get a suite of tests where you can click a button and see all those tests pass and  you know the system works.  And when you have that and then up on your screen comes some dirty code and you think,  oh, I should clean it, you've got those tests.  So you try.  You make a small change.  Maybe I can change that variable's name.  And you run the tests.  Oh, they still pass.  Maybe I can take that function and split it into two functions.  Run the tests.  Oh, they still pass.  Maybe I can take that class and split it up into two classes.  Run the test.  Oh, I don't know if I can walk, put it back.  When you have those tests and when you trust them, you can clean the code.  You will clean the code because suddenly the cost of changing code approaches zero.  The risk goes way down.  You've got a suite of tests that tells you it's okay.  Don't worry.  I'll just make a little change here.  Oh, the tests still pass.  This is so nice.  Of course, the tests have to run quickly.  It doesn't help you if the tests take five hours to run.  So you need the tests to run like this.  That's a design challenge, by the way.  I pose it to you as a design challenge.  Keep your tests running fast.  Your designers, you can figure out a way to do that.  Software is a sensitive discipline.  And by that I mean that if I have a binary executable that's maybe a megabyte long,  how many bits are in that?  Eight million bits, small program.  Eight million bits.  How many of those bits could I flip from one to zero or zero to one and crash the system?  Well, there is one in there that will do it.  How many of those one bits are in there that will do that?  Probably thousands.  There are probably thousands of bits inside that eight million, which if I just flip them  from one to zero, it will crash the system.  Our systems are sensitive to failure at the bit level.  Hardly any systems are that sensitive.  We could walk around this building and find some bolts and some girders, take a few of  them out.  The building wouldn't fall down.  We could go out onto the highway and start removing bolts off the bridges.  The bridges would stay up there for a while.  We could go out drinking tonight and kill 10,000 brain cells.  Probably enjoy the process.  Humans are not sensitive at the one cell level.  Bridges are not sensitive at the one bolt level, but software is sensitive at the one  bit level.  The only way to create correct software is to make sure that all eight million of those  bits is correct.  That's hard to do.  Most engineering disciplines don't have that problem.  Most engineering disciplines live inside a robust world where there's room for error.  We don't have that.  What other discipline is similar to ours?  Well, there is one.  It's accounting.  The accountants, they work with spreadsheets and there are digits on those spreadsheets,  not all of them, but certain digits on those spreadsheets which if they get wrong will  take the company down and send the executives to jail.  How do accountants prevent that from happening?  Sometimes they don't.  But how do they try to prevent that from happening?  They have a discipline.  That discipline is called double entry bookkeeping.  They say everything twice.  They enter every transaction once on the debit side, once on the credit side, and those two  transactions follow separate mathematical pathways until there's this lovely subtraction  on the balance sheet that must yield a zero.  If it doesn't yield a zero, something went wrong.  If it does yield a zero, we hope nothing went wrong.  It's possible that you could make complementary entries that gave you a false zero.  That can happen.  The odds are very unlikely.  No judge would throw you in jail if you destroyed the company based on that kind of an error.  When accountants enter their transactions, they enter it in the debit side, they enter  it on the credit side, they get the zero on the balance sheet, then they enter the next  ones.  They invented this 500 years ago, probably after a king started cutting off some heads.  Test-driven development is double entry bookkeeping.  We say everything twice, once on the test side, once on the production code side.  We say them at the same time, just like they enter the transactions at the same time.  We do that for exactly the same reason.  We have to get all our bits right.  They've got to get all their digits right.  We've got to get all our bits right.  How can we as professionals treat our material, our code, with less respect than accountants  treat theirs?  Are they better than us at the moment they are?  But should they be?  Is their stuff, are their spreadsheets more important to the company than our code?  Why do they have a discipline like that?  Do we don't, or at least we don't follow it?  Why is it that we, thumb our nose and say, ah, there's test-driven development stuff.  Let me write the code.  Can you imagine an accountant saying, ah, there's double entry bookkeeping stuff.  Let me just enter the transactions.  I'll enter them right.  What a lot of crap.  Do accountants have a QA department that they hand their spreadsheets to?  Here, we're done with our spreadsheet's QA.  Now see if you can find any problems with them.  Why do we have QA departments?  Why has our industry invented an entire department to make sure that we don't screw up?  Why?  Because we screwed up.  And our companies are going, ah, well, what are we going to do about these developers?  They're just pushing crap out to the door.  And, well, let's create another group, QA.  That'll make sure it doesn't happen.  And then you know what happens.  The developers say, ah, we got a QA department now.  That means I can meet any deadline.  Give me a deadline.  I'll check the code in by that deadline.  It'll have some bugs, but that's their problem.  I worked for a company once where everything was always done on time.  Right?  The schedule was perfect.  Everything was always done on time.  Sometimes a programmer would achieve his goal by checking in a blank source file.  There was nothing in it, but he checked it in.  Okay, check that off the schedule, and then QA gets it.  Hey, wait, there's no code here.  Well, that's a bug.  Put that on the schedule to fix it on the bug.  That's not on the development side of the schedule.  If you are practicing test-driven development or any kind of testing discipline, the goal  is to cover the code with tests to make sure that you have tested every line of code you  write.  Does it make sense to have any lesser goal?  For example, should you be satisfied with 60% code coverage?  Let's say you've got a tool that measures code coverage, and it says, oh, 60% of your  code is covered.  Is that good?  A lot of us would look at it and think that's great.  But what it means is that you don't know if 40% of the code works.  Is that acceptable?  No.  What if an accountant said, 40% of our transactions are double entry?  Yeah, they're 60%, we're not so sure about.  But 40% of them we're pretty sure are right.  Isn't that good enough?  No, of course it's not good enough.  There is no meaningful goal short of 100%.  If you're looking at code coverage, that's the only number to think of.  How am I going to get this up to 100%?  You never will get it up to 100%.  That's okay.  It's an asymptotic goal.  You never fall for the silly line that says, you know, 70% is pretty damn good.  No, it's not.  It's awful.  70% is terrible.  It means there's 30% of your code that you've got no idea about.  This is the end result of QA.  When you write a bunch of code, you hand it to QA, what should they find?  Nothing.  QA should find nothing.  They should wonder why the heck they have a job.  And that should be your goal.  You should be looking over at those QA guys and going, you're not going to find anything.  Next year you won't be here.  You'd like to take QA and have them do a slightly different job, actually, because having them  work at the back end is an awful thing to do to people.  Imagine that this is your job.  You're going to, has anybody seen what the QA test plan looks like?  There it is.  Anybody seen the QA test plan?  That's not the QA test plan.  That's the table of contents for the QA test plan.  The guy holding it out to me is the QA manager of a large internet travel organization.  He's got 80,000 manual tests that he farms out to India every six weeks.  It costs him a million dollars every six weeks to run those manual tests.  He's holding it out to me because he's just come back from a meeting with his boss who  just came back from a meeting with the CFO.  The CFO said, what the hell is this million dollars every six weeks that we're spending?  He's talking to me now saying, I just got my budget cut by 50%.  Which half of these tests should I not run?  I told him, it doesn't matter.  You can cut the document any one of a bunch of different ways.  You're not going to know if half your system works.  This is the end result of manual testing.  Manual tests grow and grow because the features of the system grow and grow.  At some point, the effort required to execute those manual tests will become so expensive  that some financial officer will spot it and say that must be cut.  You will lose your tests.  It will happen.  The only solution to that is to automate those tests.  I don't want QA finding anything and I want all the tests that QA writes.  I want QA writing the tests, by the way.  I want all the tests that QA writes to be executed automatically.  I don't want any manual tests in the system.  Manual tests are fundamentally immoral.  That doesn't mean that you can't have people sitting there operating the system.  I like the idea of exploratory tests, but I don't want them given a script.  These are the tests you must pass.  Imagine that's your job.  You've got to follow that script.  Has anybody looked at what that's like?  Enter P9-537 into the username field.  Hit return.  Enter QX723 into the password field.  Click login.  Did the welcome screen come up?  This is your job.  And you're going to repeat that every six weeks.  One last.  One of the things a professional learns how to do, and it's very, very difficult, is to  say no.  If your boss comes to you and says, we've got to have this by Friday, and you know it  cannot be done by Friday, you must say no.  You don't understand.  We must have it by Friday.  No.  Now, you understand that if we don't have this by Friday, we're going to lose a lot  of money.  I understand that.  So will you get it to us by Friday?  No.  Why not?  Because it's not possible.  Well, can you try?  No.  You must never say that you will try.  What behavior are you going to change because you said you would try?  How will your strategy now be altered because you said you would try?  How are the odds of succeeding made different because you said you would try?  The reason you said you would try was to make him go away.  And what he heard was that you would do it.  And what you meant was get out of my face by saying that you would try, you told a lie.  And that lie will come back, of course, because you won't get it done by Friday.  And then they'll say, well, you said you'd try.  Well, yeah, I tried.  Well, you didn't try enough.  Thank you for your attention.  I'll see you all at another talk.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.  Thank you.
Robert_C_0_8303.png Robert_C_8304_11771.png Robert_C_11772_12575.png Robert_C_12576_14963.png Robert_C_14964_24347.png Robert_C_24348_27803.png Robert_C_27804_31919.png Robert_C_31920_39503.png Robert_C_39504_46223.png Robert_C_46224_46859.png Robert_C_46860_47411.png Robert_C_47412_48755.png Robert_C_48756_51827.png Robert_C_51828_55895.png Robert_C_55896_56651.png Robert_C_56652_69683.png Robert_C_69684_76475.png Robert_C_76476_78887.png Robert_C_78888_79727.png Robert_C_79728_80171.png Robert_C_80172_80219.png Robert_C_80220_84575.png Robert_C_84576_87875.png Robert_C_87876_90192.png