Advanced Test Driven Development
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Martin, Robert Cecil
Once you start to get good at TDD, you begin to learn the nuances of the discipline such as the fact that tests and code grow in very opposite directions. As the tests get more specific, the code gets more generic. You also begin to learn that in the red/green/refactor cycle there are just a few standard gestures that move a test from red to green. In this talk, Uncle Bob will discuss those gestures, called transformations, and will present the idea that they when applied in a particular order and priority they can have a profound effect on the resulting code. This one is at the bleeding edge of TDD research; and will leave you with something to ponder.
 Hello, hello, hello. I'm going to keep my voice down. Can you hear me all the way up there? Everybody okay up there? Yes, I can see you barely because the light is in my eyes. Good. I'm going to see if I can keep my voice low so that I don't strain it. Imagine that my outstretched arms represent the age of the earth from its formation four and a half billion years ago until now. Where are the dinosaurs? Right there. In my first and second knuckle at the very tip of my long middle finger, that's where the dinosaurs were born, that's where the dinosaurs were destroyed. The dinosaurs are very, very recent. We don't like to think about it that way. We think of the dinosaurs as being ancient, but they were only destroyed 65 million years ago out of an age of four and a half billion years. Here, right where my wrist is, is when life crawled out of the ocean. The history of life on the surface of the planet is only as long as my hand. But where on this scale did life begin in the ocean? The answer to that is here. Over three and a half billion years ago, there was life on this planet. Microscopic life, bacterial life, but life. And we know this because we measure the ages of the rocks using radioactive decay. And we see tiny fossils of bacteria here. And we see the rock structures, the stromatolites that they created. And most important, we see the rust. Because those ancient life forms were autotrophic. They turned sunlight into energy. And they released oxygen. And that oxygen combined with the iron in the sea to create rust. And all during this time, all the way to here, that rusting was occurring. It was only here that finally all the iron in the ocean was consumed by the oxygen. And now oxygen could build up in the atmosphere. Prior to this, there was no free oxygen anywhere in the atmosphere. But this is not what we're supposed to be talking about. I just find it fun. How many of you watch my videos? Lots of you. And you notice that I begin all my videos with a science lecture. I do this because it's fun. I also do it because it's something of a trademark. 20 years ago, 25 years ago, I taught a number of courses in C++. And I would break every hour. And at the end of the hour, I would get everybody to come back in the room. And I noticed that they were all talking to each other. And I could not begin my lecture. And I would try to hush everybody, but they would continue talking. So I stumbled on the idea of lecturing about something interesting, but not part of the curriculum. And I found that people almost immediately stopped talking because they wanted to hear it. So I've been using that technique ever since. Now it's become a trademark, so I do it no matter what. The name of today's talk is Advanced Test Driven Development, the Transformation Priority Premise. For the next hour, I'm going to talk to you about test-driven development and some advanced concepts in this idea. How many of you are test-driven developers? Look at that. Now, of course, it's a self-selecting crowd, but I'd say that was 80% of you. If I'd asked that question five years ago, it would have been a third of you. If I'd asked that question five years ago or seven or eight years ago, it would have been a small smattering of you. Now we have 80%. Now you're attending a course or a talk on advanced test-driven development, so I assume you were attracted to it. Because of that, still, the ratio is impressive. What does it mean? It means that the discipline is gaining ground. This controversial discipline introduced to us 14 years ago by Kent Beck has slowly been gaining ground, like a rolling snowball. Wherever I go, more and more people are claiming, at least, to be doing test-driven development. I'm going to test that claim, by the way, by defining it for you. Procedure-driven development is composed of three laws. The first law is that you are not allowed, by the discipline, to write any production code until you have first written a failing unit test. You cannot write any production code. You have to first write a unit test that fails. Once it fails, then you can write production code. And oh, by the way, not compiling is failing. The second law is you are not allowed to write more of a unit test than is sufficient to fail. As soon as the unit test fails, or fails to compile, you must stop writing it and start writing production code. The third law is the worst of them all. The third law says you are not allowed to write more production code than is sufficient to pass the currently failing test. If you follow these three laws, you will find yourself trapped into a cycle that is 10 seconds long. You write a tiny bit of unit test code, but it won't compile, because the functions you're calling have not yet been written. So you must stop writing the unit test and start writing production code. But you will add just the barest amount of production code, and that will make the unit test compile. You must stop writing a production code and go back to the unit test. You will add a little bit more unit test code until it fails. You will add a little bit more production code until that test passes, and you will oscillate back and forth between these two streams of code on a second-by-second basis. Ten seconds around the turn, maybe 20, maybe 30. How many of you are doing test driven development? That's a significantly reduced number. That's the discipline. Now, disciplines. Disciplines are arbitrary. We choose them because we made some decision. So for example, how does a doctor scrub before surgery? In the United States, at least in some hospitals, they teach their doctors to get a brush, soap up the brush, and then do 10 strokes across the side of the finger, test 10 strokes across the top of the finger, 10 strokes on the other side of the finger, 10 strokes across the face of the finger, 10 strokes across the nail, next finger. That's a discipline. It's an arbitrary discipline. Is 10 strokes the right number? Do you have to do the four quadrants? Could you do three quadrants? It's not relevant. It's just a discipline. It's arbitrary. And yet, the doctors teach each other this discipline, and they follow it, and they watch each other in the scrub room. They watch to make sure that they're following the discipline as it was taught to them. How many of you are pilots? Anybody here learn to fly? One of you. Maybe more. I don't know. When you learn to fly, you learn disciplines. One of the most important disciplines is the checklist. Before you do anything, you pull out the checklist, and you make sure that you follow the checklist items. You first get to the airplane. You get the checklist out, and the checklist says, walk around the airplane. Look at the different parts of the airplane. Make sure the airplane is fine. It's a discipline. And you learn this discipline, and your instructor says, start here. Walk around this way. Walk that way. Around the plane. Look at this. Look at that. It's all on the checklist. And he observes. You do it. He watches you. He's decided that you've learned the discipline. He puts a signature in your log, saying that you are now allowed to do this unsupervised. Those are disciplines. Disciplines invented by professions. Professions that decided they needed disciplines. Does our profession need discipline? Some day, maybe it's already happened, something terrible is going to happen. I don't know when this will happen, but it will. Some poor software guy is going to do some stupid thing. And tens of thousands of people will die. Is this possible? It's absolutely possible. How much software is running around you at the moment? And not in your iPhones, and not in your smartphones, and not in your laptops. How much software is running in the walls of this building? Software that you depend upon for your life. Is there software running in the smoke detectors? Is there software running in the fire alarms? Is there software that controls the opening and locking of the doors? Are there elevators around here? Is there software that controls the elevators? Go out on the road. How much code runs in every car out there? Would you be surprised to find out there's 100 million lines of code in a modern car? A hundred million lines of code in a modern car? That should scare the hell out of you. Because you know what that code is. Most of the world does not. If they did, they would ban the cars. And you know this. Someday something horrible will happen. Tens of thousands of people will die because of a software error. And the politicians of the world will rise up in righteous indignation as they must. And they will point their fingers squarely at us. And they will ask us the question, how could you have let this happen? And our answer must be a good one. Because if we answer by saying, well, my boss made me do it. Or we had to get to market on time. Or I just didn't feel like following any disciplines. If that was our answer, then the politicians of the world will do the only thing that they can do. The only thing they're good at, and even that's questionable. They will legislate. They will pass laws. They will regulate us. They will tell us what languages we can use, what practices we must follow, what books we must read, what tests we must pass, what computers we can operate on. And we will all become civil servants. Don't underestimate this. Our society now depends for its life on us. Software is everywhere. We are the only ones who know what that software is really like. Eventually the rest of the world will find out. They almost found out with the American healthcare system, healthcare.gov, which turned into an incredible disaster, a software disaster of nearly catastrophic measure. Here was a law passed through both houses of the United States Congress, signed into law by the President of the United States, and they could not get the software to execute. It was a terrible disaster. As a result of that, the United States government is now contemplating a new cabinet position reporting directly to the President, the CTO of the United States of America. What would this guy do? Thinking about that scares the hell out of me. Why TDD? I went through the three laws. Now let's talk about what those laws by you. If we adopt a discipline like this, just because we think we ought to be disciplined, then we're stuck in this funny little loop, this 20-second loop. What does that loop buy us? Well, if you're in this 20-second loop or this 10-second loop, it means that 10 seconds ago, everything worked. Everything you were working on. You executed, passed all its tests. What would your life be like if 10 seconds ago, or a minute ago, or five minutes ago, everything worked? And I mean, everything worked. How much debugging would you do? The answer to that is, well, you're not going to do a lot of debugging. How many of you are good at the debugger? I mean, you know the debugger. The debugger is your tool. You know how to set breakpoints and watchpoints. You've got all the hotkeys down. I mean, you know how to watch the variables change and get to this breakpoint three times and get to that breakpoint seven times so you can debug. This is not a skill to be desired. You don't want to be good at the debugger. The only way you get good at the debugger is to spend a lot of time debugging. I don't want you spending a lot of time debugging. I want your time to be spent making tests pass. I want your time to be spent writing code that doesn't need to be debugged. Now, you're still going to debug. It is still software. It's still hard. But the amount of time you will spend in the debugger becomes very small. So small that you lose the control that you had. Your fingers don't remember the hotkeys anymore. The debugger becomes a foreign tool. You can still use it. But you don't use it enough. And that's a good thing. But never mind that. What else? How many of you have integrated a third-party package? You buy some third-party package from some source. You get some zip file. You unzip it. In there, there's DLLs. There's maybe some source code. Inevitably, there will be a PDF. That PDF is a manual. That manual is written by some tech writer. And at the back of that manual, there's an ugly appendix with all the code examples. Where's the first place you go? Your programmers. You don't want to read what the tech writer wrote. You go to the code examples first. You look at the code examples. The code examples will tell you the truth. If you're lucky, you can copy and paste those code examples into your application and fiddle them into working. What you are writing when you write unit tests in this tiny little cycle are the code examples for the whole system. You want to know how some part of the system works? There is a test that tells you how that part of the system works. You want to know how to call some API function? There's a test, probably several tests, that tell you every way that you can call that API. You want to know how to create some object? There is a test, a suite of tests, that creates that object every way it can be created. And these tests are little documents. Little documents that describe how to use the system at its most detailed level. And these documents are written in a language you understand. They are utterly unambiguous. They cannot get out of sync with the application. They are so formal that they execute. They are the perfect kind of document. But never mind that. How many of you have written unit tests after writing the code? Ah, now these are all the people who said they were doing test driven development. It seems logical to us to write the code first and write the code second. But if you do it that way, then you will inevitably come across as you are writing tests. By the way, how much fun is that? It's not fun. Why isn't it fun? Because you already know the code works. You've tested it manually. You went through it all. You wrote it all. You brought it up on a screen. You did it. It works fine. Ah, now I've got to write unit tests. Why do I have to write unit tests? Because some process guys said I have to write unit tests. So you write them, but you write them begrudgingly. You're not happy about it. They're not fun anymore. And you write this one and you write that one and whatever. Write this one. Okay, fine. Yeah, I think I got most of it done, whatever, and you're done. Or, and this happens frequently enough, you come across the function that is hard to test. The function that you look at it and think, oh, wait a minute, if I try to test that, it'll erase every row in the database. Man, I don't think I'm going to test that when I saw it work when I tested it manually. I'll just let that one go. And you leave a hole in your test suite. If you write your test first, that can't happen. If you write your test first, you must design the code, the production code, to be testable. There will be no function that's hard to test because you can't write the function that's hard to test. You write the test first. And a testable function is testable because it's decoupled. So the only way to write testable code is to decouple it. The act of writing the test first forces you to decouple things that you hadn't thought you ought to decouple. You hadn't even thought about decoupling. You wind up with a much less coupled design when you write your test first. You also wind up, more importantly, with a test suite that covers everything. Anybody out there have a coverage number? A goal for coverage that you're trying to hit? What's your coverage goal? 100%. Is that realistic? Hell no. No one can get a 100% coverage. But that's the only meaningful goal. If your goal is 80%, then what you are saying is that it's okay if 20% of the code doesn't work. Your goal has to be 100%. Even though you cannot hit this goal. It's an asymptotic goal. It's like losing weight. You never get to your target weight. But you keep on trying to lose. What happens when you have a test suite that tests almost everything? A test suite that runs quickly, maybe in a minute, two minutes, three minutes, and it tests almost everything. It tests so much that if it passes, you are willing to deploy. That's the goal, by the way. The goal is to get the test suite to pass, and if the test suite passes, you deploy. No other QA, no other stuff after the fact, no manual testing after that. If the test suite passes, you deploy. If you had that, what could you do with it? How many of you have been slowed down by bad code? Okay, if you look around, you see that's unanimous. Why did we write it? If we know it slows us down, why would we write it? Well, there's all kinds of reasons why you'd write it. The real reason why we write bad code is because we always write bad code. It's hard to write good code. It's especially hard to write good code when we're trying to make it work. There's this problem. We've got to get something to work. All of our focus is on getting this code to work. You're writing this code, you're writing this code, and you're testing it manually, and you're writing it, you're testing it manually, and you're not working, and you write the code, and all of a sudden it works, and you back away very carefully, and you check it in, and you walk away. And of course, the code's a mess, but once you've gotten it working, you don't want to touch it. How many of you have brought code up on your screen, and your first reaction to that code is, huh, this is a mess, I should clean it, and your next reaction is, no, I'm not touching it. Because you know if you touch it, you will break it, and if you break it, it will become yours. So you walk away, you leave the code in a mess, you react in fear, fear of the code, fear of a change to the code. I submit to you that if you fear changing the code, the only thing that can happen to your code is that it will rot, because you will never do anything that improves it. You will only do things that make it worse, and worse, and worse, and worse, and you will continue to slow down, and slow down, and slow down until you are at a virtual stand still. Some of you may be in that position now, possibly a majority of you, maybe at that virtual stand still where estimates have grown to months that used to be weeks. Why? Because you're afraid of what happens to the code. If you had a suite of tests that you believed in that ran quickly, then that code can come up on your screen and you think, oh, I should clean that, and your next thought is, hmm, I think I'll change the name of that variable, run the tests. Oh, that didn't break anything, huh? I think I'll take that function and split it into two functions, run the tests. Oh, that didn't break anything. Huh, well, maybe I'll take that class that's a little too large, I'll split it into two classes, run the tests. Oh, that didn't break anything. If you have the suite of tests, you eliminate the fear. If you eliminate the fear, you can clean. If you can clean, you can go fast. But you need a suite of tests that you trust with your life. You need a suite of tests that is so bulletproof that you can deploy based on it. And the only way I know of to get a suite of tests that good is to write the test first, to follow the discipline. As absurd as the discipline may sound. That discipline does guarantee you a suite of tests that covers just about everything. I'd like you to consider how incredibly irresponsible and unprofessional it is to be afraid of the code you have created, to react in fear of changing this thing that you made. What do we call this stuff that we make? We call it software. And the first word there is soft. Why is that first word soft? Because we expect it to be easy to change. If we had wanted it to be hard to change, we would have called it hardware. We don't. We invented software so that we could easily change the behavior of machines. At this, our industry has failed. We make it hard to change the software. We become afraid of the software. This is an incredible failure of our industry, and it must be fixed. We are not the only industry to have faced this problem. Other industries have had the same issue. One of those industries is accounting. Even the poor accountants, how much like software is accounting? If you were to give me a thumb drive, that's a nail file. It is remarkable that I cannot tell the difference between 8 gigabytes and a nail file. 8 gigabytes. Give this to me. Put your application on it. Maybe your application is 200 megabytes. That's what? 1,600 million bits. I can find one bit in that 1,600 million. I can find one bit. Flip one bit and make your application unusable. Your application is sensitive to failure at the bit level. There are individual bits that will destroy your application. That is not true, I hope, of this theater. There's no single pulley that if it fails will have me crashing to the floor. I hope. There's no single cable which if it fails will have me crashing to the floor. I hope. We could go out on the road and take bolts out of bridges. They wouldn't fall down right away because as humans we don't like single points of failure. But software is loaded with single points of failure. There's probably thousands of bits in here which if I flip them will completely corrupt the application. What other industry had that problem? Accountants did. Accountants had that problem. A single digit on just the right spreadsheet at just the right time can completely corrupt the status of the books cause the company to go under bankruptcy, send all the executives to jail. We've seen it happen. How did accountants deal with that? They invented a discipline 500 years ago. The name of that discipline is double entry bookkeeping. Every transaction gets entered two times. Once on the liability side, once on the asset side. Then they follow separate sums until there is a subtraction on the balance sheet that must yield a zero. The accountants in the old days would enter a single transaction on the liability side, the same transaction on the asset side. They would do the sums. They'd do the subtraction on the balance sheet. They'd get a zero. They knew they hadn't made an error. Then they would do the next transaction. They did not do all the assets first, all the liability second, do the sums, do the subtraction, and get a 37. I wonder which one of those was wrong. Test driven development is double entry bookkeeping. It is exactly the same discipline. It's done for precisely the same reason. Everything is said twice. Once on the production code side, once on the test side, they follow complementary execution pathways to wind up at a Boolean result, pass or fail. Did we do them one transaction at a time? One little bit in the test side, one little bit on the production code side, run them, they pass. Now if you were so upset about having to write the test first, I would not complain bitterly if you wrote the code first and the test second so long as you followed the same cycle. As soon as the code doesn't compile, you make it compile by writing just enough unit test to make it compile. As soon as the code does something that the test don't cover, you write just enough of the test to make it cover. As long as your oscillation between the two streams is 10 seconds long, I don't really care which one comes first. The problem with people who write their unit test after is not that they're writing unit test after. It's that they wait a long time before they write the unit test. Now if you've adopted this discipline, the first thing you face is just the unfamiliarity with it. It's difficult, it's odd. You've got to figure out some way to write a test, but it's hard to do that until you learn about things like test doubles and mocking and some of the other interesting techniques that come along. And as you learn those things, it gets easier. After a few weeks, it can become pretty easy knowledge. So what are the deeper implications? What once you become familiar with test-driven development, what are the deeper rules? Let me show you some of them. You can see this, right? A little bit of Java code. I am going to do for you the good old prime factors cata. The prime factors cata is an exercise in test-driven development that test-driven developers do as a practice, a warm-up. Something you do in the morning just to get ready, do the prime factors cata, good, now I'm done, now I can actually get to work. Just like a warm-up exercise. The goal of this is to compute the prime factors of a given integer. The origin of this cata comes from my son who was in sixth grade, this would have been about 2005. He came home from school with this homework, calculate the prime factors of a bunch of integers. And I said to him, well son, I'm glad to help you with this, go to your bedroom, do the best you can, by the time you're done I will have written you a program that will allow you to check your work. It won't give you the answers, but you can type in your answers and it will tell you if they're right or wrong. I sent him off to his bedroom and then I began to write this program in Ruby. I'll do it for you in Java. I sat down at my kitchen table and I thought about it for a minute. I said, how am I going to find the prime factors of an integer? And I thought to myself, well what I need is a list of prime numbers. If I have a list of prime numbers, then I could divide my candidate number by these prime numbers and any of them that divide evenly are a factor. And I'll just walk through the list of prime numbers up to some limit and that'll help me find all the prime factors. How will I compute the list of prime numbers? There's an old algorithm called the SIV of erratostinies. I'll use that algorithm to create an array of prime numbers and then I will divide through by the prime numbers. That's been perfectly reasonable to me. But I had been doing test-driven development for about five years so I was pretty new at it. And I thought to myself, wait, wait, wait. When you're doing test-driven development, you don't start with a big design. Since that time I've modified that slightly, I actually do start with big designs. I just don't believe them. So here I had a big design, big in some definition of the word big. And I thought, I'm not going to believe that design. I'm going to write the tests and let the tests lead me. And here's what happened. What you see here is the very first test right here. Note that the factors of one is the list with nothing in it. One has no prime factors. If I run this test, you will see that it fails because the factors function is returning in null. You can see that right here. Oops, sorry. You can see that right here. The factors of function takes an n, an integer, and it returns a null. I can make this pass by doing the simplest of changes. I'm going to take this null, which by the way is a very degenerate constant, and I'm going to transform it into a new array list of integers. And of course, it wants me to import that. And now if I run my test, my test should pass. I'm a programmer. I got it to pass. The burst of endorphins rations to your head, you know that you are a god. The machine is your slave. Time for the next test. The factors of two is the list with a two in it. The prime factors of two is just two. Oh, this fails, horror of horrors. How can I make this pass? I can make this pass by modifying my algorithm. And now watch this carefully. I still want that constant. I just don't want to return it. I'm going to have to modify it before I return it. So I'm going to transform this constant into a variable named factors. The constant is still there, but now it's held by an identifier named factors. And now I must modify that. I must put an if statement in. If n is greater than one. factors.add two. Ha! Now most of you are thinking, well, what kind of crazy nonsense was that? There's no design there. You're just hacking an if statement in. Watch what happens to that if statement. Watch what this if statement does. It's fascinating. Next test. The factors of three is a list with a three in it. This should fail. Notice now I'm getting into a hypothesis experiment loop. I write the test. I expect it to fail. That's my hypothesis. Ah, my hypothesis is correct. It fails. Ah, now I must make it pass. And to make this pass, we're going to play a game of golf. Golf in programming, in test driven development, is to make the test pass with the fewest key strokes possible. How do I make this test pass with the fewest key strokes possible? Two to n. Note that that was a change from a constant to a variable. A change from something specific to something more general. In fact, if you think carefully now, all the changes we have made to the production code have been a change towards generality. Every change has made the production code a little bit more general. I didn't have to do that. I could have put more if statements in. I could have said if n equals one, return a two. If n equals three, return a three. If n equals four, return a two and a two. I could have done that. But that would violate this new rule that we're beginning to smell. And the new rule that we're beginning to smell is this. With every new test, the tests become more and more constraining. The tests become more and more specific. Our response on the production code side is to make the production code more and more general. So everything we must do on the production code side must be a move towards generality. As the tests get more specific, the code gets more generic. Let's see if that works. Factors of four is a list with a four in it. No. Where are my pair programmers? The list with a two and a two in it. This should fail. Yeah, fails. How do I get that to pass? Well, if I look at this code here, I think, well, I can get this to pass by putting braces around this because I still need to know that n is greater than one. The only time I want to return an empty list is if n is equal to one. Then here I'm going to say if n is divisible by two, boy, do I hate that code, then I can add two. So if it's divisible by two, I will put a two in the list. And then, and then, and then, I will reduce n by two. So I pull the factor out of n, falling out of the loop. And I look at this and say four. Four will work because it's greater than one. It is divisible by two. I will put the two in the list. I will convert the four to a two, and I will put the second two in the list. And that will work. It doesn't work. Oh. But the test that failed was the test for the number two. Look at that code again. So I put a two in the list, if I am doing the prime factors of two, well, two is greater than one. Two is divisible by two. I will put the two in the list. I will convert the n to a one, and I will add it. I never want a one in there. So I'll put another if statement in. And now you are completely convinced that all I'm doing here is hacking away at this. You're just adding if statements. But that if statement is an interesting one. It's the same as the one above it. It's the same predicate. Now when you have two if statements in a row, in fact, I can make this a little more interesting. I can take this if statement. I can move it completely out of the loop. That still passes. So now I have two if statements in a row with the same predicate. And that should smell to you like an unwound loop. That last if statement looks like the end condition of a loop. But never mind. We have more tests to do. Factors of five is the list with a five in it, because five is prime. Pass or fail? Pass. That's nice. Okay. That factors of six is the list with a two and a three in it. Pass or fail? Pass. That makes sense though. Six is divisible by two. Take the two out, leaves the three, puts the three. Yep, works fine. Yep, good. Okay. Factors of seven is the list with a seven in it, because seven is prime. Pass, three in a row. Three in a row. As bad as that algorithm looks, there's something right about it. Oh, you'll see. He asked me when I know when to stop. You'll see. Oh, yeah. Don't worry about that. I didn't want that up there. Go and build. Thank you. They've changed the key bindings, nasty little people. All right. Next test. Factors of eight is the list with two and two and two in it. Eight is two cubed. This will fail. There's nothing in my code that will put three things in the list. Yes, it fails. It puts a two and a four in the list. That makes perfect sense. How can I get this to pass? And now we'll play another game of golf. How can I get this to pass with the fewest possible keystrokes? Recursing, he says. There's a faster way. If to wow. I sat at my kitchen table. I turned that if to a while. I saw it pass and a chill went down my spine. What happened there? And it occurred to me. A while is a general form of an if statement. An if is a degenerate form of a while loop. Again, this is a move towards generality. I have made the code more general simply by letting the if statement continue until it was false. That's interesting. Let's do nine. Nine is three cubed. That should fail because nothing will put a three in there two times. It does fail. Actually, it failed by sticking a nine in there. How can I make this pass? I look at this code and I realize that right here, I have a little engine, a little loop that factors out all the twos. Now I could repeat that loop like so. I will take a copy of that. And I will paste that copy in there and I will turn all the twos into threes. So now I have a loop that takes out all the threes and I believe that that will pass. Yes, but that violates my rule. Actually, it violates several rules. By duplicating that loop, I have not made the code more general. In fact, I've made the code more specific. It is now tuned to three. Secondly, I've duplicated code. That's bad. So I'm going to get rid of this. I know what I want to do now. I just need to do it in a more general way. And a more general way would be to put this code here into another loop. Modifying the two. So first thing I'm going to have to do is change that two to a variable named divisor. First thing I'm going to have to do is take this code here and put it into a loop. When will that loop end? When n is reduced to one. When I have found all the factors of n, n will have been reduced to one. So I will execute this loop while n is greater than one. Look what happened to that if statement. Of course, I need to take the initializer outside of the loop now. And I'd better increment that divisor. That works. Fascinating. Ah, but, but. There's some cleanup I could do here. Because this loop cannot, cannot exit until n is a one. Which means this if statement no longer applies. It really was the terminating condition on an unwound loop. There's a little bit of cleanup I can do here still. While loops are wordy, let's turn them into for loops. This one is a simple for loop. It has no initializer. It does have an incrementer. The incrementer is just that. Let me get rid of that. I think that works. Yeah, that works. Now I can get rid of those horrible braces. I am on a mission to destroy all braces. I know, I know you're supposed to put braces in around every if statement and every while loop. But I have a different view of this. I'm going to work hard to make every while loop and every if statement one line long and then they don't need braces. If you want to put the braces in, you put them in. Oh, I think I can change that to a for loop. And the initializer of that for loop is right there. And the incrementer of that for loop is right there. I think that will still work. And I don't need these ugly braces anymore. Let's get rid of those. Well, that's a nice pretty little algorithm, isn't it? I wonder how it works. Let's see if it does work. Okay, we need some big number with lots of prime factors. Two times two times three times three times five times seven times 11 times 11 times 13. That's a bunch of prime numbers. Is a list of, let's take that and move it down. And we'll say two comma two comma three comma three comma five comma seven comma 11 comma 11 comma 13. Oh, we're done. We're not actually done. Well, we're done. There's a small improvement you can make to speed the algorithm up in the case of very large prime numbers. You don't need to loop while n is greater than one. You can loop until n is greater than the square root of the original n. But and that'll speed things up by a fair bit in the case of very large prime numbers. Other than that, however, this is the algorithm. Notice that it has nothing to do with finding a bunch of prime numbers and dividing through by prime numbers. It's an entirely different algorithm. Where did this algorithm come from? Well, it came out of my brain, obviously. I was sitting at the kitchen table. I developed this algorithm one little test case at a time. It's not like it magically happened. And yet it did not come about because I thought it through. I did think it through. I just didn't think it through beforehand. I thought it through while I was developing it. One little test case at a time. This implies that it is possible to create algorithms incrementally, to solve algorithmic problems incrementally, one test case at a time, if you follow the rule that everything you do to the production code becomes more and more general. If you think about that long enough, you'll realize, yes, the tests get more and more constraining, the code gets more and more general. At some point, it gets general enough to pass every test. Is it possible to derive algorithms incrementally one test case at a time? In this case, it certainly was. Is it generally possible? Let's do a thought experiment. You and I? The sort algorithm. I'm going to hand you an array of integers. I want to know what test cases you would pose in order to pass a sort algorithm and what changes you would make incrementally to that sort algorithm to make them pass. We will begin. What is the very first test case? No integers at all, an empty array. We always start with the most degenerate case. The most degenerate case is an empty array. Sorting an empty array is trivial. How do you solve it? You return an empty array. Okay. First test case is now passing. Second test case. One integer doesn't matter what the integer is, it's already sorted. This fails. It fails our current implementation, which returns an empty list. How can we make it pass? Return the input array. Notice how similar that was to returning the N. We had a 2 at first, then we returned an N. In this case, we will return the empty array. Interesting. Now, we will return the input array. Next test case. Two integers in order. Passes. Next test case. Two integers out of order. Fails. How do we make it pass? Switch them if they're out of order. So there's an if statement. Oh, that's similar to this, isn't it? An if statement. And if that if statement fails, we swap them. Remember you said that. Right? Yeah. Remember you said that. Passes. Next test case. Three integers in order. Passes. Three integers with the first two out of order. Passes. Three integers with the second two out of order. Fails. How do we make this pass? By making the code more general. You put the if statement with the swap into a loop. And you loop through the array, swapping if they're out of order. Passes. Three integers in reverse order. Fails. Because you swapped the first two, you swapped the second two, but the first two are still out of order. How do you make that pass? Put the loop that compares and sorts into another loop. That's actually one element shorter. What have we invented? All sort. The worst possible sort algorithm. So maybe test driven development is a great way to derive terrible algorithms. But this guy here gave us an answer to one of those early test cases. Two elements out of order. And he said compare them and swap them. There was a different way to make that test pass. It was not necessary to compare and swap them. What you could have done instead is to compare them and then return wholly new arrays, allocate brand new arrays out of whole cloth and put the elements in the right order. Not changing the existing array, creating a new array. Those of you who attended my last talk will recognize that as the functional solution. Creating the elements inside the array is non-functional because we're using assignment to do it. Now, it turns out, and I won't do this derivation for you. I'll let you do this at home because it's fascinating. It turns out that if you take that course, you put in the if statement and you create completely new arrays, you very quickly wind up at a quick sort. The quick sort falls out. The quick sort is probably the best possible algorithm. So apparently, there are forks in the road. As you are writing tests, there are choices you can make. Maybe two or three different ways to make a test pass. And if you choose the right course, you get to a better algorithm. What criteria can we use to make that choice? And it turns out, there are a whole list of possible things that you can look for, but the first thing to look for is assignment. If your solution involves assignment, find a solution that does not. If no such solution exists, then you have to go on with assignment. But if you can do it without assigning, you may wind up at a better algorithm. This is called the transformation priority premise. The premise that production code can be transformed into ever more general states. And there is a priority to the order in which you make those transformations. You prefer transformations that don't have assignment in them. There's a whole bunch of other preferences. If you look up the transformation priority premise, you will see a list of these possible predicates that were used for transformations. You'll find the documents on this interesting. They cover the sort algorithm. They cover Fibonacci. They cover a bunch of algorithms. And it's something interesting to try on your own. Which brings me right to the end of the talk so I can take one question. Anybody out there want to ask a question? Yeah, in the back. So the cycle in test driven development is supposed to be red, green refactor. Yes, that's true. And your question was that sometimes my tests passed immediately. Yeah, that does happen from time to time. Sometimes you pose a test that you think will fail and it doesn't. It passes. Okay. Then you don't have to go through the rest of the cycle. You thought it was going to be red, it turned green. That's a benefit. It doesn't happen very often, but it does happen. And with that, thank you all for your attention. See you next time. Thank you.
 Hello, hello, hello.  I'm going to keep my voice down.  Can you hear me all the way up there?  Everybody okay up there?  Yes, I can see you barely because the light is in my eyes.  Good.  I'm going to see if I can keep my voice low so that I don't strain it.  Imagine that my outstretched arms represent the age of the earth from its formation four  and a half billion years ago until now.  Where are the dinosaurs?  Right there.  In my first and second knuckle at the very tip of my long middle finger, that's where  the dinosaurs were born, that's where the dinosaurs were destroyed.  The dinosaurs are very, very recent.  We don't like to think about it that way.  We think of the dinosaurs as being ancient, but they were only destroyed 65 million years  ago out of an age of four and a half billion years.  Here, right where my wrist is, is when life crawled out of the ocean.  The history of life on the surface of the planet is only as long as my hand.  But where on this scale did life begin in the ocean?  The answer to that is here.  Over three and a half billion years ago, there was life on this planet.  Microscopic life, bacterial life, but life.  And we know this because we measure the ages of the rocks using radioactive decay.  And we see tiny fossils of bacteria here.  And we see the rock structures, the stromatolites that they created.  And most important, we see the rust.  Because those ancient life forms were autotrophic.  They turned sunlight into energy.  And they released oxygen.  And that oxygen combined with the iron in the sea to create rust.  And all during this time, all the way to here, that rusting was occurring.  It was only here that finally all the iron in the ocean was consumed by the oxygen.  And now oxygen could build up in the atmosphere.  Prior to this, there was no free oxygen anywhere in the atmosphere.  But this is not what we're supposed to be talking about.  I just find it fun.  How many of you watch my videos?  Lots of you.  And you notice that I begin all my videos with a science lecture.  I do this because it's fun.  I also do it because it's something of a trademark.  20 years ago, 25 years ago, I taught a number of courses in C++.  And I would break every hour.  And at the end of the hour, I would get everybody to come back in the room.  And I noticed that they were all talking to each other.  And I could not begin my lecture.  And I would try to hush everybody, but they would continue talking.  So I stumbled on the idea of lecturing about something interesting, but not part of the  curriculum.  And I found that people almost immediately stopped talking because they wanted to hear  it.  So I've been using that technique ever since.  Now it's become a trademark, so I do it no matter what.  The name of today's talk is Advanced Test Driven Development, the Transformation Priority  Premise.  For the next hour, I'm going to talk to you about test-driven development and some advanced  concepts in this idea.  How many of you are test-driven developers?  Look at that.  Now, of course, it's a self-selecting crowd, but I'd say that was 80% of you.  If I'd asked that question five years ago, it would have been a third of you.  If I'd asked that question five years ago or seven or eight years ago, it would have  been a small smattering of you.  Now we have 80%.  Now you're attending a course or a talk on advanced test-driven development, so I assume  you were attracted to it.  Because of that, still, the ratio is impressive.  What does it mean?  It means that the discipline is gaining ground.  This controversial discipline introduced to us 14 years ago by Kent Beck has slowly been  gaining ground, like a rolling snowball.  Wherever I go, more and more people are claiming, at least, to be doing test-driven development.  I'm going to test that claim, by the way, by defining it for you.  Procedure-driven development is composed of three laws.  The first law is that you are not allowed, by the discipline, to write any production  code until you have first written a failing unit test.  You cannot write any production code.  You have to first write a unit test that fails.  Once it fails, then you can write production code.  And oh, by the way, not compiling is failing.  The second law is you are not allowed to write more of a unit test than is sufficient to fail.  As soon as the unit test fails, or fails to compile, you must stop writing it and start  writing production code.  The third law is the worst of them all.  The third law says you are not allowed to write more production code than is sufficient  to pass the currently failing test.  If you follow these three laws, you will find yourself trapped into a cycle that is 10 seconds  long.  You write a tiny bit of unit test code, but it won't compile, because the functions you're  calling have not yet been written.  So you must stop writing the unit test and start writing production code.  But you will add just the barest amount of production code, and that will make the unit  test compile.  You must stop writing a production code and go back to the unit test.  You will add a little bit more unit test code until it fails.  You will add a little bit more production code until that test passes, and you will  oscillate back and forth between these two streams of code on a second-by-second basis.  Ten seconds around the turn, maybe 20, maybe 30.  How many of you are doing test driven development?  That's a significantly reduced number.  That's the discipline.  Now, disciplines.  Disciplines are arbitrary.  We choose them because we made some decision.  So for example, how does a doctor scrub before surgery?  In the United States, at least in some hospitals, they teach their doctors to get a brush, soap  up the brush, and then do 10 strokes across the side of the finger, test 10 strokes across  the top of the finger, 10 strokes on the other side of the finger, 10 strokes across the  face of the finger, 10 strokes across the nail, next finger.  That's a discipline.  It's an arbitrary discipline.  Is 10 strokes the right number?  Do you have to do the four quadrants?  Could you do three quadrants?  It's not relevant.  It's just a discipline.  It's arbitrary.  And yet, the doctors teach each other this discipline, and they follow it, and they watch  each other in the scrub room.  They watch to make sure that they're following the discipline as it was taught to them.  How many of you are pilots?  Anybody here learn to fly?  One of you.  Maybe more.  I don't know.  When you learn to fly, you learn disciplines.  One of the most important disciplines is the checklist.  Before you do anything, you pull out the checklist, and you make sure that you follow the checklist  items.  You first get to the airplane.  You get the checklist out, and the checklist says, walk around the airplane.  Look at the different parts of the airplane.  Make sure the airplane is fine.  It's a discipline.  And you learn this discipline, and your instructor says, start here.  Walk around this way.  Walk that way.  Around the plane.  Look at this.  Look at that.  It's all on the checklist.  And he observes.  You do it.  He watches you.  He's decided that you've learned the discipline.  He puts a signature in your log, saying that you are now allowed to do this unsupervised.  Those are disciplines.  Disciplines invented by professions.  Professions that decided they needed disciplines.  Does our profession need discipline?  Some day, maybe it's already happened, something terrible is going to happen.  I don't know when this will happen, but it will.  Some poor software guy is going to do some stupid thing.  And tens of thousands of people will die.  Is this possible?  It's absolutely possible.  How much software is running around you at the moment?  And not in your iPhones, and not in your smartphones, and not in your laptops.  How much software is running in the walls of this building?  Software that you depend upon for your life.  Is there software running in the smoke detectors?  Is there software running in the fire alarms?  Is there software that controls the opening and locking of the doors?  Are there elevators around here?  Is there software that controls the elevators?  Go out on the road.  How much code runs in every car out there?  Would you be surprised to find out there's 100 million lines of code in a modern car?  A hundred million lines of code in a modern car?  That should scare the hell out of you.  Because you know what that code is.  Most of the world does not.  If they did, they would ban the cars.  And you know this.  Someday something horrible will happen.  Tens of thousands of people will die because of a software error.  And the politicians of the world will rise up in righteous indignation as they must.  And they will point their fingers squarely at us.  And they will ask us the question, how could you have let this happen?  And our answer must be a good one.  Because if we answer by saying, well, my boss made me do it.  Or we had to get to market on time.  Or I just didn't feel like following any disciplines.  If that was our answer, then the politicians of the world will do the only thing that they  can do.  The only thing they're good at, and even that's questionable.  They will legislate.  They will pass laws.  They will regulate us.  They will tell us what languages we can use, what practices we must follow, what books  we must read, what tests we must pass, what computers we can operate on.  And we will all become civil servants.  Don't underestimate this.  Our society now depends for its life on us.  Software is everywhere.  We are the only ones who know what that software is really like.  Eventually the rest of the world will find out.  They almost found out with the American healthcare system, healthcare.gov, which turned into  an incredible disaster, a software disaster of nearly catastrophic measure.  Here was a law passed through both houses of the United States Congress, signed into  law by the President of the United States, and they could not get the software to execute.  It was a terrible disaster.  As a result of that, the United States government is now contemplating a new cabinet position  reporting directly to the President, the CTO of the United States of America.  What would this guy do?  Thinking about that scares the hell out of me.  Why TDD?  I went through the three laws.  Now let's talk about what those laws by you.  If we adopt a discipline like this, just because we think we ought to be disciplined, then we're  stuck in this funny little loop, this 20-second loop.  What does that loop buy us?  Well, if you're in this 20-second loop or this 10-second loop, it means that 10 seconds  ago, everything worked.  Everything you were working on.  You executed, passed all its tests.  What would your life be like if 10 seconds ago, or a minute ago, or five minutes ago,  everything worked?  And I mean, everything worked.  How much debugging would you do?  The answer to that is, well, you're not going to do a lot of debugging.  How many of you are good at the debugger?  I mean, you know the debugger.  The debugger is your tool.  You know how to set breakpoints and watchpoints.  You've got all the hotkeys down.  I mean, you know how to watch the variables change and get to this breakpoint three times  and get to that breakpoint seven times so you can debug.  This is not a skill to be desired.  You don't want to be good at the debugger.  The only way you get good at the debugger is to spend a lot of time debugging.  I don't want you spending a lot of time debugging.  I want your time to be spent making tests pass.  I want your time to be spent writing code that doesn't need to be debugged.  Now, you're still going to debug.  It is still software.  It's still hard.  But the amount of time you will spend in the debugger becomes very small.  So small that you lose the control that you had.  Your fingers don't remember the hotkeys anymore.  The debugger becomes a foreign tool.  You can still use it.  But you don't use it enough.  And that's a good thing.  But never mind that.  What else?  How many of you have integrated a third-party package?  You buy some third-party package from some source.  You get some zip file.  You unzip it.  In there, there's DLLs.  There's maybe some source code.  Inevitably, there will be a PDF.  That PDF is a manual.  That manual is written by some tech writer.  And at the back of that manual, there's an ugly appendix with all the code examples.  Where's the first place you go?  Your programmers.  You don't want to read what the tech writer wrote.  You go to the code examples first.  You look at the code examples.  The code examples will tell you the truth.  If you're lucky, you can copy and paste those code examples into your application and fiddle  them into working.  What you are writing when you write unit tests in this tiny little cycle are the code examples  for the whole system.  You want to know how some part of the system works?  There is a test that tells you how that part of the system works.  You want to know how to call some API function?  There's a test, probably several tests, that tell you every way that you can call that API.  You want to know how to create some object?  There is a test, a suite of tests, that creates that object every way it can be created.  And these tests are little documents.  Little documents that describe how to use the system at its most detailed level.  And these documents are written in a language you understand.  They are utterly unambiguous.  They cannot get out of sync with the application.  They are so formal that they execute.  They are the perfect kind of document.  But never mind that.  How many of you have written unit tests after writing the code?  Ah, now these are all the people who said they were doing test driven development.  It seems logical to us to write the code first and write the code second.  But if you do it that way, then you will inevitably come across as you are writing tests.  By the way, how much fun is that?  It's not fun.  Why isn't it fun?  Because you already know the code works.  You've tested it manually.  You went through it all.  You wrote it all.  You brought it up on a screen.  You did it.  It works fine.  Ah, now I've got to write unit tests.  Why do I have to write unit tests?  Because some process guys said I have to write unit tests.  So you write them, but you write them begrudgingly.  You're not happy about it.  They're not fun anymore.  And you write this one and you write that one and whatever.  Write this one.  Okay, fine.  Yeah, I think I got most of it done, whatever, and you're done.  Or, and this happens frequently enough, you come across the function that is hard to test.  The function that you look at it and think, oh, wait a minute, if I try to test that,  it'll erase every row in the database.  Man, I don't think I'm going to test that when I saw it work when I tested it manually.  I'll just let that one go.  And you leave a hole in your test suite.  If you write your test first, that can't happen.  If you write your test first, you must design the code, the production code, to be testable.  There will be no function that's hard to test because you can't write the function that's  hard to test.  You write the test first.  And a testable function is testable because it's decoupled.  So the only way to write testable code is to decouple it.  The act of writing the test first forces you to decouple things that you hadn't thought  you ought to decouple.  You hadn't even thought about decoupling.  You wind up with a much less coupled design when you write your test first.  You also wind up, more importantly, with a test suite that covers everything.  Anybody out there have a coverage number?  A goal for coverage that you're trying to hit?  What's your coverage goal?  100%.  Is that realistic?  Hell no.  No one can get a 100% coverage.  But that's the only meaningful goal.  If your goal is 80%, then what you are saying is that it's okay if 20% of the code doesn't  work.  Your goal has to be 100%.  Even though you cannot hit this goal.  It's an asymptotic goal.  It's like losing weight.  You never get to your target weight.  But you keep on trying to lose.  What happens when you have a test suite that tests almost everything?  A test suite that runs quickly, maybe in a minute, two minutes, three minutes, and it  tests almost everything.  It tests so much that if it passes, you are willing to deploy.  That's the goal, by the way.  The goal is to get the test suite to pass, and if the test suite passes, you deploy.  No other QA, no other stuff after the fact, no manual testing after that.  If the test suite passes, you deploy.  If you had that, what could you do with it?  How many of you have been slowed down by bad code?  Okay, if you look around, you see that's unanimous.  Why did we write it?  If we know it slows us down, why would we write it?  Well, there's all kinds of reasons why you'd write it.  The real reason why we write bad code is because we always write bad code.  It's hard to write good code.  It's especially hard to write good code when we're trying to make it work.  There's this problem.  We've got to get something to work.  All of our focus is on getting this code to work.  You're writing this code, you're writing this code, and you're testing it manually, and  you're writing it, you're testing it manually, and you're not working, and you write the code,  and all of a sudden it works, and you back away very carefully, and you check it in,  and you walk away.  And of course, the code's a mess, but once you've gotten it working, you don't want to  touch it.  How many of you have brought code up on your screen, and your first reaction to that code  is, huh, this is a mess, I should clean it, and your next reaction is, no, I'm not touching  it.  Because you know if you touch it, you will break it, and if you break it, it will become  yours.  So you walk away, you leave the code in a mess, you react in fear, fear of the code,  fear of a change to the code.  I submit to you that if you fear changing the code, the only thing that can happen to  your code is that it will rot, because you will never do anything that improves it.  You will only do things that make it worse, and worse, and worse, and worse, and you will  continue to slow down, and slow down, and slow down until you are at a virtual stand  still.  Some of you may be in that position now, possibly a majority of you, maybe at that virtual stand  still where estimates have grown to months that used to be weeks.  Why?  Because you're afraid of what happens to the code.  If you had a suite of tests that you believed in that ran quickly, then that code can come  up on your screen and you think, oh, I should clean that, and your next thought is, hmm,  I think I'll change the name of that variable, run the tests.  Oh, that didn't break anything, huh?  I think I'll take that function and split it into two functions, run the tests.  Oh, that didn't break anything.  Huh, well, maybe I'll take that class that's a little too large, I'll split it into two  classes, run the tests.  Oh, that didn't break anything.  If you have the suite of tests, you eliminate the fear.  If you eliminate the fear, you can clean.  If you can clean, you can go fast.  But you need a suite of tests that you trust with your life.  You need a suite of tests that is so bulletproof that you can deploy based on it.  And the only way I know of to get a suite of tests that good is to write the test first,  to follow the discipline.  As absurd as the discipline may sound.  That discipline does guarantee you a suite of tests that covers just about everything.  I'd like you to consider how incredibly irresponsible and unprofessional it is to be afraid of the  code you have created, to react in fear of changing this thing that you made.  What do we call this stuff that we make?  We call it software.  And the first word there is soft.  Why is that first word soft?  Because we expect it to be easy to change.  If we had wanted it to be hard to change, we would have called it hardware.  We don't.  We invented software so that we could easily change the behavior of machines.  At this, our industry has failed.  We make it hard to change the software.  We become afraid of the software.  This is an incredible failure of our industry, and it must be fixed.  We are not the only industry to have faced this problem.  Other industries have had the same issue.  One of those industries is accounting.  Even the poor accountants, how much like software is accounting?  If you were to give me a thumb drive, that's a nail file.  It is remarkable that I cannot tell the difference between 8 gigabytes and a nail file.  8 gigabytes.  Give this to me.  Put your application on it.  Maybe your application is 200 megabytes.  That's what?  1,600 million bits.  I can find one bit in that 1,600 million.  I can find one bit.  Flip one bit and make your application unusable.  Your application is sensitive to failure at the bit level.  There are individual bits that will destroy your application.  That is not true, I hope, of this theater.  There's no single pulley that if it fails will have me crashing to the floor.  I hope.  There's no single cable which if it fails will have me crashing to the floor.  I hope.  We could go out on the road and take bolts out of bridges.  They wouldn't fall down right away because as humans we don't like single points of failure.  But software is loaded with single points of failure.  There's probably thousands of bits in here which if I flip them will completely corrupt  the application.  What other industry had that problem?  Accountants did.  Accountants had that problem.  A single digit on just the right spreadsheet at just the right time can completely corrupt  the status of the books cause the company to go under bankruptcy, send all the executives  to jail.  We've seen it happen.  How did accountants deal with that?  They invented a discipline 500 years ago.  The name of that discipline is double entry bookkeeping.  Every transaction gets entered two times.  Once on the liability side, once on the asset side.  Then they follow separate sums until there is a subtraction on the balance sheet that  must yield a zero.  The accountants in the old days would enter a single transaction on the liability side,  the same transaction on the asset side.  They would do the sums.  They'd do the subtraction on the balance sheet.  They'd get a zero.  They knew they hadn't made an error.  Then they would do the next transaction.  They did not do all the assets first, all the liability second, do the sums, do the subtraction,  and get a 37.  I wonder which one of those was wrong.  Test driven development is double entry bookkeeping.  It is exactly the same discipline.  It's done for precisely the same reason.  Everything is said twice.  Once on the production code side, once on the test side, they follow complementary execution  pathways to wind up at a Boolean result, pass or fail.  Did we do them one transaction at a time?  One little bit in the test side, one little bit on the production code side, run them,  they pass.  Now if you were so upset about having to write the test first, I would not complain bitterly  if you wrote the code first and the test second so long as you followed the same cycle.  As soon as the code doesn't compile, you make it compile by writing just enough unit test  to make it compile.  As soon as the code does something that the test don't cover, you write just enough of  the test to make it cover.  As long as your oscillation between the two streams is 10 seconds long, I don't really  care which one comes first.  The problem with people who write their unit test after is not that they're writing unit  test after.  It's that they wait a long time before they write the unit test.  Now if you've adopted this discipline, the first thing you face is just the unfamiliarity  with it.  It's difficult, it's odd.  You've got to figure out some way to write a test, but it's hard to do that until you  learn about things like test doubles and mocking and some of the other interesting techniques  that come along.  And as you learn those things, it gets easier.  After a few weeks, it can become pretty easy knowledge.  So what are the deeper implications?  What once you become familiar with test-driven development, what are the deeper rules?  Let me show you some of them.  You can see this, right?  A little bit of Java code.  I am going to do for you the good old prime factors cata.  The prime factors cata is an exercise in test-driven development that test-driven developers do  as a practice, a warm-up.  Something you do in the morning just to get ready, do the prime factors cata, good, now  I'm done, now I can actually get to work.  Just like a warm-up exercise.  The goal of this is to compute the prime factors of a given integer.  The origin of this cata comes from my son who was in sixth grade, this would have been  about 2005.  He came home from school with this homework, calculate the prime factors of a bunch of  integers.  And I said to him, well son, I'm glad to help you with this, go to your bedroom, do the  best you can, by the time you're done I will have written you a program that will allow  you to check your work.  It won't give you the answers, but you can type in your answers and it will tell you if  they're right or wrong.  I sent him off to his bedroom and then I began to write this program in Ruby.  I'll do it for you in Java.  I sat down at my kitchen table and I thought about it for a minute.  I said, how am I going to find the prime factors of an integer?  And I thought to myself, well what I need is a list of prime numbers.  If I have a list of prime numbers, then I could divide my candidate number by these  prime numbers and any of them that divide evenly are a factor.  And I'll just walk through the list of prime numbers up to some limit and that'll help  me find all the prime factors.  How will I compute the list of prime numbers?  There's an old algorithm called the SIV of erratostinies.  I'll use that algorithm to create an array of prime numbers and then I will divide through  by the prime numbers.  That's been perfectly reasonable to me.  But I had been doing test-driven development for about five years so I was pretty new at  it.  And I thought to myself, wait, wait, wait.  When you're doing test-driven development, you don't start with a big design.  Since that time I've modified that slightly, I actually do start with big designs.  I just don't believe them.  So here I had a big design, big in some definition of the word big.  And I thought, I'm not going to believe that design.  I'm going to write the tests and let the tests lead me.  And here's what happened.  What you see here is the very first test right here.  Note that the factors of one is the list with nothing in it.  One has no prime factors.  If I run this test, you will see that it fails because the factors function is returning  in null.  You can see that right here.  Oops, sorry.  You can see that right here.  The factors of function takes an n, an integer, and it returns a null.  I can make this pass by doing the simplest of changes.  I'm going to take this null, which by the way is a very degenerate constant, and I'm  going to transform it into a new array list of integers.  And of course, it wants me to import that.  And now if I run my test, my test should pass.  I'm a programmer.  I got it to pass.  The burst of endorphins rations to your head, you know that you are a god.  The machine is your slave.  Time for the next test.  The factors of two is the list with a two in it.  The prime factors of two is just two.  Oh, this fails, horror of horrors.  How can I make this pass?  I can make this pass by modifying my algorithm.  And now watch this carefully.  I still want that constant.  I just don't want to return it.  I'm going to have to modify it before I return it.  So I'm going to transform this constant into a variable named factors.  The constant is still there, but now it's held by an identifier named factors.  And now I must modify that.  I must put an if statement in.  If n is greater than one.  factors.add two.  Ha!  Now most of you are thinking, well, what kind of crazy nonsense was that?  There's no design there.  You're just hacking an if statement in.  Watch what happens to that if statement.  Watch what this if statement does.  It's fascinating.  Next test.  The factors of three is a list with a three in it.  This should fail.  Notice now I'm getting into a hypothesis experiment loop.  I write the test.  I expect it to fail.  That's my hypothesis.  Ah, my hypothesis is correct.  It fails.  Ah, now I must make it pass.  And to make this pass, we're going to play a game of golf.  Golf in programming, in test driven development, is to make the test pass with the fewest key  strokes possible.  How do I make this test pass with the fewest key strokes possible?  Two to n.  Note that that was a change from a constant to a variable.  A change from something specific to something more general.  In fact, if you think carefully now, all the changes we have made to the production code  have been a change towards generality.  Every change has made the production code a little bit more general.  I didn't have to do that.  I could have put more if statements in.  I could have said if n equals one, return a two.  If n equals three, return a three.  If n equals four, return a two and a two.  I could have done that.  But that would violate this new rule that we're beginning to smell.  And the new rule that we're beginning to smell is this.  With every new test, the tests become more and more constraining.  The tests become more and more specific.  Our response on the production code side is to make the production code more and more  general.  So everything we must do on the production code side must be a move towards generality.  As the tests get more specific, the code gets more generic.  Let's see if that works.  Factors of four is a list with a four in it.  No.  Where are my pair programmers?  The list with a two and a two in it.  This should fail.  Yeah, fails.  How do I get that to pass?  Well, if I look at this code here, I think, well, I can get this to pass by putting braces  around this because I still need to know that n is greater than one.  The only time I want to return an empty list is if n is equal to one.  Then here I'm going to say if n is divisible by two, boy, do I hate that code, then I can  add two.  So if it's divisible by two, I will put a two in the list.  And then, and then, and then, I will reduce n by two.  So I pull the factor out of n, falling out of the loop.  And I look at this and say four.  Four will work because it's greater than one.  It is divisible by two.  I will put the two in the list.  I will convert the four to a two, and I will put the second two in the list.  And that will work.  It doesn't work.  Oh.  But the test that failed was the test for the number two.  Look at that code again.  So I put a two in the list, if I am doing the prime factors of two, well, two is greater  than one.  Two is divisible by two.  I will put the two in the list.  I will convert the n to a one, and I will add it.  I never want a one in there.  So I'll put another if statement in.  And now you are completely convinced that all I'm doing here is hacking away at this.  You're just adding if statements.  But that if statement is an interesting one.  It's the same as the one above it.  It's the same predicate.  Now when you have two if statements in a row, in fact, I can make this a little more interesting.  I can take this if statement.  I can move it completely out of the loop.  That still passes.  So now I have two if statements in a row with the same predicate.  And that should smell to you like an unwound loop.  That last if statement looks like the end condition of a loop.  But never mind.  We have more tests to do.  Factors of five is the list with a five in it, because five is prime.  Pass or fail?  Pass.  That's nice.  Okay.  That factors of six is the list with a two and a three in it.  Pass or fail?  Pass.  That makes sense though.  Six is divisible by two.  Take the two out, leaves the three, puts the three.  Yep, works fine.  Yep, good.  Okay.  Factors of seven is the list with a seven in it, because seven is prime.  Pass, three in a row.  Three in a row.  As bad as that algorithm looks, there's something right about it.  Oh, you'll see.  He asked me when I know when to stop.  You'll see.  Oh, yeah.  Don't worry about that.  I didn't want that up there.  Go and build.  Thank you.  They've changed the key bindings, nasty little people.  All right.  Next test.  Factors of eight is the list with two and two and two in it.  Eight is two cubed.  This will fail.  There's nothing in my code that will put three things in the list.  Yes, it fails.  It puts a two and a four in the list.  That makes perfect sense.  How can I get this to pass?  And now we'll play another game of golf.  How can I get this to pass with the fewest possible keystrokes?  Recursing, he says.  There's a faster way.  If to wow.  I sat at my kitchen table.  I turned that if to a while.  I saw it pass and a chill went down my spine.  What happened there?  And it occurred to me.  A while is a general form of an if statement.  An if is a degenerate form of a while loop.  Again, this is a move towards generality.  I have made the code more general simply by letting the if statement continue until it  was false.  That's interesting.  Let's do nine.  Nine is three cubed.  That should fail because nothing will put a three in there two times.  It does fail.  Actually, it failed by sticking a nine in there.  How can I make this pass?  I look at this code and I realize that right here, I have a little engine, a little loop  that factors out all the twos.  Now I could repeat that loop like so.  I will take a copy of that.  And I will paste that copy in there and I will turn all the twos into threes.  So now I have a loop that takes out all the threes and I believe that that will pass.  Yes, but that violates my rule.  Actually, it violates several rules.  By duplicating that loop, I have not made the code more general.  In fact, I've made the code more specific.  It is now tuned to three.  Secondly, I've duplicated code.  That's bad.  So I'm going to get rid of this.  I know what I want to do now.  I just need to do it in a more general way.  And a more general way would be to put this code here into another loop.  Modifying the two.  So first thing I'm going to have to do is change that two to a variable named divisor.  First thing I'm going to have to do is take this code here and put it into a loop.  When will that loop end?  When n is reduced to one.  When I have found all the factors of n, n will have been reduced to one.  So I will execute this loop while n is greater than one.  Look what happened to that if statement.  Of course, I need to take the initializer outside of the loop now.  And I'd better increment that divisor.  That works.  Fascinating.  Ah, but, but.  There's some cleanup I could do here.  Because this loop cannot, cannot exit until n is a one.  Which means this if statement no longer applies.  It really was the terminating condition on an unwound loop.  There's a little bit of cleanup I can do here still.  While loops are wordy, let's turn them into for loops.  This one is a simple for loop.  It has no initializer.  It does have an incrementer.  The incrementer is just that.  Let me get rid of that.  I think that works.  Yeah, that works.  Now I can get rid of those horrible braces.  I am on a mission to destroy all braces.  I know, I know you're supposed to put braces in around every if statement and every while  loop.  But I have a different view of this.  I'm going to work hard to make every while loop and every if statement one line long  and then they don't need braces.  If you want to put the braces in, you put them in.  Oh, I think I can change that to a for loop.  And the initializer of that for loop is right there.  And the incrementer of that for loop is right there.  I think that will still work.  And I don't need these ugly braces anymore.  Let's get rid of those.  Well, that's a nice pretty little algorithm, isn't it?  I wonder how it works.  Let's see if it does work.  Okay, we need some big number with lots of prime factors.  Two times two times three times three times five times seven times 11 times 11 times 13.  That's a bunch of prime numbers.  Is a list of, let's take that and move it down.  And we'll say two comma two comma three comma three comma five comma seven comma 11 comma  11 comma 13.  Oh, we're done.  We're not actually done.  Well, we're done.  There's a small improvement you can make to speed the algorithm up in the case of very  large prime numbers.  You don't need to loop while n is greater than one.  You can loop until n is greater than the square root of the original n.  But and that'll speed things up by a fair bit in the case of very large prime numbers.  Other than that, however, this is the algorithm.  Notice that it has nothing to do with finding a bunch of prime numbers and dividing through  by prime numbers.  It's an entirely different algorithm.  Where did this algorithm come from?  Well, it came out of my brain, obviously.  I was sitting at the kitchen table.  I developed this algorithm one little test case at a time.  It's not like it magically happened.  And yet it did not come about because I thought it through.  I did think it through.  I just didn't think it through beforehand.  I thought it through while I was developing it.  One little test case at a time.  This implies that it is possible to create algorithms incrementally, to solve algorithmic  problems incrementally, one test case at a time, if you follow the rule that everything  you do to the production code becomes more and more general.  If you think about that long enough, you'll realize, yes, the tests get more and more  constraining, the code gets more and more general.  At some point, it gets general enough to pass every test.  Is it possible to derive algorithms incrementally one test case at a time?  In this case, it certainly was.  Is it generally possible?  Let's do a thought experiment.  You and I?  The sort algorithm.  I'm going to hand you an array of integers.  I want to know what test cases you would pose in order to pass a sort algorithm and what  changes you would make incrementally to that sort algorithm to make them pass.  We will begin.  What is the very first test case?  No integers at all, an empty array.  We always start with the most degenerate case.  The most degenerate case is an empty array.  Sorting an empty array is trivial.  How do you solve it?  You return an empty array.  Okay.  First test case is now passing.  Second test case.  One integer doesn't matter what the integer is, it's already sorted.  This fails.  It fails our current implementation, which returns an empty list.  How can we make it pass?  Return the input array.  Notice how similar that was to returning the N.  We had a 2 at first, then we returned an N.  In this case, we will return the empty array.  Interesting.  Now, we will return the input array.  Next test case.  Two integers in order.  Passes.  Next test case.  Two integers out of order.  Fails.  How do we make it pass?  Switch them if they're out of order.  So there's an if statement.  Oh, that's similar to this, isn't it?  An if statement.  And if that if statement fails, we swap them.  Remember you said that.  Right?  Yeah.  Remember you said that.  Passes.  Next test case.  Three integers in order.  Passes.  Three integers with the first two out of order.  Passes.  Three integers with the second two out of order.  Fails.  How do we make this pass?  By making the code more general.  You put the if statement with the swap into a loop.  And you loop through the array, swapping if they're out of order.  Passes.  Three integers in reverse order.  Fails.  Because you swapped the first two, you swapped the second two, but the first two are still  out of order.  How do you make that pass?  Put the loop that compares and sorts into another loop.  That's actually one element shorter.  What have we invented?  All sort.  The worst possible sort algorithm.  So maybe test driven development is a great way to derive terrible algorithms.  But this guy here gave us an answer to one of those early test cases.  Two elements out of order.  And he said compare them and swap them.  There was a different way to make that test pass.  It was not necessary to compare and swap them.  What you could have done instead is to compare them and then return wholly new arrays, allocate  brand new arrays out of whole cloth and put the elements in the right order.  Not changing the existing array, creating a new array.  Those of you who attended my last talk will recognize that as the functional solution.  Creating the elements inside the array is non-functional because we're using assignment  to do it.  Now, it turns out, and I won't do this derivation for you.  I'll let you do this at home because it's fascinating.  It turns out that if you take that course, you put in the if statement and you create  completely new arrays, you very quickly wind up at a quick sort.  The quick sort falls out.  The quick sort is probably the best possible algorithm.  So apparently, there are forks in the road.  As you are writing tests, there are choices you can make.  Maybe two or three different ways to make a test pass.  And if you choose the right course, you get to a better algorithm.  What criteria can we use to make that choice?  And it turns out, there are a whole list of possible things that you can look for, but  the first thing to look for is assignment.  If your solution involves assignment, find a solution that does not.  If no such solution exists, then you have to go on with assignment.  But if you can do it without assigning, you may wind up at a better algorithm.  This is called the transformation priority premise.  The premise that production code can be transformed into ever more general states.  And there is a priority to the order in which you make those transformations.  You prefer transformations that don't have assignment in them.  There's a whole bunch of other preferences.  If you look up the transformation priority premise, you will see a list of these possible  predicates that were used for transformations.  You'll find the documents on this interesting.  They cover the sort algorithm.  They cover Fibonacci.  They cover a bunch of algorithms.  And it's something interesting to try on your own.  Which brings me right to the end of the talk so I can take one question.  Anybody out there want to ask a question?  Yeah, in the back.  So the cycle in test driven development is supposed to be red, green refactor.  Yes, that's true.  And your question was that sometimes my tests passed immediately.  Yeah, that does happen from time to time.  Sometimes you pose a test that you think will fail and it doesn't.  It passes.  Okay.  Then you don't have to go through the rest of the cycle.  You thought it was going to be red, it turned green.  That's a benefit.  It doesn't happen very often, but it does happen.  And with that, thank you all for your attention.  See you next time.  Thank you.
Robert_C_0_21503.png Robert_C_21504_21551.png Robert_C_21552_21599.png Robert_C_21600_49283.png Robert_C_49284_49355.png Robert_C_49356_49655.png Robert_C_49656_53819.png Robert_C_53820_53867.png Robert_C_53868_54251.png Robert_C_54252_54395.png Robert_C_54396_54959.png Robert_C_54960_55235.png Robert_C_55236_56027.png Robert_C_56028_56363.png Robert_C_56364_56555.png Robert_C_56556_56783.png Robert_C_56784_57023.png Robert_C_57024_57515.png Robert_C_57516_57731.png Robert_C_57732_58415.png Robert_C_58416_58739.png Robert_C_58740_59063.png Robert_C_59064_61943.png Robert_C_61944_62027.png Robert_C_62028_62507.png Robert_C_62508_62795.png Robert_C_62796_62867.png Robert_C_62868_64235.png Robert_C_64236_64667.png Robert_C_64668_65183.png Robert_C_65184_65723.png Robert_C_65724_65819.png Robert_C_65820_65915.png Robert_C_65916_65987.png Robert_C_65988_66035.png Robert_C_66036_66839.png Robert_C_66840_66995.png Robert_C_66996_67115.png Robert_C_67116_67367.png Robert_C_67368_67691.png Robert_C_67692_67823.png Robert_C_67824_67955.png Robert_C_67956_68663.png Robert_C_68664_68711.png Robert_C_68712_68903.png Robert_C_68904_69287.png Robert_C_69288_69515.png Robert_C_69516_70115.png Robert_C_70116_71375.png Robert_C_71376_71627.png Robert_C_71628_72083.png Robert_C_72084_72155.png Robert_C_72156_72263.png Robert_C_72264_72527.png Robert_C_72528_72635.png Robert_C_72636_73115.png Robert_C_73116_73511.png Robert_C_73512_73811.png Robert_C_73812_73979.png Robert_C_73980_74315.png Robert_C_74316_74903.png Robert_C_74904_75143.png Robert_C_75144_75551.png Robert_C_75552_75647.png Robert_C_75648_76187.png Robert_C_76188_76451.png Robert_C_76452_76751.png Robert_C_76752_77375.png Robert_C_77376_77447.png Robert_C_77448_77567.png Robert_C_77568_77687.png Robert_C_77688_77747.png Robert_C_77748_77975.png Robert_C_77976_78539.png Robert_C_78540_78851.png Robert_C_78852_79331.png Robert_C_79332_92303.png Robert_C_92304_92356.png