Agile Kaizen
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Bellware, Scott
Kaizen is the unstated, ever-present cornerstone of Agile Development, but it's if often seen as distant and abstract thing rather than one of the tangible Agile practices, like standups, testing, or estimation. Seen as a "soft skill", Kaizen is in fact it's the hard science, the central motivator, the justification, and the traffic cop of other Agile practices. This presentation delivers Kaizen without the typical esoteric language, and re-casts it as the tangible, central aspect of Agile Development that all other practices pivot around. It presents the tools and techniques of Kaizen that you can put to work in your team right now in the pursuit of ever higher levels of performance.
 Good afternoon. Thanks for coming. In the best case scenario, there are a number of things that I'd like to accomplish with this presentation. First of all, I'd like to challenge some perspectives of the things that we choose to focus on and things that we deem relevant as well as the things that we tend to dismiss and deem irrelevant and challenge some of those ideas, especially in regards to current methodology and agile development. And to recognize the things that we have deemed unimportant or out of our control that are actually crucial within our reach and within our control and that the promise of getting these things under control is a promise of greater prosperity. So this is a chart of your typical software project and our ability to make progress on it. Most software projects progress from this point of the beginning where we have enthusiasm to this point in the end here where we have a rewrite passing through a corridor that we could call the pit of despair and denial. What's decreasing here over time with this graph is showing is the decline in productivity or the decline in the ability to get anything done. It's typically when things start getting to the point where we can't get things done, where it starts getting difficult that we get into the denial, then we get into the despair where it becomes obvious and then of course we just get to the point of declaring bankruptcy and declaring a rewrite. We call this the software crash. And what happens here is that all the changes that we've made over time in the system have made it more and more difficult to make subsequent changes in the future at the same rate that we were able to make previous changes. Scrum hasn't changed this, Agile hasn't changed this, Kanban hasn't changed this, Test Automation hasn't affected it, Continuous Deployment hasn't affected this, Pivotal Tracker hasn't changed this, Trello has affected it, Version 1 hasn't affected it. All the things that we're doing haven't stopped the fact that software still has an end date and it's usually much earlier than it's supposed to be. And that's because things are out of control. The things that we are controlling with all of these countermeasures in Agile are not actually the things that need to be controlled in order to stop projects from failing or evolving into this pit of despair. So you can also think of that graph as the loss of control graph. It shows our ability to make progress over time, which ultimately means that progress and productivity are related to control. So if it was your goal in your software effort when you started to achieve this collapse, this failure, if that was your goal then your project is under control because you achieved what you expected. If this is not your goal then you could clearly surmise that things are out of control. And we can probably say that most software efforts are out of control. And the things that we're controlling are not the things that give us an ability to avoid the crash. The other side of this is that there's no reason why software shouldn't continually get better. Software, there's an old Agile story, an Agile adage about somebody who's restoring furniture and who goes into their workshop every weekend in their spare time and continues to work on this furniture. And after a long time of working on the furniture they come out and the product is worse than when it started. We would think of that as absurd. But it's not absurd with software. We basically think of that as normal. So if we start to change our focus from the things that change the shape of the curve from a downward trend to an upward trend then whenever we start to control the things that steal this productivity. This talk is called Agile Kaizen. I'm going to talk about Agile but we'll do the Agile part later. But just to set a baseline I want to say what Agile is for me, what I think of this, how it works for me. I think of this as an ability to make changes now in a way that don't obstruct me from making any changes in the future. So for the rest of this for a little while we're going to focus on the Kaizen part and then we'll return to the Agile. So the reason why you may have this instead of this is this. And we can call that complexity. Complexity is a mathematical thing. It's not something that is emotional. It's not something that has to do with difficulty or easiness or lack of easiness. It's simply something that you can measure in your software or something you can measure mathematically. If you put complexity that is constantly increasing over productivity that should be constantly increasing the best you can hope for is this which is a constant ability to make progress. And that's okay. It's okay that this is a flat line because this is just about the best that it can possibly be. If there's never more increasing demand and pressure and you're still able to sustain the rate of progress then you are inevitably improving at a faster rate or at least as faster rate as the things that are trying to drag the project down. And the rule here which is an old lean rule or not really lean but there's a quote from an old Toyota guy, I say old, somebody from the past that if you're not moving forward you're falling behind. The reason for this is that the more the time progresses the more moving parts you have in your system the more easy it is to make mistakes and mistakes in software are often invisible. If you don't counter this counter productivity you're going to end up with this crash. Whether you end up with this or this it's what you focus on that makes a difference. And where you end up depends entirely on where your focus is. So to get this from this we need to add something, some kind of anti-counter productivity or whatever that is. And the more you add of this the more you're able to sustain a steady level of progress. Or said otherwise in a system that's continually growing which most systems are you've got to keep getting better just in order to make progress at the same rate. So let's go back to Kaizen. There's a common English term for Kaizen. This is the only time I'll ask for audience participation. It's a pretty common term so anybody who knows it just shout it out. Just shout it out. Yep. What were you going to say? Right. So exactly. So this is what I was looking for. Right on. Continuous improvement. This is the common colloquial term we have for this. Here's what the encyclopedia has to say about Kaizen. The Sino-Japanese word Kaizen simply means good change with no inherent meaning either to continuous or philosophy. In Japanese dictionaries or everyday use the word refers to any improvement one time or continuous, large or small in the same sense that the English word improvement. However given the common practice in Japanese of labeling industrial or business improvement techniques with the word Kaizen because there's no specific Japanese word that means continuous improvement per se in especially case of oft-emulated practices spearheaded by Toyota the word Kaizen in English is typically applied to measures for implementing continuous improvement or even taken to mean a Japanese philosophy of continuous improvement. So really it's just jargon. And when we talk about it in terms of jargon we say continuous improvement we end up talking about this idea of a soft side. And when asked what Kaizen is people often say it's trying to do a little bit better every day. Which isn't really a hard science. It isn't a hard practice. Agileists might say it's what I'm doing when I'm doing agile retrospectives. And because these explanations are common and because they seem to be so fluffy this kind of squishy stuff that maybe HR people or scrum masters want us to do we technologists we scientists often dismiss it as bunk. I'd like to suggest that this limited view of Kaizen comes from viewing it through a limited lens and that there's much more to it. For example this word continuous improvement this jargon is much more accurately represented as how it's actually done which is closer to relentless improvement rather than continuous improvement. It's not improvement when it's convenient it's improvement always especially when it's inconvenient and irrespective of when it's inconvenient. And in a Kaizen culture we don't just wait till we have time to do Kaizen like we do with continuous improvement which is a which is a which is an event. There's no event in Kaizen. It's the way things are done. It's the air thing that is breathe. It's the water that's between the tissues that holds everything together. Even the soft side of Kaizen is as hard as that. But let's talk about the hard side. The work of Kaizen is just as tangible as everything else you do like daily stand-ups, automated testing, iteration planning, any other concrete practice you have. And the hard side of Kaizen you could say is a science. It's a science of restoring productivity. You might then say that Kaizen is this anti-counter productivity. So let's quickly divert into a summary of things that I think we already know about software. Are you guys familiar with this old saying garbage in, garbage out? This saying is as old as computing. In all software programs you're essentially made up of three things. An input, a process, and an output. And if the input is good and the process is good, then the output is probably good. If the input isn't good and the process is still good, the output is still not going to be good. And just as a sidebar, if you put good stuff into a bad process you're going to get bad stuff out of it. But the point that I'm making here is it's very hard to go from bad input to good output. What would it take to go from bad input to good output? You'd have to add a lot of energy, a lot of effort. And even then you may end up with an outcome that is just less stinky. Not really something you can say is good, just a little bit less undesirable. And unfortunately a lot of stakeholders would say hey, good enough. And in Kaizen and in Lean we say good enough is not good enough. If this is what you're trying to avoid, then the idea of good enough is a primitive industrial perspective that is ultimately the reason that this crash happens. Lean saying Kaizen, that the goal in fact is perfection. And before anybody retreats to the old adage that perfection is the enemy of good, I'm talking about perfection, a direction that we're going in rather than perfection, a destination. So we've revisited input process output. We know that garbage in leads to garbage out. I want to talk about a superstation of software management then. Most software management and most software development and most software developers and software workers focus on this. This is the part where we do the process and the delivery. This is my work and getting it done. This is the just ship it part of software development. This has been one of the most harmful slogans that we've adopted in software development. This is the get my work done perspective. The just ship it stuff. This has been harmful to us. And part of the reason is that this is really the part of the process that we should be focusing on. This is the part that Lean and Kaizen are focused on. And the reason for this is that garbage in is garbage out. And there is in any work a series of inputs, processes and outputs. Every output is the input to the next stage of the process. So we work in teams. We transfer work to each other. When we feel where our work is done, we've handed it off. And we feel that we are making cumulative, cumulative, that all of a sudden seemed like a funny word. Feel that we are making cumulatively more progress simply because we are doing work and finishing work and shipping. And it's easy to get a false sense of progress when our evaluation of what a complete task is ends with our own job function. That is to say, when we view our view is limited to just getting our part of the job done and just shipping it off to the next person in the Bucket Brigade. Frequently the reality is that as we are handing off work, something is getting lost. We are handing it off and we are leaving something out of the delivery. We believe that this is happening, but we can observe in reality that over the life of the effort, this is happening. And here is what's happening. This is a graph, conceptual graph, of the progress that we make. When we transfer work between one function or one person to another, what's happening is we lose the momentum at every handoff that we gained in the previous stage. So as we hand work from one person to another, at every handoff, the progress that we made, the speed that we gained, the momentum that we gained, as soon as we handed off to somebody else, we lose that progress and the next person starts from scratch. Instead of having a cumulative increase in momentum, we basically end up with a net, well, we end up ultimately back where we started. So the key focus here is momentum transfer. So when a transfer between two people happens, the person on the receiving end suffers delays. That person must figure out how to proceed with their duties and with their work. And once they figured out how to do their work, they ultimately can get down to business and do their work, but we don't pay attention to this downtime and I'll submit to you that if you do a basic experiment and measure how much of this downtime is present in your work, you might find something rather surprising. So something is lost in the transfer as evidenced by the delays in starting work at the beginning of a recipient's stage of receiving the work, then we should be able to get some measure of that delay. So here's something to try and here's something that I've done on a number of teams. Every time you start a task, any time any team member starts a task, log the time. And when you get stuck, log the time. And when you get unstuck, log the time. And when you step away from your work, log the time because that's not part of the metric. And at the end of the day, tally up the total time you spend making progress and the total time you spend not making progress. Now the interesting side effect of this is the team also has to have a meaningful conversation about what progress is and what progress isn't. And that's not something that should be defined for you. You should really, it's a meaningful exercise to do this. But you should know that some things are not progress. Trying to figure out somebody else's work is not progress. The reason for that is they've already figured it out and somebody else doing it is rework. That's counted in the lost side of the balance sheet. We call these progress outages or just call them outages. And a good reason for this, this brings the focus of the stark reality to the business, what's happening in the business. This is a de facto outage. And when we expose this notion of outages, when we classify them as outages, the business can become motivated to make improvements to resolve these outages. And the good thing about this measurement is that there's no shortcut around it. Or as Deming said, it does not happen all at once. There is no instant pudding. You're going to have to work at getting this, but there's no way around it. You can't just make out, you can't just take shortcuts and then end up with more outages. The outages will never go away even if you try to invoke some expediencies. Those actions that are taken will further exacerbate the problems and they'll show out on the outage logs as more outages. So in the face of further shortcuts, if you see outages and you have a management reaction to have some expedites or some shortcuts, ultimately what you'll see is that we're still trending towards the crash. An alternative of this, if you find it too heavy handed to log your time, because that can be taken incorrectly, it can be taken as something forceful and something oppressive by workers, then just log the incidents of the outages. Log a description of them and that they happened. And report these outages during your team sync up. So whether you do a daily scrum or something called a stand up or whatever your practice is, whether it's daily or not daily, whenever this happens, report them. Make them very visible. Make it really clear so that the team starts to understand, scrum masters understand, stakeholders start to understand that there are outages. And ultimately we're going to bring awareness to these things. And what you'll find in the end typically is that most developers spend most of their time trying to figure out how to start their work than actually doing the work. And further to that, you can think of many small tasks that a developer or any worker is doing in the process. Not just a story card as a task, but any individual task. The time that we take trying to figure out how to fit our work into the existing system, most of our time is spent doing that. That's lost time. That's outage time. So dealing with the root cause of this pervasive problem means that we change our focus to the input stage. It means the job isn't done when we just ship it. The job is only considered done when the next person in the process has regained the momentum that the person in the previous stage has. So let's think about this from the terms of a relay race when somebody is passing a baton on a relay race. We don't really consider the passage of a baton on a relay race. Well, let's just say how it's done. When it's done, there's a moment where both participants' hands are on the baton. And the person passing the baton has the greater responsibility in successfully navigating and executing that handoff. In software, this is typically what we do. This is called a push system. This is what happens when we're focused on just ship it. I do my work and I drop it off. And the next person is not my responsibility. Work isn't done until the next person in the next stage is enabled to work at the same pace as the person in the previous stage. So in a push system, we just drop things off in a pull system. The handoff isn't done until the recipient says it is. Sure. The recipient can say I'm ready to go. Can simply be I understand enough to proceed without you being here. And it's just that simple. But the interesting thing about that is that time should actually be measured as well. Because we can start to have interesting, interesting, we can get some understanding of how long it takes to quote, get somebody up to speed. And I don't like this term get somebody up to speed because it trivializes the meaningful nature of this interaction between two workers. All of it. In fact, the most frequent handoff is from yourself to yourself at a later date. For a programmer, that's true. Programmers more often handoff work to themselves at a later date and still suffer the downtime. So we say the work isn't complete if the work transferred to the next person causes an outage in that stage of the work. So when we handoff the work, it means that both participants are hands on the work. And it's the recipient who was authorized to declare that the handoff is complete. And the person making the handoff is obliged to continue with the work until the recipient's momentum is raised to the level that existed prior to the handoff. And work is basically considered incomplete until the transfer of momentum is accomplished. There's another word for incomplete work. Work in process. And those of you who've heard in Kanban, this is a salient issue, so I want to present it to you in this light. Incomplete work is work in process. The purpose of Kanban is to create a tightly synchronized series of handoffs where there's no slack between the participants. And the reason for this is really critical. In a push system where work is just dropped off for another recipient to pick up later, the work product is in fact not validated to be complete or sufficient. That means that the work piles up, work that has been dropped off can pile up, and has not been validated to be free of flaws. And when flaws exist that are not detected immediately, they're going to be built upon. And they become encased in software. And then that software is built upon. And once that happens, the software flaw will never be removed. It'll be impossible to fix. And the quality of the software will never exceed that particular level. And by flaw, I mean design flaw. I don't mean something you can find with an automated test. That's the easy stuff. I'm talking about things that are fundamental violations of structural design principles that dictate whether you'll have this or you'll have this. You can't discover these things with automated tests, but you can discover them with test driven development. And in answer to the inevitable question as to whether or not test driven development is dead, my answer is this. Only if the fundamental flaws of structural design physics have been suspended and design principles no longer matter, then test driven development is dead. Sometimes developers can't even see these flaws. So if the input materials are flawed, then the processing is going to be troubled, and then the outcome is going to be troubled. And the best way to validate the input materials, whether or not they're flawed, is to execute the handoff with both participants on hand. It's very simple. If the recipient of the handoff has difficulty understanding how to start, we should reconsider one of two things. Well, there's two things that we need to think of. The quality of the design, the clarity of the design, and the level of capability of the recipient. And we have countermeasures for both of those. And the person doing the handoff, as they are the person that have done the work, are qualified, capable, and should be willing to bring that person's skill, knowledge, and capability up to a level where they're capable of working on it. So you'll be able to understand, you'll start to get an understanding of the effectiveness of your design in relation to the, its effect on the human workflow by measuring what's happening at the handoffs. So you want to go from this to this. You don't want to end up with this. That's what we're after. So the purpose of Kanban then is to create a highly synchronized workflow without slack in the handoffs so that handoffs can be used to highlight problems in the design. And another way, the purpose of Kanban is to show you where to apply Kaizen. And if you're doing Kanban, simply to put stickers on a wall and work in process limits, there's really not a hell of a lot of value in that over simply just calling it scrum or XP or work. Let's talk about technical debt. I think this has been one of the most harmful ideas in software development. And here's why. All technical debt is in complete work. So for anybody doing Kanban, what value are your work in process limits for every stage in the work when you are not counting technical debt as work in process? This is a very clever accounting system. Usually in accounting systems we'd call that cooking the books or fraud. And all in complete work is hiding design flaws. And if we build upon those design flaws, we won't be able to fix them. And design flaws are the root cause of this. If it's your intention to end up with this, then we should allow technical debt to be a legitimate allowance of your production system. The problem with technical debt is that we gave it a name and now we think it's okay. That's like saying because we gave syphilis a name, we should all go out and get it. And if you really, really need technical debt to exist within your system, and sometimes you do, it's because you've burned through your safety margin because you haven't paid attention to your safety margin. And you've done that likely because you felt that necessary improvements are really too insignificant and too negligible to require investment. So all teams have a capacity. This is the high water mark beyond which they cannot make any further efforts. This is the absolute limit of how much they can work. And there's amount of work that, oh, I'm missing a label, unfortunately. Yeah, let's do that. So that yellow line is the safety limit. The line between the red and the yellow, that's your safety margin. And that green line is the current level of work. The safety margin exists to allow for handling of unforeseen circumstances without degrading the system or the team's abilities to respond to it without driving it further towards the pit of despair. Most software managers don't have a notion of the safety limits. Not that they don't manage it. They don't know that this technique or this topic is relevant. It's not president in the day-to-day management work or thinking. Some don't recognize it as significance. And as a result, most teams operate past their safety limit and they push their work into the safety margin. They consume the safety margin as if it's an allocatable resource for day-to-day work. And when something unexpected happens, there's no available capacity to handle it. The inevitable result is this. This is what happens when the safety margin is misappropriated as a resource for some other purpose. Let's talk about the design. Say a design has a good outer service. It's something that you can build on. And as you go forward, you build upon what you had put in place previously and you continue to add to the system. And then you get a design flaw. This is technical debt. This is where you build over the flaw. This is technical bankruptcy. This is where you build over the thing that you built over the flaw. At this point, it's practically impossible with the budgets that we usually operate on to fix this flaw. And if you build on the flaws as if they're the foundations for the future work, then you can expect your foundation to be flawed. And if you do that, this is what you get. This is what happens when design flaws are encased in construction. So think of design flaws like cells, like living cells. They fit together nicely. They work together nicely, cooperatively, with their adjacent cells. Sick, misshapen, deformed cells cause their adjacent cells to be deformed. A unit of software that is not correctly designed, everything else that interacts with that software will take on some of allowances for the fact that that thing that they're using is not correctly shaping. That means all the things that are adjacent are also going to be poorly designed and this spreads out. So the worst software problems exist on a cellular level. The problem, you could say, is in the composition of the sand, not in the composition of the bricks or the construction of the house. If the sand is flawed, you can pretty much bet that the integrity of the house is questionable. Software problems are more like cancer then. You often don't see them with the naked eye and the untrained naked eye certainly won't see them and may even deny their existence, mostly because it's invisible to them. The question is, how much cancer would you personally be willing to withstand? You want your systems to grow healthily from seedlings to saplings to some mature creature. And the twisted nature of software that has cancer is that it's not visible to users. A user's experience can go undisturbed while a developer's experience is already demonstrating signs of the impending doom. The level at which users interact with the software doesn't display the problems of productivity of working with the software. Software can work even while it's not workable. So we have to separate our perspective on, well, you have to separate your perspective if you want continuity of your business that's based on software. If your goal is a crash, then you don't really have to worry about this. And for a while there in all systems, both the users and developers have had a happy, healthy, enjoyable experience of the system. And then developers will start to understand, they are the early warning sign. Developers, testers, operators. I'm saying developers mostly as a bias because I'm a developer. But the whole team, anyone involved, anyone hands on. And as you raise these messages up through the ranks, you might have brought these concerns to your boss about what you're seeing on the horizon. But anyone that isn't hands on in the work won't be able to see the same thing. And ultimately, many of these concerns are dismissed. This is when good people leave, when projects become unworkable. And that's when software dies. So let's talk about why this happens. It happens because mistakes were made that could not be repaired. This happened. So how do we deal with this? You have to learn how to make fewer mistakes. You can learn how to make fewer mistakes by improving relentlessly. But here's the crux issue here. Developers often don't know what a mistake is. And even when they're experiencing the mistake, even when they're experiencing the injury, developer cultural issues can motivate developers to not admit that it's happening. So the ultimate goal here, the ultimate countermeasure, is that developers have to master design principles and their application. That should go without saying, but it's quite rare. And here's a complication to this. In mastery, let's say that those first few, that these vertical lines represent steps in a progression of knowledge or capability or career or talent or skill or whatever you want to call it. It's not just seniority. It's just a measure of the passage of time. And people can pass time without getting better. This is a representation of people getting better. Each subsequent step towards mastery tends to be much more subtle than the last step before it. So if I'm talking to a junior developer who is going from this step to this step about why it's necessary to change the design from this subtle step to make this subtle step, they will very likely see this as negligible and irrelevant, not worth the investment. If I speak to a business stakeholder about this, well, frankly, one, they may not understand me at all. And two, they absolutely will not understand the significance. But here's the thing about the mastery curve. As you make successively more fine refinements in your skills in a good direction, in the right direction, your capabilities or the capabilities of the team increase exponentially. These small changes, if they are the right changes, will increase the capacity of the business at a geometric rate, which is the countermeasure to the geometric rate at which software is naturally wanting to degrade, which says what Kaizen is saying is that all of the things that you are saying in the just-shippet mentality that don't need to be done, these words that we use like bike shedding, yak shaving, navel gazing, these are all ways of delegitimizing a meaningful conversation about making meaningful progress towards an evolution of skills that are hard to see from a distance, they're hard to get, they take time to work on, they take a lot of practice, but yield the highest value in software development efforts. So we can say about Kaizen then, it's a relentless pursuit of mastery whose goal is to restore the safety margin so that you can deal with those unanticipated situations without any sacrifice to the prosperity. I thought I had a lot more content, but it went very fast. So this is where you can reach me. My illustrations were done by my wife. If you are interested in custom illustrations for your presentations at work, please drop them up or align. And if there's any conversation that you'd like to have, then I say we get into it. Sir. Do you have any other examples of projects that abuse this approach and actually demonstrate for you? Do you have any claims that will improve your stuff? Yeah, well, I've done these in my projects. I've done them only with teammates who are interested in doing it. And I've done it for years with myself and my own data. And it takes a good long time to produce good hard metrics. But what we do get out of this early on is indications from, and I don't think this is necessarily science, but good gut feelings, which can be wrong. But over time, we've seen improvements in the, at least in the reduction of incidences of outages, which I think is a really good first level, first goal to hit, even if we're not logging times and handoffs and stuff like that. That's the ideal. But if we're just seeing reduction in outages, ultimately what we're seeing is the ability to do work without downtime. So yes, I've seen better outcomes. Sir, if I can have a follow-up. Sure. What design practices do you recommend? Nothing that hasn't been known for decades. So aside from practices, which, so I'm going to do, I'm going to do a presentation tomorrow on test driven development. I obviously feel that that's a valuable thing. But I also feel that it's incredibly misunderstood what that serves. And it really, it's to support design. It's to evaluate whether design principles are being respected. So I'd rather talk about the design principles that I think that matter. And I think that they're the tried and true things. So I'm not even going to use the common words for the design principles, just on the off chance that I would say them and they would put you right to sleep and you'd gloss over because you know. The most important thing I think in software design, to keep software design itself workable and productive, is to avoid entanglement. So I'll call it entanglement. That's pieces of software that are tied to other pieces of software so that if you change one you must change the other. And there are certain things that we do commonly that we think are good, that we believe are good as developers, as testers, as project managers, that we think that this, a lot of it is rooted in tooling. In fact, we spend more of our efforts and our mind share on tools rather than on techniques. So a lot of the tools we use are directly correlated with a lot of these problems. Object relational mapping is going to cause you the kinds of entanglement problems that even though they look perfectly fine because the tool is encouraging them, are ultimately causing work to continually slow down over time. So that's a rough thing. That's a rough thing to contend with. I found that tools like Cucumber, new generation testing tools often contribute to more entanglements in software, more software maintenance, more difficulty in maintaining software. I think that they're much more effective ways of offering tests and acceptance tests that don't require this tooling. In fact, it requires a lot more lightweight tooling, a lot of things that are easier to learn and easier to maintain. We look at the term, we have this word testable. Testable means something. And it means something about design. It's a design quality. It's not about whether or not you can run a test or something, but it's the difficulty in running the test. In fact, it's the difficulty in setting up a test to be run that demonstrates whether something is testable. The difficulty in setting up a test to run tells you how much entanglement you have in your software. And the more entanglement you have in your software, the more things that need to be set up before you can start to learn how your software works. That is the crux of the loss of momentum. The faster you can learn how your software works, the faster you can get to work. That's all there is to it. The more obstructions you have to understanding how your software works, the longer it takes you to get your job done. So we can say that the only thing that we're trying to do is to improve clarity. And the only thing that's lacking in software that is heading towards the pit of despair is clarity. And it's really, really difficult to broach this subject with software development, people with testers, with project managers, because we're so focused on these things that we think are helping us go faster. And really, a lot of these things help us go faster in the very beginning of a project. And the very beginning of a project is the easy part. We shouldn't be adding tools to make the easy part be easier, especially if those tools make it harder to do the subsequent work. There's a reason why there's renewed interest in functional programming languages. It's directly related to this issue of productivity and of clarity. And the only thing that is boggling or bogging down software projects is the inability to understand the code. A lot of these things that we think are team problems, interpersonal problems, conflicts, and all of them, but a lot of them can be traced directly back to lines of code. Difficult relationships on teams can be traced to the stress that comes from the inability to get work done. Now, a lot of programmers may be too busy to express it that way. They may not have the patience. There can be issues with emotional maturity in expressing it that way. But at the end of the day, a lot of the things that we think that we need agile interaction remediations for can be fixed if we just do what we're supposed to do anyway, which is make the project go as fast as possible. And that's done by solving the design. But that's done by challenging paradigms, tooling paradigms. And that is a very big challenge. I've been trying to do that for many years in the.NET world. That was during the time where we're bringing up agile in.NET and bringing up open source in.NET. I work in the Rails world now, and the problem is exactly the same now in the Rails world as it was in.NET in 2007, that there's a sort of orthodoxy of the way things are done. We believe the tooling is good. We don't question whether the tooling is good, but we can actually evaluate the code and mathematically demonstrate that the qualities of entanglement are all over the place. It's just that we have to recognize that that's the problem. That was a long cop out to your question. I think the worst thing to do would be to take the suggestions I've made to a team and require them to do it. What I've done, I'll do it myself. I've got three years of data. I'm just interested in this because I've been interested in why it's so damn hard to get anything done in software development. It's not supposed to be that way. When I've talked about it with other folks in organizations I've worked at or clients, I'll mention it. Some folks who are interested in personal performance get into it. One of the things I think that's been effective for customers that I work with who are struggling with time is to just track the outages. Forget about the measurements. I like the measurements because I'm interested in seeing what it means over years. If it's just tracking the incidents of outages, how many a day, whether you had them today, where these outages tend to concentrate in the system, that's a really simple thing to do. It also allows developers the freedom to say, this is a really challenge. This is where my time went. It frees them to say, I didn't get anything done and here's why. I'm going to call that an outage. We're going to change our practices so that talking about and exposing the outages is part of the everyday work. We don't let ourselves look away from that until we get them fixed. Sooner or later, it tends to bubble up and managers and stakeholders and senior managers end up understanding that at least these delays are not due to what folks often think it's due, which is people being lazy. I would say do it with a very, very light hand and don't make it a requirement. Make it a matter of personal interest or part of the adventure, part of the exploration. Once if anybody leaves this room and goes back to their team and says, here, everybody, you are now required to log your time, fundamentally misunderstood it. I'll be highly motivated to visit you if that was the case because it's really, really not. I would understand the motivation to do it, but it would be a great disservice to the advances that you're trying to make. It would work against you. Ask them or do it yourself and demonstrate. I've done it and just started on some consulting engagements. I've just started talking about outages. Here's what I did yesterday and here were my outages. Then it starts to, if I can see that it's not me, if there's no resistance to it, then I'll continue doing it and see who takes it up and then people will ask me about what it is and what that means. We have a more meaningful conversation. Anything else? Yes? I like your statement about perfection in the direction, not the end goal. Great. That's also my way of working. But I noticed that a lot of people really reacted that and little improvements are made in the way that you're talking about. Try to have a conversation about it. I try to steer the conversation more towards how can we know that they're negligible? Then I present to them some ideas about design principles and some of these ideas. It ends up being a qualitative conversation because it's really hard to have a quantitative conversation about that. In fact, it's hard to put the practices in place to get to quantifiable measurements until people are comfortable with the qualitative aspects of it. I deal with it with a great deal of pain because I would rather that folks would have had the benefit of some of the experiences and teachers that I've had the benefit of and some of the perspective I have. You just can't put your experience into another human being. That's hopefully what constructive conversations are about. I would say, this is making me think of something else, we have a really difficult time leveraging conflict in a meaningful way and to practice conflict as an honorable thing. We tend to say it's bad, it's destructive, it's avoidable, we should avoid it at all costs. I don't think that's good. I think that from a Kaizen perspective, there's a lot of things that we do that are not solving root causes. We're bad at conflict, we should get better at conflict. There's a number of things like that in Agile. Why do we stand up, I'm sorry, I'm going off the subject of your interview. Why do we stand up and stand up meetings? Does anybody stand up during their stand up meetings? Okay. Does anybody remember what the reason why we stand up? Keep them short. Right? There's a reason why they would not be short if we sat down. Right. So if that's the problem, which we'll call that an attention problem, why don't we work on that? Why don't we all sit down, allow the problem to exist, measure the incidence of the problem, and find a meaningful way to solve the attention and focus problem. If we can fix the attention and focus problem, it means that we've improved ourselves in a very deep way. It means that we're attacking something fundamental, not just about our work at, you know, in doing a stand up meeting, but all of the work that we have. It means that we've sharpened our alertness, increased our attention, created our focus, probably increased our compassion and our understanding. So a lot of these things are crutches, they're prosthetic devices that we use to put in place that are actually blocking more deeper improvements. There's a number, you know, off the top of my head I'm blocking, because now I'm stressed to find them all to look like I've thought about this for years, which I have. But suffice to say, everything that you do with agile development should be considered a specific countermeasure for a specific problem. You should use a countermeasure, every tool, stand ups, sprint planning, retrospectives, story cards, electronic tools, testing, test driven development, automation, all of it. Every single one of things is meant to solve or address a specific problem. When we take these things and we use them irrespective of the knowledge of the problem, we're kind of behaving, no we're not kind of behaving, we are de facto behaving superstitiously. And I think that's the opposite of behaving like people who are really, really engaged and interested in solving deep problems. I think that's it. Thank you very much. Thank you You
 Good afternoon. Thanks for coming. In the best case scenario, there are a number of things  that I'd like to accomplish with this presentation. First of all, I'd like to challenge some perspectives  of the things that we choose to focus on and things that we deem relevant as well as the  things that we tend to dismiss and deem irrelevant and challenge some of those ideas, especially  in regards to current methodology and agile development. And to recognize the things  that we have deemed unimportant or out of our control that are actually crucial within  our reach and within our control and that the promise of getting these things under  control is a promise of greater prosperity. So this is a chart of your typical software  project and our ability to make progress on it. Most software projects progress from  this point of the beginning where we have enthusiasm to this point in the end here where  we have a rewrite passing through a corridor that we could call the pit of despair and  denial. What's decreasing here over time with this graph is showing is the decline  in productivity or the decline in the ability to get anything done. It's typically when  things start getting to the point where we can't get things done, where it starts getting  difficult that we get into the denial, then we get into the despair where it becomes obvious  and then of course we just get to the point of declaring bankruptcy and declaring a rewrite.  We call this the software crash. And what happens here is that all the changes that  we've made over time in the system have made it more and more difficult to make subsequent  changes in the future at the same rate that we were able to make previous changes. Scrum  hasn't changed this, Agile hasn't changed this, Kanban hasn't changed this, Test Automation  hasn't affected it, Continuous Deployment hasn't affected this, Pivotal Tracker hasn't  changed this, Trello has affected it, Version 1 hasn't affected it. All the things that  we're doing haven't stopped the fact that software still has an end date and it's usually  much earlier than it's supposed to be. And that's because things are out of control.  The things that we are controlling with all of these countermeasures in Agile are not  actually the things that need to be controlled in order to stop projects from failing or  evolving into this pit of despair. So you can also think of that graph as the loss of  control graph. It shows our ability to make progress over time, which ultimately means  that progress and productivity are related to control. So if it was your goal in your  software effort when you started to achieve this collapse, this failure, if that was your  goal then your project is under control because you achieved what you expected. If this is  not your goal then you could clearly surmise that things are out of control. And we can  probably say that most software efforts are out of control. And the things that we're  controlling are not the things that give us an ability to avoid the crash. The other  side of this is that there's no reason why software shouldn't continually get better.  Software, there's an old Agile story, an Agile adage about somebody who's restoring furniture  and who goes into their workshop every weekend in their spare time and continues to work  on this furniture. And after a long time of working on the furniture they come out and  the product is worse than when it started. We would think of that as absurd. But it's  not absurd with software. We basically think of that as normal. So if we start to change  our focus from the things that change the shape of the curve from a downward trend to  an upward trend then whenever we start to control the things that steal this productivity.  This talk is called Agile Kaizen. I'm going to talk about Agile but we'll do the Agile  part later. But just to set a baseline I want to say what Agile is for me, what I think  of this, how it works for me. I think of this as an ability to make changes now in a way  that don't obstruct me from making any changes in the future. So for the rest of this for  a little while we're going to focus on the Kaizen part and then we'll return to the Agile.  So the reason why you may have this instead of this is this. And we can call that complexity.  Complexity is a mathematical thing. It's not something that is emotional. It's not something  that has to do with difficulty or easiness or lack of easiness. It's simply something  that you can measure in your software or something you can measure mathematically.  If you put complexity that is constantly increasing over productivity that should be constantly  increasing the best you can hope for is this which is a constant ability to make progress.  And that's okay. It's okay that this is a flat line because this is just about the best  that it can possibly be. If there's never more increasing demand and pressure and you're  still able to sustain the rate of progress then you are inevitably improving at a faster  rate or at least as faster rate as the things that are trying to drag the project down.  And the rule here which is an old lean rule or not really lean but there's a quote from  an old Toyota guy, I say old, somebody from the past that if you're not moving forward  you're falling behind. The reason for this is that the more the time progresses the more  moving parts you have in your system the more easy it is to make mistakes and mistakes in  software are often invisible. If you don't counter this counter productivity you're  going to end up with this crash. Whether you end up with this or this it's what you focus  on that makes a difference. And where you end up depends entirely on where your focus  is. So to get this from this we need to add something, some kind of anti-counter productivity  or whatever that is. And the more you add of this the more you're able to sustain a  steady level of progress. Or said otherwise in a system that's continually growing which  most systems are you've got to keep getting better just in order to make progress at the  same rate. So let's go back to Kaizen. There's a common English term for Kaizen. This is  the only time I'll ask for audience participation. It's a pretty common term so anybody who  knows it just shout it out. Just shout it out. Yep. What were you going to say? Right.  So exactly. So this is what I was looking for. Right on. Continuous improvement. This is  the common colloquial term we have for this. Here's what the encyclopedia has to say about  Kaizen. The Sino-Japanese word Kaizen simply means good change with no inherent meaning  either to continuous or philosophy. In Japanese dictionaries or everyday use the word refers  to any improvement one time or continuous, large or small in the same sense that the  English word improvement. However given the common practice in Japanese of labeling industrial  or business improvement techniques with the word Kaizen because there's no specific Japanese  word that means continuous improvement per se in especially case of oft-emulated practices  spearheaded by Toyota the word Kaizen in English is typically applied to measures for implementing  continuous improvement or even taken to mean a Japanese philosophy of continuous improvement.  So really it's just jargon. And when we talk about it in terms of jargon we say continuous  improvement we end up talking about this idea of a soft side. And when asked what Kaizen  is people often say it's trying to do a little bit better every day. Which isn't really a  hard science. It isn't a hard practice. Agileists might say it's what I'm doing when I'm doing  agile retrospectives. And because these explanations are common and because they seem to be so  fluffy this kind of squishy stuff that maybe HR people or scrum masters want us to do we  technologists we scientists often dismiss it as bunk. I'd like to suggest that this  limited view of Kaizen comes from viewing it through a limited lens and that there's  much more to it. For example this word continuous improvement this jargon is much more accurately  represented as how it's actually done which is closer to relentless improvement rather  than continuous improvement. It's not improvement when it's convenient it's improvement always  especially when it's inconvenient and irrespective of when it's inconvenient. And in a Kaizen  culture we don't just wait till we have time to do Kaizen like we do with continuous improvement  which is a which is a which is an event. There's no event in Kaizen. It's the way things are  done. It's the air thing that is breathe. It's the water that's between the tissues that  holds everything together. Even the soft side of Kaizen is as hard as that. But let's talk  about the hard side. The work of Kaizen is just as tangible as everything else you do  like daily stand-ups, automated testing, iteration planning, any other concrete practice you  have. And the hard side of Kaizen you could say is a science. It's a science of restoring  productivity. You might then say that Kaizen is this anti-counter productivity. So let's  quickly divert into a summary of things that I think we already know about software. Are  you guys familiar with this old saying garbage in, garbage out? This saying is as old as  computing. In all software programs you're essentially made up of three things. An input,  a process, and an output. And if the input is good and the process is good, then the  output is probably good. If the input isn't good and the process is still good, the output  is still not going to be good. And just as a sidebar, if you put good stuff into a bad  process you're going to get bad stuff out of it. But the point that I'm making here  is it's very hard to go from bad input to good output. What would it take to go from  bad input to good output? You'd have to add a lot of energy, a lot of effort. And even  then you may end up with an outcome that is just less stinky. Not really something you  can say is good, just a little bit less undesirable. And unfortunately a lot of stakeholders would  say hey, good enough. And in Kaizen and in Lean we say good enough is not good enough.  If this is what you're trying to avoid, then the idea of good enough is a primitive industrial  perspective that is ultimately the reason that this crash happens. Lean saying Kaizen,  that the goal in fact is perfection. And before anybody retreats to the old adage that perfection  is the enemy of good, I'm talking about perfection, a direction that we're going in rather than  perfection, a destination. So we've revisited input process output. We know that garbage  in leads to garbage out. I want to talk about a superstation of software management then.  Most software management and most software development and most software developers  and software workers focus on this. This is the part where we do the process and the delivery.  This is my work and getting it done. This is the just ship it part of software development.  This has been one of the most harmful slogans that we've adopted in software development.  This is the get my work done perspective. The just ship it stuff. This has been harmful  to us. And part of the reason is that this is really the part of the process that we  should be focusing on. This is the part that Lean and Kaizen are focused on. And the reason  for this is that garbage in is garbage out. And there is in any work a series of inputs,  processes and outputs. Every output is the input to the next stage of the process. So  we work in teams. We transfer work to each other. When we feel where our work is done,  we've handed it off. And we feel that we are making cumulative, cumulative, that all of  a sudden seemed like a funny word. Feel that we are making cumulatively more progress simply  because we are doing work and finishing work and shipping. And it's easy to get a false  sense of progress when our evaluation of what a complete task is ends with our own job function.  That is to say, when we view our view is limited to just getting our part of the job done and  just shipping it off to the next person in the Bucket Brigade. Frequently the reality  is that as we are handing off work, something is getting lost. We are handing it off and  we are leaving something out of the delivery. We believe that this is happening, but we  can observe in reality that over the life of the effort, this is happening. And here  is what's happening. This is a graph, conceptual graph, of the progress that we make. When  we transfer work between one function or one person to another, what's happening is we  lose the momentum at every handoff that we gained in the previous stage. So as we hand  work from one person to another, at every handoff, the progress that we made, the speed  that we gained, the momentum that we gained, as soon as we handed off to somebody else,  we lose that progress and the next person starts from scratch. Instead of having a cumulative  increase in momentum, we basically end up with a net, well, we end up ultimately back  where we started. So the key focus here is momentum transfer. So when a transfer between  two people happens, the person on the receiving end suffers delays. That person must figure  out how to proceed with their duties and with their work. And once they figured out how  to do their work, they ultimately can get down to business and do their work, but we  don't pay attention to this downtime and I'll submit to you that if you do a basic experiment  and measure how much of this downtime is present in your work, you might find something rather  surprising. So something is lost in the transfer as evidenced by the delays in starting work  at the beginning of a recipient's stage of receiving the work, then we should be able  to get some measure of that delay. So here's something to try and here's something that  I've done on a number of teams. Every time you start a task, any time any team member  starts a task, log the time. And when you get stuck, log the time. And when you get  unstuck, log the time. And when you step away from your work, log the time because that's  not part of the metric. And at the end of the day, tally up the total time you spend  making progress and the total time you spend not making progress. Now the interesting side  effect of this is the team also has to have a meaningful conversation about what progress  is and what progress isn't. And that's not something that should be defined for you.  You should really, it's a meaningful exercise to do this. But you should know that some  things are not progress. Trying to figure out somebody else's work is not progress.  The reason for that is they've already figured it out and somebody else doing it is rework.  That's counted in the lost side of the balance sheet. We call these progress outages or just  call them outages. And a good reason for this, this brings the focus of the stark reality  to the business, what's happening in the business. This is a de facto outage. And when we expose  this notion of outages, when we classify them as outages, the business can become motivated  to make improvements to resolve these outages. And the good thing about this measurement is  that there's no shortcut around it. Or as Deming said, it does not happen all at once. There  is no instant pudding. You're going to have to work at getting this, but there's no way  around it. You can't just make out, you can't just take shortcuts and then end up with more  outages. The outages will never go away even if you try to invoke some expediencies. Those  actions that are taken will further exacerbate the problems and they'll show out on the outage  logs as more outages. So in the face of further shortcuts, if you see outages and you have  a management reaction to have some expedites or some shortcuts, ultimately what you'll  see is that we're still trending towards the crash. An alternative of this, if you find  it too heavy handed to log your time, because that can be taken incorrectly, it can be taken  as something forceful and something oppressive by workers, then just log the incidents of  the outages. Log a description of them and that they happened. And report these outages  during your team sync up. So whether you do a daily scrum or something called a stand  up or whatever your practice is, whether it's daily or not daily, whenever this happens,  report them. Make them very visible. Make it really clear so that the team starts to  understand, scrum masters understand, stakeholders start to understand that there are outages.  And ultimately we're going to bring awareness to these things. And what you'll find in the  end typically is that most developers spend most of their time trying to figure out how  to start their work than actually doing the work. And further to that, you can think of  many small tasks that a developer or any worker is doing in the process. Not just a story  card as a task, but any individual task. The time that we take trying to figure out how  to fit our work into the existing system, most of our time is spent doing that. That's  lost time. That's outage time. So dealing with the root cause of this pervasive problem  means that we change our focus to the input stage. It means the job isn't done when we  just ship it. The job is only considered done when the next person in the process has regained  the momentum that the person in the previous stage has. So let's think about this from  the terms of a relay race when somebody is passing a baton on a relay race. We don't  really consider the passage of a baton on a relay race. Well, let's just say how it's  done. When it's done, there's a moment where both participants' hands are on the baton.  And the person passing the baton has the greater responsibility in successfully navigating  and executing that handoff. In software, this is typically what we do. This is called a  push system. This is what happens when we're focused on just ship it. I do my work and  I drop it off. And the next person is not my responsibility. Work isn't done until the  next person in the next stage is enabled to work at the same pace as the person in the  previous stage. So in a push system, we just drop things off in a pull system. The handoff  isn't done until the recipient says it is. Sure. The recipient can say I'm ready to go.  Can simply be I understand enough to proceed without you being here. And it's just that  simple. But the interesting thing about that is that time should actually be measured as  well. Because we can start to have interesting, interesting, we can get some understanding  of how long it takes to quote, get somebody up to speed. And I don't like this term get  somebody up to speed because it trivializes the meaningful nature of this interaction  between two workers.  All of it. In fact, the most frequent handoff is from yourself to yourself at a later date.  For a programmer, that's true. Programmers more often handoff work to themselves at a  later date and still suffer the downtime.  So we say the work isn't complete if the work transferred to the next person causes an outage  in that stage of the work. So when we handoff the work, it means that both participants  are hands on the work. And it's the recipient who was authorized to declare that the handoff  is complete. And the person making the handoff is obliged to continue with the work until  the recipient's momentum is raised to the level that existed prior to the handoff.  And work is basically considered incomplete until the transfer of momentum is accomplished.  There's another word for incomplete work. Work in process. And those of you who've heard  in Kanban, this is a salient issue, so I want to present it to you in this light. Incomplete  work is work in process.  The purpose of Kanban is to create a tightly synchronized series of handoffs where there's  no slack between the participants. And the reason for this is really critical. In a push  system where work is just dropped off for another recipient to pick up later, the work  product is in fact not validated to be complete or sufficient. That means that the work piles  up, work that has been dropped off can pile up, and has not been validated to be free  of flaws. And when flaws exist that are not detected immediately, they're going to be  built upon. And they become encased in software. And then that software is built upon. And  once that happens, the software flaw will never be removed. It'll be impossible to  fix. And the quality of the software will never exceed that particular level.  And by flaw, I mean design flaw. I don't mean something you can find with an automated  test. That's the easy stuff. I'm talking about things that are fundamental violations of  structural design principles that dictate whether you'll have this or you'll have this.  You can't discover these things with automated tests, but you can discover them with test  driven development. And in answer to the inevitable question as to whether or not test driven development  is dead, my answer is this. Only if the fundamental flaws of structural design physics have been  suspended and design principles no longer matter, then test driven development is dead.  Sometimes developers can't even see these flaws. So if the input materials are flawed,  then the processing is going to be troubled, and then the outcome is going to be troubled.  And the best way to validate the input materials, whether or not they're flawed, is to execute  the handoff with both participants on hand. It's very simple. If the recipient of the  handoff has difficulty understanding how to start, we should reconsider one of two things.  Well, there's two things that we need to think of. The quality of the design, the clarity  of the design, and the level of capability of the recipient. And we have countermeasures  for both of those. And the person doing the handoff, as they are the person that have  done the work, are qualified, capable, and should be willing to bring that person's skill,  knowledge, and capability up to a level where they're capable of working on it.  So you'll be able to understand, you'll start to get an understanding of the effectiveness  of your design in relation to the, its effect on the human workflow by measuring what's  happening at the handoffs. So you want to go from this to this. You don't want to end  up with this. That's what we're after. So the purpose of Kanban then is to create a  highly synchronized workflow without slack in the handoffs so that handoffs can be used  to highlight problems in the design. And another way, the purpose of Kanban is to show you  where to apply Kaizen. And if you're doing Kanban, simply to put stickers on a wall and  work in process limits, there's really not a hell of a lot of value in that over simply  just calling it scrum or XP or work. Let's talk about technical debt. I think this has  been one of the most harmful ideas in software development. And here's why. All technical  debt is in complete work. So for anybody doing Kanban, what value are your work in process  limits for every stage in the work when you are not counting technical debt as work in  process? This is a very clever accounting system. Usually in accounting systems we'd  call that cooking the books or fraud. And all in complete work is hiding design flaws.  And if we build upon those design flaws, we won't be able to fix them. And design flaws  are the root cause of this. If it's your intention to end up with this, then we should allow  technical debt to be a legitimate allowance of your production system. The problem with  technical debt is that we gave it a name and now we think it's okay. That's like saying  because we gave syphilis a name, we should all go out and get it. And if you really,  really need technical debt to exist within your system, and sometimes you do, it's because  you've burned through your safety margin because you haven't paid attention to your safety  margin. And you've done that likely because you felt that necessary improvements are really  too insignificant and too negligible to require investment. So all teams have a capacity.  This is the high water mark beyond which they cannot make any further efforts. This is the  absolute limit of how much they can work. And there's amount of work that, oh, I'm missing  a label, unfortunately. Yeah, let's do that. So that yellow line is the safety limit. The  line between the red and the yellow, that's your safety margin. And that green line is  the current level of work. The safety margin exists to allow for handling of unforeseen  circumstances without degrading the system or the team's abilities to respond to it without  driving it further towards the pit of despair. Most software managers don't have a notion  of the safety limits. Not that they don't manage it. They don't know that this technique  or this topic is relevant. It's not president in the day-to-day management work or thinking.  Some don't recognize it as significance. And as a result, most teams operate past their  safety limit and they push their work into the safety margin. They consume the safety  margin as if it's an allocatable resource for day-to-day work. And when something unexpected  happens, there's no available capacity to handle it. The inevitable result is this. This is  what happens when the safety margin is misappropriated as a resource for some other purpose. Let's  talk about the design. Say a design has a good outer service. It's something that you  can build on. And as you go forward, you build upon what you had put in place previously  and you continue to add to the system. And then you get a design flaw. This is technical  debt. This is where you build over the flaw. This is technical bankruptcy. This is where  you build over the thing that you built over the flaw. At this point, it's practically  impossible with the budgets that we usually operate on to fix this flaw. And if you build  on the flaws as if they're the foundations for the future work, then you can expect your  foundation to be flawed. And if you do that, this is what you get. This is what happens  when design flaws are encased in construction. So think of design flaws like cells, like  living cells. They fit together nicely. They work together nicely, cooperatively, with  their adjacent cells. Sick, misshapen, deformed cells cause their adjacent cells to be deformed.  A unit of software that is not correctly designed, everything else that interacts with  that software will take on some of allowances for the fact that that thing that they're  using is not correctly shaping. That means all the things that are adjacent are also  going to be poorly designed and this spreads out. So the worst software problems exist  on a cellular level. The problem, you could say, is in the composition of the sand, not  in the composition of the bricks or the construction of the house. If the sand is flawed, you  can pretty much bet that the integrity of the house is questionable. Software problems  are more like cancer then. You often don't see them with the naked eye and the untrained  naked eye certainly won't see them and may even deny their existence, mostly because  it's invisible to them. The question is, how much cancer would you personally be willing  to withstand? You want your systems to grow healthily from seedlings to saplings to some  mature creature. And the twisted nature of software that has cancer is that it's not  visible to users. A user's experience can go undisturbed while a developer's experience  is already demonstrating signs of the impending doom. The level at which users interact with  the software doesn't display the problems of productivity of working with the software.  Software can work even while it's not workable. So we have to separate our perspective on,  well, you have to separate your perspective if you want continuity of your business that's  based on software. If your goal is a crash, then you don't really have to worry about  this. And for a while there in all systems, both the users and developers have had a happy,  healthy, enjoyable experience of the system. And then developers will start to understand,  they are the early warning sign. Developers, testers, operators. I'm saying developers  mostly as a bias because I'm a developer. But the whole team, anyone involved, anyone  hands on. And as you raise these messages up through the ranks, you might have brought  these concerns to your boss about what you're seeing on the horizon. But anyone that isn't  hands on in the work won't be able to see the same thing. And ultimately, many of these  concerns are dismissed. This is when good people leave, when projects become unworkable.  And that's when software dies. So let's talk about why this happens. It happens because  mistakes were made that could not be repaired. This happened. So how do we deal with this?  You have to learn how to make fewer mistakes. You can learn how to make fewer mistakes by  improving relentlessly. But here's the crux issue here. Developers often don't know what  a mistake is. And even when they're experiencing the mistake, even when they're experiencing  the injury, developer cultural issues can motivate developers to not admit that it's  happening. So the ultimate goal here, the ultimate countermeasure, is that developers  have to master design principles and their application. That should go without saying,  but it's quite rare. And here's a complication to this. In mastery, let's say that those  first few, that these vertical lines represent steps in a progression of knowledge or capability  or career or talent or skill or whatever you want to call it. It's not just seniority.  It's just a measure of the passage of time. And people can pass time without getting better.  This is a representation of people getting better. Each subsequent step towards mastery  tends to be much more subtle than the last step before it. So if I'm talking to a junior  developer who is going from this step to this step about why it's necessary to change the  design from this subtle step to make this subtle step, they will very likely see this  as negligible and irrelevant, not worth the investment. If I speak to a business stakeholder  about this, well, frankly, one, they may not understand me at all. And two, they absolutely  will not understand the significance. But here's the thing about the mastery curve.  As you make successively more fine refinements in your skills in a good direction, in the  right direction, your capabilities or the capabilities of the team increase exponentially.  These small changes, if they are the right changes, will increase the capacity of the  business at a geometric rate, which is the countermeasure to the geometric rate at which  software is naturally wanting to degrade, which says what Kaizen is saying is that all  of the things that you are saying in the just-shippet mentality that don't need to be done, these  words that we use like bike shedding, yak shaving, navel gazing, these are all ways of delegitimizing  a meaningful conversation about making meaningful progress towards an evolution of skills that  are hard to see from a distance, they're hard to get, they take time to work on, they take  a lot of practice, but yield the highest value in software development efforts.  So we can say about Kaizen then, it's a relentless pursuit of mastery whose goal is to restore  the safety margin so that you can deal with those unanticipated situations without any  sacrifice to the prosperity.  I thought I had a lot more content, but it went very fast.  So this is where you can reach me.  My illustrations were done by my wife.  If you are interested in custom illustrations for your presentations at work, please drop  them up or align.  And if there's any conversation that you'd like to have, then I say we get into it.  Sir.  Do you have any other examples of projects that abuse this approach and actually demonstrate  for you?  Do you have any claims that will improve your stuff?  Yeah, well, I've done these in my projects.  I've done them only with teammates who are interested in doing it.  And I've done it for years with myself and my own data.  And it takes a good long time to produce good hard metrics.  But what we do get out of this early on is indications from, and I don't think this is  necessarily science, but good gut feelings, which can be wrong.  But over time, we've seen improvements in the, at least in the reduction of incidences  of outages, which I think is a really good first level, first goal to hit, even if we're  not logging times and handoffs and stuff like that.  That's the ideal.  But if we're just seeing reduction in outages, ultimately what we're seeing is the ability  to do work without downtime.  So yes, I've seen better outcomes.  Sir, if I can have a follow-up.  Sure.  What design practices do you recommend?  Nothing that hasn't been known for decades.  So aside from practices, which, so I'm going to do, I'm going to do a presentation tomorrow  on test driven development.  I obviously feel that that's a valuable thing.  But I also feel that it's incredibly misunderstood what that serves.  And it really, it's to support design.  It's to evaluate whether design principles are being respected.  So I'd rather talk about the design principles that I think that matter.  And I think that they're the tried and true things.  So I'm not even going to use the common words for the design principles, just on the off  chance that I would say them and they would put you right to sleep and you'd gloss over  because you know.  The most important thing I think in software design, to keep software design itself workable  and productive, is to avoid entanglement.  So I'll call it entanglement.  That's pieces of software that are tied to other pieces of software so that if you change  one you must change the other.  And there are certain things that we do commonly that we think are good, that we believe are  good as developers, as testers, as project managers, that we think that this, a lot of  it is rooted in tooling.  In fact, we spend more of our efforts and our mind share on tools rather than on techniques.  So a lot of the tools we use are directly correlated with a lot of these problems.  Object relational mapping is going to cause you the kinds of entanglement problems that  even though they look perfectly fine because the tool is encouraging them, are ultimately  causing work to continually slow down over time.  So that's a rough thing.  That's a rough thing to contend with.  I found that tools like Cucumber, new generation testing tools often contribute to more entanglements  in software, more software maintenance, more difficulty in maintaining software.  I think that they're much more effective ways of offering tests and acceptance tests that  don't require this tooling.  In fact, it requires a lot more lightweight tooling, a lot of things that are easier to  learn and easier to maintain.  We look at the term, we have this word testable.  Testable means something.  And it means something about design.  It's a design quality.  It's not about whether or not you can run a test or something, but it's the difficulty  in running the test.  In fact, it's the difficulty in setting up a test to be run that demonstrates whether  something is testable.  The difficulty in setting up a test to run tells you how much entanglement you have in  your software.  And the more entanglement you have in your software, the more things that need to be  set up before you can start to learn how your software works.  That is the crux of the loss of momentum.  The faster you can learn how your software works, the faster you can get to work.  That's all there is to it.  The more obstructions you have to understanding how your software works, the longer it takes  you to get your job done.  So we can say that the only thing that we're trying to do is to improve clarity.  And the only thing that's lacking in software that is heading towards the pit of despair  is clarity.  And it's really, really difficult to broach this subject with software development, people  with testers, with project managers, because we're so focused on these things that we think  are helping us go faster.  And really, a lot of these things help us go faster in the very beginning of a project.  And the very beginning of a project is the easy part.  We shouldn't be adding tools to make the easy part be easier, especially if those tools  make it harder to do the subsequent work.  There's a reason why there's renewed interest in functional programming languages.  It's directly related to this issue of productivity and of clarity.  And the only thing that is boggling or bogging down software projects is the inability to  understand the code.  A lot of these things that we think are team problems, interpersonal problems, conflicts,  and all of them, but a lot of them can be traced directly back to lines of code.  Difficult relationships on teams can be traced to the stress that comes from the inability  to get work done.  Now, a lot of programmers may be too busy to express it that way.  They may not have the patience.  There can be issues with emotional maturity in expressing it that way.  But at the end of the day, a lot of the things that we think that we need agile interaction  remediations for can be fixed if we just do what we're supposed to do anyway, which is  make the project go as fast as possible.  And that's done by solving the design.  But that's done by challenging paradigms, tooling paradigms.  And that is a very big challenge.  I've been trying to do that for many years in the.NET world.  That was during the time where we're bringing up agile in.NET and bringing up open source  in.NET.  I work in the Rails world now, and the problem is exactly the same now in the Rails world  as it was in.NET in 2007, that there's a sort of orthodoxy of the way things are done.  We believe the tooling is good.  We don't question whether the tooling is good, but we can actually evaluate the code  and mathematically demonstrate that the qualities of entanglement are all over the place.  It's just that we have to recognize that that's the problem.  That was a long cop out to your question.  I think the worst thing to do would be to take the suggestions I've made to a team and  require them to do it.  What I've done, I'll do it myself.  I've got three years of data.  I'm just interested in this because I've been interested in why it's so damn hard to get  anything done in software development.  It's not supposed to be that way.  When I've talked about it with other folks in organizations I've worked at or clients,  I'll mention it.  Some folks who are interested in personal performance get into it.  One of the things I think that's been effective for customers that I work with who are struggling  with time is to just track the outages.  Forget about the measurements.  I like the measurements because I'm interested in seeing what it means over years.  If it's just tracking the incidents of outages, how many a day, whether you had them today,  where these outages tend to concentrate in the system, that's a really simple thing to  do.  It also allows developers the freedom to say, this is a really challenge.  This is where my time went.  It frees them to say, I didn't get anything done and here's why.  I'm going to call that an outage.  We're going to change our practices so that talking about and exposing the outages is  part of the everyday work.  We don't let ourselves look away from that until we get them fixed.  Sooner or later, it tends to bubble up and managers and stakeholders and senior managers  end up understanding that at least these delays are not due to what folks often think it's  due, which is people being lazy.  I would say do it with a very, very light hand and don't make it a requirement.  Make it a matter of personal interest or part of the adventure, part of the exploration.  Once if anybody leaves this room and goes back to their team and says, here, everybody,  you are now required to log your time, fundamentally misunderstood it.  I'll be highly motivated to visit you if that was the case because it's really, really not.  I would understand the motivation to do it, but it would be a great disservice to the  advances that you're trying to make.  It would work against you.  Ask them or do it yourself and demonstrate.  I've done it and just started on some consulting engagements.  I've just started talking about outages.  Here's what I did yesterday and here were my outages.  Then it starts to, if I can see that it's not me, if there's no resistance to it, then  I'll continue doing it and see who takes it up and then people will ask me about what  it is and what that means.  We have a more meaningful conversation.  Anything else?  Yes?  I like your statement about perfection in the direction, not the end goal.  Great.  That's also my way of working.  But I noticed that a lot of people really reacted that and little improvements are  made in the way that you're talking about.  Try to have a conversation about it.  I try to steer the conversation more towards how can we know that they're negligible?  Then I present to them some ideas about design principles and some of these ideas.  It ends up being a qualitative conversation because it's really hard to have a quantitative  conversation about that.  In fact, it's hard to put the practices in place to get to quantifiable measurements  until people are comfortable with the qualitative aspects of it.  I deal with it with a great deal of pain because I would rather that folks would have had the  benefit of some of the experiences and teachers that I've had the benefit of and some of the  perspective I have.  You just can't put your experience into another human being.  That's hopefully what constructive conversations are about.  I would say, this is making me think of something else, we have a really difficult time leveraging  conflict in a meaningful way and to practice conflict as an honorable thing.  We tend to say it's bad, it's destructive, it's avoidable, we should avoid it at all  costs.  I don't think that's good.  I think that from a Kaizen perspective, there's a lot of things that we do that are not solving  root causes.  We're bad at conflict, we should get better at conflict.  There's a number of things like that in Agile.  Why do we stand up, I'm sorry, I'm going off the subject of your interview.  Why do we stand up and stand up meetings?  Does anybody stand up during their stand up meetings?  Okay.  Does anybody remember what the reason why we stand up?  Keep them short.  Right?  There's a reason why they would not be short if we sat down.  Right.  So if that's the problem, which we'll call that an attention problem, why don't we work  on that?  Why don't we all sit down, allow the problem to exist, measure the incidence of the problem,  and find a meaningful way to solve the attention and focus problem.  If we can fix the attention and focus problem, it means that we've improved ourselves in  a very deep way.  It means that we're attacking something fundamental, not just about our work at, you know, in  doing a stand up meeting, but all of the work that we have.  It means that we've sharpened our alertness, increased our attention, created our focus,  probably increased our compassion and our understanding.  So a lot of these things are crutches, they're prosthetic devices that we use to put in  place that are actually blocking more deeper improvements.  There's a number, you know, off the top of my head I'm blocking, because now I'm stressed  to find them all to look like I've thought about this for years, which I have.  But suffice to say, everything that you do with agile development should be considered  a specific countermeasure for a specific problem.  You should use a countermeasure, every tool, stand ups, sprint planning, retrospectives,  story cards, electronic tools, testing, test driven development, automation, all of it.  Every single one of things is meant to solve or address a specific problem.  When we take these things and we use them irrespective of the knowledge of the problem, we're  kind of behaving, no we're not kind of behaving, we are de facto behaving superstitiously.  And I think that's the opposite of behaving like people who are really, really engaged  and interested in solving deep problems.  I think that's it.  Thank you very much.  Thank you  You
Scott_Bellware_-_Agile_Kaizen_0_431.png Scott_Bellware_-_Agile_Kaizen_432_1211.png Scott_Bellware_-_Agile_Kaizen_1212_2399.png Scott_Bellware_-_Agile_Kaizen_2400_3767.png Scott_Bellware_-_Agile_Kaizen_3768_4091.png Scott_Bellware_-_Agile_Kaizen_4092_4763.png Scott_Bellware_-_Agile_Kaizen_4764_5039.png Scott_Bellware_-_Agile_Kaizen_5040_6443.png Scott_Bellware_-_Agile_Kaizen_6444_6827.png Scott_Bellware_-_Agile_Kaizen_6828_7163.png Scott_Bellware_-_Agile_Kaizen_7164_7835.png Scott_Bellware_-_Agile_Kaizen_7836_9479.png Scott_Bellware_-_Agile_Kaizen_9480_9611.png Scott_Bellware_-_Agile_Kaizen_9612_9659.png Scott_Bellware_-_Agile_Kaizen_9660_9899.png Scott_Bellware_-_Agile_Kaizen_9900_11639.png Scott_Bellware_-_Agile_Kaizen_11640_15323.png Scott_Bellware_-_Agile_Kaizen_15324_15767.png Scott_Bellware_-_Agile_Kaizen_15768_15839.png Scott_Bellware_-_Agile_Kaizen_15840_16535.png Scott_Bellware_-_Agile_Kaizen_16536_17963.png Scott_Bellware_-_Agile_Kaizen_17964_18119.png Scott_Bellware_-_Agile_Kaizen_18120_18335.png Scott_Bellware_-_Agile_Kaizen_18336_18611.png Scott_Bellware_-_Agile_Kaizen_18612_19067.png Scott_Bellware_-_Agile_Kaizen_19068_19247.png Scott_Bellware_-_Agile_Kaizen_19248_20171.png Scott_Bellware_-_Agile_Kaizen_20172_20615.png Scott_Bellware_-_Agile_Kaizen_20616_21155.png Scott_Bellware_-_Agile_Kaizen_21156_22115.png Scott_Bellware_-_Agile_Kaizen_22116_22463.png Scott_Bellware_-_Agile_Kaizen_22464_22643.png Scott_Bellware_-_Agile_Kaizen_22644_23987.png Scott_Bellware_-_Agile_Kaizen_23988_24083.png Scott_Bellware_-_Agile_Kaizen_24084_24839.png Scott_Bellware_-_Agile_Kaizen_24840_25199.png Scott_Bellware_-_Agile_Kaizen_25200_26879.png Scott_Bellware_-_Agile_Kaizen_26880_26951.png Scott_Bellware_-_Agile_Kaizen_26952_27599.png Scott_Bellware_-_Agile_Kaizen_27600_28439.png Scott_Bellware_-_Agile_Kaizen_28440_28847.png Scott_Bellware_-_Agile_Kaizen_28848_29807.png Scott_Bellware_-_Agile_Kaizen_29808_30743.png Scott_Bellware_-_Agile_Kaizen_30744_31163.png Scott_Bellware_-_Agile_Kaizen_31164_32255.png Scott_Bellware_-_Agile_Kaizen_32256_32723.png Scott_Bellware_-_Agile_Kaizen_32724_34523.png Scott_Bellware_-_Agile_Kaizen_34524_35219.png Scott_Bellware_-_Agile_Kaizen_35220_35531.png Scott_Bellware_-_Agile_Kaizen_35532_35795.png Scott_Bellware_-_Agile_Kaizen_35796_36851.png Scott_Bellware_-_Agile_Kaizen_36852_37199.png Scott_Bellware_-_Agile_Kaizen_37200_37367.png Scott_Bellware_-_Agile_Kaizen_37368_39947.png Scott_Bellware_-_Agile_Kaizen_39948_40079.png Scott_Bellware_-_Agile_Kaizen_40080_40511.png Scott_Bellware_-_Agile_Kaizen_40512_42275.png Scott_Bellware_-_Agile_Kaizen_42276_42575.png Scott_Bellware_-_Agile_Kaizen_42576_42623.png Scott_Bellware_-_Agile_Kaizen_42624_43187.png Scott_Bellware_-_Agile_Kaizen_43188_45071.png Scott_Bellware_-_Agile_Kaizen_45072_45839.png Scott_Bellware_-_Agile_Kaizen_45840_46079.png Scott_Bellware_-_Agile_Kaizen_46080_46367.png Scott_Bellware_-_Agile_Kaizen_46368_47123.png Scott_Bellware_-_Agile_Kaizen_47124_47387.png Scott_Bellware_-_Agile_Kaizen_47388_49799.png Scott_Bellware_-_Agile_Kaizen_49800_50939.png Scott_Bellware_-_Agile_Kaizen_50940_51563.png Scott_Bellware_-_Agile_Kaizen_51564_52187.png Scott_Bellware_-_Agile_Kaizen_52188_52259.png Scott_Bellware_-_Agile_Kaizen_52260_52415.png Scott_Bellware_-_Agile_Kaizen_52416_52571.png Scott_Bellware_-_Agile_Kaizen_52572_52679.png Scott_Bellware_-_Agile_Kaizen_52680_53099.png Scott_Bellware_-_Agile_Kaizen_53100_53495.png Scott_Bellware_-_Agile_Kaizen_53496_57911.png Scott_Bellware_-_Agile_Kaizen_57912_58463.png Scott_Bellware_-_Agile_Kaizen_58464_83495.png Scott_Bellware_-_Agile_Kaizen_83496_85516.png