Retrospectives for Humans
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Eckhardt, Courtney
Seattle has two of the longest floating bridges in the world, and in 1990, one of them sank while it was being repurposed. This accident was a classic complex systems failure with a massive PR problem and great documentation. That combination is an excellent frame for talking about incident retrospectives- the good, the bad, the vaguely confusing and unsatisfying. Come for the interesting disaster story, stay to learn about the language of blame and how to ask warm, thoughtful engineering questions.
 All right, thank you for coming to Retrospectives for Humans. Before we begin today, I want to acknowledge this has been a really frightening and difficult few weeks for a lot of people with tragedy and violence touching a lot of people that I know and if that's you, I'd like to offer my sympathies. It can be difficult to function normally and pay attention to everyday business at times like these. It can be even harder to learn new concepts. So please do what you need to do during this, take care of yourself. I don't think this talk requires any content warnings. If it does and I've missed it, I apologize, I would appreciate hearing from you afterwards. All right, I'm here to talk about retrospectives. These are incident retrospectives, but most of the things I'm going to say today will probably also be useful to you in an agile retrospective, should you choose to do that. The subtitle of this talk could also be words mean things. Let's talk disasters where no one was hurt. Seattle has the two longest floating bridges in the world and in 1990, one of them sank over the Thanksgiving day weekend in a storm while it was being repurposed. The public reporting to this day is extremely simplistic, but the official investigation found that there were five factors involved and all of them were required for the bridge to fail. At the time that the bridge sank, you can see this is a picture of that happening, they were scraping off the surface of the bridge with water, it's called hydro demolition. They were going to change the direction of traffic on the bridge because they were building a span nearby, so they were going to expand how much traffic the bridge could take. The EPA told them that they were not allowed to let the water run off into the lake, that that was a serious environmental problem, so they had to figure out a way to store the water so that it could be trucked off and treated. You can't store water in piles, so they needed to find a container to put it in. The thing that occurred to them after much study was that they would store it in the hollow pontoons of the bridge itself. Floating bridge is made of these hollow sections like a ship's hull, it's made out of concrete in this case and this bridge was actually floating higher than it was designed to float and had lots of extra floatation ability that they decided to use in order to store the water so they could truck it away. When this happened, no one was on the bridge, it was over the holiday weekends everyone was off and one pontoon sank slowly and dragged seven more down after it, so this was a slow cascading failure, extra fun. This is a video of some of the coverage of what happened. I'm going to play it for a few minutes. Built in 1940 in Seattle, a Lacey V. Murrow bridge was named after the director of Washington State's Department of Transportation. The bridge stood for 50 years, but in November of 1990, a rehabilitation and maintenance operation on the bridge led to disaster. Spanning a short but deep section of Seattle's Lake Washington, the Murrow bridge floated on a row of 25 hollow concrete pontoons, each 350 feet long. The pontoons were bolted together for stability and anchored to the bottom of the lake. The conditions are just right for use of such a structure because the water conditions under which they're installed are perfect. It is very seldom is there any ice, very seldom is there a heavy current and the water is very deep, 200 or more feet. At the fall of 1990, the Murrow bridge was undergoing a facelift by means of hydro demolition, a process in which highly pressurized water is used to break up the roadway. Since environmental regulations prohibited dumping the contaminated water into the lake, it was temporarily stored in the empty pontoons via access holes which were cut into the tops and sides of some sections. The water level inside the pontoons was carefully monitored so the structure would not be compromised. But on Thanksgiving weekend with bridge crews short handed, a heavy storm hit the Seattle area. Rainwater drained into the open pontoons. Winds from the storm whipped up the surface of the lake, propelling splash water into the pontoons as well. On the morning of November 25th, eight waterlogged pontoons sunk into the lake, taking those sections of the bridge down with them and severing 12 anchoring cables of a bridge that was being constructed nearby. While luckily no one was injured, the cost of damage was estimated at $69 million. An investigation of the accident came to a sobering, if unsurprising conclusion. Their very extensive study and very sophisticated computer modeling methods the conclusion was that concrete pontoons simply don't float when they're full of water. A very unsophisticated motive. The Lacey v. Murrow Bridge was rebuilt two years later. Though it looks the same from the outside, the pontoons that support the bridge have been divided into smaller sections to minimize damage from leaks, a precaution that underscores the structural similarities between floating bridges and floating ships. The pontoons were designed with multiple cells, not unlike the multiple cell super tankers so that failure in one place would not bring down the entire pontoon. So he got the laugh he was looking for. That's not what I wanted to have happen. Apologies. Where is my mouse? Okay. So this was a giant disaster. They checked a lot of stuff. I'm not even going to read all of that off to you. It was an incredibly thorough investigation. I was able to get a paper that was published by the people who fronted this investigation and read the original text of what they determined the failure to be. This part I am going to read. This is what they found was the problem, several problems. The loads that created significant leakage were the combined effects of all accumulations of water, including rain after the windstorm, longitudinal flow on the surface of the bridge, and pumping through November 24th, 1990. These loads caused static moments, that's to say pressures, that exceeded the threshold for leakage. Existing cracks were open sufficiently to allow water to leak into the pontoon. Excessive and accelerated sinking began at this time. So there's a lot of things that we've got going on here. We found that there were existing cracks in the bridge. The survey did in fact find that this bridge was built in the 40s, 30s. He told you at the beginning of the video. Now I can't remember. So the bridge was in excellent shape for its age. It had cracks that were expected in concrete of that age, but not as many as they would have expected. The cracks were smaller than expected. The bridge was floating higher than designed. That's the extra flotation capacity that I talked about before. So the bridge was not just in decent shape. It was in spectacular shape. It was dramatically over engineered. And so they found they needed to put the water someplace. They did very careful surveys of the bridge and how much flotation there was, and they calculated an error, like a buffer for their error, and worked really hard to make sure that they were being careful when they did this. It was very thorough. But still somehow this happened. The official investigation missed something huge. The crews were having trouble pumping the water out on the schedule that was said by the engineers. And there is a mention of that in the paper, but there's no explanation as to why. I don't know if they didn't do an investigation or if they simply didn't document the result of their investigation. But there was a social problem here. There was a schedule set. It involved trucks that were pumping water, carrying it off the bridge. It involved water being stored evenly, hopefully across the pontoons that they were storing it in. And for some reason, crews couldn't stick to that schedule. We don't know why. And so we don't truly understand what happened here. We know that the water was higher in some pontoons and lower in others, which was causing torque on the bridge, which helped open those cracks. We know that the trucking crews were behind. There could have been not enough trucks. There could have been not enough drivers. There could have been a strike. There could have been pay problems. Any number of things could have gone into this. And we're never going to truly understand. So as thorough as this was, as really good as the true investigation was, and as different as that is from the soundbite that made us all laugh, we still don't truly understand what happened here, which is just such a good representation to me of the things that it's easy to miss in a retrospective. So we all laughed. He says, you fill a bridge full of water at six. But no one does things that they think are going to blow up the world. It's easy to look at that and be like, oh, you stored water in the bridge. Oh, my God. But they stored water in the bridge very carefully with a very firm plan for how they were going to continue to have this be OK, and it still didn't work. So the lesson here is remember that even as you have that feeling of, like, you know, face palm, here we go again, it's probably more complicated than that. Asking about what a person knew and what the conditions were at the time that they made a decision will give you good insight for effective change. So let's do better than that. Now, this is going to be a crash course. I condensed, like, double the amount of content into this time. So we're going to run fast. If you have more questions, come find me afterwards. Facilitation, we're going to talk about how to have a good meeting. Other ways to think of this, servant leadership, psychological safety. As a facilitator, your job is to keep the conversation blame-free, not let people make bad jokes, and to run a pretty decent meeting that people are willing to sit in instead of checking out of. So let's talk about how we talk about all of this. First thing, foundational Miller's law. In order to understand what another person is saying, you must assume that it is true and try to decide what it could be true of. You're being asked to imagine what it is like to be another person, probably at another time, at another place, in another context. This is the only way to truly understand a system or a person or an environment or a happenstance that was not one that you personally lived through, or even one that somebody else lived through next to you, because no two people ever have the exact same experience, even in the exact same moments. I learned about this from Suzette Hiddon-Elgin in the general art of verbal self-defense, which is a fairly old book at this point. I do recommend it. It talks a lot about language and how we talk to each other and the ways that we can create conflict and de-escalate conflict. This is a critical tool to use when you're conducting a retrospective. Keep this in mind. Started here. Everything else is going to build on this. Let's talk about blame. English is a pretty blamey language. I might say to myself or to another person, you knocked over that face. Why did you knock over that face? Those are both statements that are about blame and fault. You. Yeah, you. I'm talking to you. Don't just walk away from me. That feels awful, right? That's terrible. Starting a sentence with you draws a line between yourself and the person you're speaking to. So does using it too often. It creates an oppositional conversation. There's the person saying you and the person who's being youed at. Why? Why is another part of blame? Why did you do that? Why did you do it like that? That also feels terrible. Asking a question that starts with why is a request for a justification. It immediately puts the person you're addressing on the defensive. Why questions get answers in a gentile language because I knocked it over? The face is broken because I knocked it over. A gentile language has blame in the very grammar. It talks about a person doing a thing. It's strongly remembered by English speakers. So then after you've asked this question and you've gotten this answer, you've ensured that everyone who heard it remembers that I knocked over the face. Other things not to say to people, especially in a retrospective, but often at other times too. You always. You never. Every time we do this, you should. You should just. You should only do it like this. There's more, but these are the least wanted. If you have a sibling, anything your sibling ever shouted at you when they were mad that made you see red goes on this list. Anything that your child, if you have a child, has said to you that just made you want to completely blow your top because kids are really good at that, goes on this list. You might, if you're talking about an API or an RFC, use these words. Especially like RFCs have this very prescriptive language, right? So sometimes we can't avoid it. But remember that when you describe the behavior of a system, you are describing the decisions that some people made when they built it, right? It's more upsetting if those people didn't know the consequences of the decisions that they were making, which they didn't. No one can be perfectly prescient. So you are saying things about decisions that people made that they probably didn't even understand that they were completely making. Why didn't you just fix it the last time this happened? How many times have we heard this, right? This is terrible. I'm going to take this apart because this is not compassionate and it's not effective at helping us understand the problem. This construct assumes that it's happened before, maybe more than once, that it could have been fixed the last time it happened. Easily, that's the just. That you was the right one to fix it and that there was no good reason not to fix it. That's a lot of assumptions in one very short sentence. Here's some better things to say. How could you or how could we? What happened? What do you know? What did you know? What if? What if we tried this? What if we tried that? Could we next time try this? What do you think about this proposal that I have? What would you have wanted to know in the moment where you did that and you made that decision? All of these are designed to get more complex answers when used in a question. When you're asking a question, think about what you can say that will get you the longest answer. It's a habit that a lot of us get into in business communication. We try to be short and declarative and stick to the point, especially when you're sending emails to an executive who only reads three sentences. Sometimes that's your manager. It's not my manager, but sometimes. We get into this habit of being really concise. That is an extremely useful habit in a lot of circumstances. In a retrospective, you want to ask complicated questions. They get complicated answers. This is a creative process, both for you and the other people who are participating. One of the things that all of you are trying to do together is imagine a better world. When you ask these better questions, what you're looking for is called a remediation item, an idea for a change that we can make to code, to documentation, to playbooks, to processes that will give a different outcome in the future. At Heroku, we call this contributing factor discovery because complex systems have complex failures, as we saw with the bridge. There's no one specific thing that caused that bridge to fail. If we talk about a single root cause, that constrains our thinking. It keeps us considering that there's only one reason that something broke, which is never the case. We don't deal with spherical cows on frictionless inclined planes. We deal with the real world, and it's very complex. We often hear that something went wrong because of human error, but human error isn't where you should stop your investigation. It's where you should start it. People make the best decisions that they can with the information they have at the time. How did the human make the error? What allowed the error to happen? How did the error take the system down? How long did it take the human to notice the error? The answers to those questions can give you things that you can change so that next time if the exact same thing happens, you'll get a different result. Fallacy, now that we know about this kind of mistake, we can just not do it again. That doesn't work. Who's going to get more sleep next week? We all think we're going to, are we? Probably not. Depending on humans to avoid errors is unreasonable. People can't be perfectly vigilant. They can't be better arrested. They can't just not have a screaming baby next time. They can't have a cat who's not trying to climb on them, and that's what he always does every night. The human you had today is the human you're going to have tomorrow and the human you're going to have next week, and you have to plan for that human, not an idealized human. Okay. Let's talk about how to have a great meeting. Who's with me? All right. If you're facilitating, have an agenda. Stick to it. Stay on time. Stay on topic. It helps keep you from missing things by running out of time. It lets participants know what you'll cover and when. It helps participants stay with the group because people who have fallen behind are self-conscious and are less likely to participate, and you won't be able to hear what they had to say about the incident and their perspective on it. Digressions can be boring for the group or can inhibit group participation, especially if one person is filibustering or has that hobby horse that they drag out every time. That discourages other people from sharing. Staying on track helps make sure everyone stays engaged. If you're a participant, your job is to help the group stay on track. Before you share something in a retrospective, ask yourself if what you have to say is on topic and needs the entire group's attention. If one of those things is not true, save it for a different context. Find a different way to bring it up. Try to work with the facilitator to keep everything moving forward. Keep your eye on the agenda. Stay engaged. Try not to respond to people slacking you. It's really hard, but try. Even if you have nothing to contribute right now, because if you're paying attention to someone else and what they say, you might have a response that moves the conversation further along. It's not just about what each particular person says. It's about the interaction between the participants and the conclusions that you can all come to together. Finally, note-taking. I don't have time to go into my very long rant about note-taking. Try to make sure it's fair. Try to rotate it. Pitch in if someone else is taking notes in a Google Doc or some other kind of shared format. Try to make sure that it's not always on one person, and it should never be on the person who's facilitating, because you can't pay attention to everyone else in the room and be taking notes at the same time. It's not a possible task. As a facilitator, you're going to need to practice interrupting. It's necessary when people start to rant, complain, get off topic. It can be super, super uncomfortable. Remind yourself that you can't achieve the goals of their retrospective if behaviors that take the meeting off track go unchecked. The best way to practice this is to think about a time that you experienced when someone needed to be interrupted and they weren't. Remember how unpleasant and frustrating the result was. Everyone's just sort of sitting there, sitting. You're pretty sure everyone else around you is really unhappy too. We're not getting anywhere. This person is taking out all the air out of the room, and we're not coming to any useful conclusions here. Remember that feeling wherever it feels in you. Then imagine yourself in that moment having said, thanks, teammate, but we're really far afield and we need to come back to the topic so that we can finish on time. Or everyone working this incident did the best they could, and we're here to figure out what we need to know to help things go better next time. Or even, wait a minute, please, if you need time to think. Do this gently but firmly. The person you're interrupting may even come to you and thank you later. That happened to me with a large billing outage that we had a few years ago. The senior exec for billing was really upset and I had to shut him down. He came to me after I was in set. I was really angry because I didn't realize that you were actually on it. I thought that because everyone was so calm and so good at this, it meant that you didn't care and I was wrong. So thank you for telling me. It's not always going to be like that. But sometimes it is. As a facilitator and as a participant, look at who's talking and who isn't. One of your jobs as a facilitator is to make sure everyone gets a chance to speak even if they choose to decline that chance because they don't have anything to say. Don't assume that just because a person is silent, it means they have nothing to contribute. They might be shy, they might be having a hard time interjecting, they might be having a bad day. Even eye on the participants to see who might want to speak and be afraid to interrupt. Things to watch for. Weakling, leaning forward, looking lips, biting lips, waving their hand a little bit. Lips pursed as if they're ready to speak, frowning, nodding, other really engaged behaviors. Sometimes those are the behaviors of someone who is excited to say something next. Sometimes it's active listening and you ask them if they have something to contribute and they don't in that moment, that's also fine. Try to take time at least twice during your retrospectives to stop the discussion and invite anyone who hasn't spoken to speak up. When you do that, you need to wait. Wait until it gets uncomfortable and then wait a little longer. Most times when people ask, do you have any, anyone have any questions, anyone have anything to say, they wait, okay, let's go. That's too quick. It's too quick for someone who needs time to marshal their thoughts or to marshal their courage or whatever it is that they need internally in order to be able to speak, you need to give them time for that. Let's talk about humor. Or we could not talk about humor, but let's talk about humor. It's really easy to mess up when you're making jokes in retrospective. You might have heard the saying comedy is tragedy plus time. In a retrospective, there has not been enough time. Furthermore, someone, maybe everyone in that retrospective feels like they screwed up and they cost the company money. They cost their teammates time and maybe sleep and they made everyone present have yet another meeting and maybe even their manager is there. So it's critical to avoid making people uncomfortable by making jokes that are inappropriate in this context. Jokes you should not make. Anything that makes anyone in the room feel even a little bit uncomfortable, even if they don't really notice it or they won't share it with you. A lot of times when people are uncomfortable, they're not ready to share that they're uncomfortable with the person who is the proximate cause of that feeling. Bad jokes. Anything your parents say when they're mad or they think you haven't been home recently enough, anything you say back, anything a manager or other employee ever said to you that made you feel bad. Anything you ever said to a co-worker that made them laugh uncomfortably and maybe once a little bit. Jokes about getting fired or firing people. Really a bad plan. Here's some things I shouldn't have to mention and I'm going to say them anyway. Race, gender, fat jokes, jokes about disabilities, any joke that contains the word crazy, jokes about any one person or their decisions or their wisdom or their plans or their face. You can probably make jokes about Murphy's law, entropy, computers being terrible. You can be really serious about silly things. One of my favorite examples is Las Vegas, colloquially known as fabulous. Maybe you can make a joke about something they are proud of about themselves. This is tricky. If you knew I liked making jokes about myself as the girl with her hair on fire, which I do, you might be able to make that joke. It might be okay. The issue with that joke is if somebody else in the room doesn't know that I think that's fun, it can make that person uncomfortable. I would usually still try to avoid it in a retrospective. If you didn't know I made that joke about myself, don't make it. If you can't make jokes, how do you listen to the mood? You can be kind, thoughtful, caring, call out successes. You can thank someone for their honesty and contributions. You can be warm and welcoming. What about when things go wrong? What do you do when other people blame or make bad jokes? What about when you make a mistake as a participant or a facilitator? If you mess up, you can tell you've made someone uncomfortable, you made a bad joke, you said something, blame me by accident, apologize, correct yourself and move on. It's important to be sincere, but matter of fact. Don't take time to wallow. Don't be performative. Just say, I'm sorry. Let me rephrase that. Let me ask you what you would have liked to know at the time. That's it. If you take time to feel shame and embarrassment and guilt in the retrospective, you're taking everyone else's time to feel that with you. It takes away from the mood of gentle inquiry and curiosity that we're trying to have. It's not effective if you're apologizing in almost any context. To put your guilt out to the person who's asked for your apology or to the person to whom you are apologizing. It's much better to process that yourself with close friends, with your manager, with your mentor, whatever and keep the meeting moving. Sometimes people will blame themselves and you might need to intervene. This is particularly hard. You just need a short request that they stop and then you need to move on. If the person is embarrassed or wants to explain, if they're having that moment where they want to have their guilt with everyone, give them just a moment, thank them sincerely and then move on again. Try things like this is a blameless retrospective and that means not blaming anyone, including yourself. We're here to make sure no one gets blamed, including you. Please don't blame yourself. That's not what we're here for. Critically you don't have to be witty. You don't have to be clever. You don't have to be on top of your game. You just need something very simple that asks someone to do something different than what they just did. Don't try to win. Don't try to one up anybody. Your goal is to remind people of the ground rules and keep things moving. If someone asks a blaming question, remind them that we're trying to avoid blames. This is a blameless retrospective. Can I help you rephrase that? Here's how I would suggest that you say what I think that you meant to say, does that sound right to you? And keep it going. Someone makes a bad joke. This is all you've got to say. Please don't make jokes like that in a retrospective. Very simple. Move on. You don't have to define it. You don't have to have a conversation about what the boundaries and the edges are and like was this inappropriate or was that inappropriate. All you have to say is please don't make jokes like that in a retrospective. During the retro is not the time to have a teachable moment or to try to stage an intervention. You don't need to convince anyone of anyone. You just need them to stop as quickly as possible so everyone else can get back to business. If their joke is something like I fat fingered it, again with a self-blame, you can gently say one of the things I said before or you can just ask them to not make a joke like that in a retrospective. That's also reasonable. Once you've gotten them to stop and the group has moved on, don't bring it up again. It's done. As far as you're concerned, it's finished. We never need to talk about this again. If they come to you and ask why it was inappropriate, you can choose to explain if you feel like you're in a place where you can have that conversation or you cannot. You can just reiterate that it's not appropriate for a retrospective and ask them to please not do it again. All right. So what's the point? Your retrospectives are part of the communication structures of the organization which produces the systems that you're running. It's turtles all the way down. Many people in this room are familiar with it, but I didn't know that Melvin Conway is still alive. I didn't know that he coined this in 1968. This is just such a brilliant insight to me that you can look at a service or an application and you can see the fault lines and where people didn't know how to talk to each other and where they weren't able to figure out what the UI should look like compared to what the API looks like before it all went out the door. Results of your retrospectives will be reflected in the product that you produce. Let's go make bigger, more interesting mistakes. Some acknowledgments. I won't read them all out. I am indebted to a lot of people for a lot of the sort of linguists and social concepts in here, some of the technical concepts, but also in how to teach and how to address people when I'm trying to show them and not just tell them what I'm trying to do. Thank you all for coming. It looks like I have a little bit of time for questions. Okay, so the question is how do you approach it when someone is senior and they are blaming or making uncomfortable junior folks in a retrospective? That sounds to me like the kind of cultural problem where you need probably an executive sponsor. You need someone near that person's level who is on board with the idea of a blame with social perspective who can talk about it and who can sort of help you spread that word to those folks. Just depending on the relationship that you have with the person who is doing the blaming, you might be able to take them aside afterwards and talk to them yourself. There are some people in my company where I think I have a good enough relationship that I would be able to do that, but it still always helps to have someone at the high level who is on board with what you're doing and can help evangelize. That is super hard. Sometimes, oh, I apologize. Yes, how do you interrupt someone who does not take a pause between their sentences? It's super hard. What I will do sometimes, all of our retrospectives are on hangouts because we're a remote company. I will try to break in a couple of times. In some ways, I have a slight advantage in that I'm usually the only woman in the meeting. If I'm speaking as the facilitator, I don't have to remind them why I'm speaking. The sound of my voice reminds them. As a facilitator, you might have to remind them. If you try and break in three times, you can't get in. I have not yet gone to the point of non-consensual muting, but that might be a thing. In a physical environment, I might even stand up and physically take the floor and say, the thing that you need to do, please save that for later. The question is, where do I find the knowledge base? I am not completely sure. If you come up to me afterwards, I can give you my card and I can give you the slides for the longer version of this, which has at least some of that, and that offers for anybody in the room. I don't know, but I also haven't been watching what other people have written. There might be more out there that I'm not aware of. Recommendations for a manager book club. The book that I recommended before is the Gentle Art of Verbal Self-Defense. It's a long title. There's a bunch of different versions of that book, which were written for specifically different contexts. I think that the original is actually quite good. I didn't think that the others added a lot in terms of contextuality. I always recommend that book with a little asterisk, which is that the author wrote it, and she is a pacifist. She was an absolute pacifist, and so she believed that it was always better to redirect, reframe, to take the hurt herself than to defend. And so you can see that in the way that she recommends handling some of these things. I don't always agree as an adult with her tactics, but I think that it's a really excellent way to start thinking about language. Other books. I feel like there's some, I need to, I will try to collect a list of books and tweet them. Yeah, okay. So the question is, what do you do when things get a little bit off track? They're still related to what you're talking about, but too in depth. And the shorthand we use at Heroku is you're off in the weeds, let's come back to the point, let's schedule a follow up if we need to talk about the technical details of this further. Most companies where I've talked to my peers, they have some shorthand like that, you're really involved in the details that are not appropriate for this moment. So I would look for that shorthand within your company if it exists. Thank you.
 All right, thank you for coming to Retrospectives for Humans.  Before we begin today, I want to acknowledge this has been a really frightening and difficult  few weeks for a lot of people with tragedy and violence touching a lot of people that  I know and if that's you, I'd like to offer my sympathies.  It can be difficult to function normally and pay attention to everyday business at times  like these.  It can be even harder to learn new concepts.  So please do what you need to do during this, take care of yourself.  I don't think this talk requires any content warnings.  If it does and I've missed it, I apologize, I would appreciate hearing from you afterwards.  All right, I'm here to talk about retrospectives.  These are incident retrospectives, but most of the things I'm going to say today will  probably also be useful to you in an agile retrospective, should you choose to do that.  The subtitle of this talk could also be words mean things.  Let's talk disasters where no one was hurt.  Seattle has the two longest floating bridges in the world and in 1990, one of them sank  over the Thanksgiving day weekend in a storm while it was being repurposed.  The public reporting to this day is extremely simplistic, but the official investigation  found that there were five factors involved and all of them were required for the bridge  to fail.  At the time that the bridge sank, you can see this is a picture of that happening, they  were scraping off the surface of the bridge with water, it's called hydro demolition.  They were going to change the direction of traffic on the bridge because they were building  a span nearby, so they were going to expand how much traffic the bridge could take.  The EPA told them that they were not allowed to let the water run off into the lake, that  that was a serious environmental problem, so they had to figure out a way to store the  water so that it could be trucked off and treated.  You can't store water in piles, so they needed to find a container to put it in.  The thing that occurred to them after much study was that they would store it in the  hollow pontoons of the bridge itself.  Floating bridge is made of these hollow sections like a ship's hull, it's made out of concrete  in this case and this bridge was actually floating higher than it was designed to float  and had lots of extra floatation ability that they decided to use in order to store the water  so they could truck it away.  When this happened, no one was on the bridge, it was over the holiday weekends everyone  was off and one pontoon sank slowly and dragged seven more down after it, so this was a slow  cascading failure, extra fun.  This is a video of some of the coverage of what happened.  I'm going to play it for a few minutes.  Built in 1940 in Seattle, a Lacey V. Murrow bridge was named after the director of Washington  State's Department of Transportation.  The bridge stood for 50 years, but in November of 1990, a rehabilitation and maintenance operation  on the bridge led to disaster.  Spanning a short but deep section of Seattle's Lake Washington, the Murrow bridge floated  on a row of 25 hollow concrete pontoons, each 350 feet long.  The pontoons were bolted together for stability and anchored to the bottom of the lake.  The conditions are just right for use of such a structure because the water conditions under  which they're installed are perfect.  It is very seldom is there any ice, very seldom is there a heavy current and the water is  very deep, 200 or more feet.  At the fall of 1990, the Murrow bridge was undergoing a facelift by means of hydro demolition,  a process in which highly pressurized water is used to break up the roadway.  Since environmental regulations prohibited dumping the contaminated water into the lake,  it was temporarily stored in the empty pontoons via access holes which were cut into the tops  and sides of some sections.  The water level inside the pontoons was carefully monitored so the structure would not be compromised.  But on Thanksgiving weekend with bridge crews short handed, a heavy storm hit the Seattle  area.  Rainwater drained into the open pontoons.  Winds from the storm whipped up the surface of the lake, propelling splash water into  the pontoons as well.  On the morning of November 25th, eight waterlogged pontoons sunk into the lake, taking those sections  of the bridge down with them and severing 12 anchoring cables of a bridge that was being  constructed nearby.  While luckily no one was injured, the cost of damage was estimated at $69 million.  An investigation of the accident came to a sobering, if unsurprising conclusion.  Their very extensive study and very sophisticated computer modeling methods the conclusion was  that concrete pontoons simply don't float when they're full of water.  A very unsophisticated motive.  The Lacey v. Murrow Bridge was rebuilt two years later.  Though it looks the same from the outside, the pontoons that support the bridge have  been divided into smaller sections to minimize damage from leaks, a precaution that underscores  the structural similarities between floating bridges and floating ships.  The pontoons were designed with multiple cells, not unlike the multiple cell super tankers  so that failure in one place would not bring down the entire pontoon.  So he got the laugh he was looking for.  That's not what I wanted to have happen.  Apologies.  Where is my mouse?  Okay.  So this was a giant disaster.  They checked a lot of stuff.  I'm not even going to read all of that off to you.  It was an incredibly thorough investigation.  I was able to get a paper that was published by the people who fronted this investigation  and read the original text of what they determined the failure to be.  This part I am going to read.  This is what they found was the problem, several problems.  The loads that created significant leakage were the combined effects of all accumulations  of water, including rain after the windstorm, longitudinal flow on the surface of the bridge,  and pumping through November 24th, 1990.  These loads caused static moments, that's to say pressures, that exceeded the threshold  for leakage.  Existing cracks were open sufficiently to allow water to leak into the pontoon.  Excessive and accelerated sinking began at this time.  So there's a lot of things that we've got going on here.  We found that there were existing cracks in the bridge.  The survey did in fact find that this bridge was built in the 40s, 30s.  He told you at the beginning of the video.  Now I can't remember.  So the bridge was in excellent shape for its age.  It had cracks that were expected in concrete of that age, but not as many as they would  have expected.  The cracks were smaller than expected.  The bridge was floating higher than designed.  That's the extra flotation capacity that I talked about before.  So the bridge was not just in decent shape.  It was in spectacular shape.  It was dramatically over engineered.  And so they found they needed to put the water someplace.  They did very careful surveys of the bridge and how much flotation there was, and they  calculated an error, like a buffer for their error, and worked really hard to make sure  that they were being careful when they did this.  It was very thorough.  But still somehow this happened.  The official investigation missed something huge.  The crews were having trouble pumping the water out on the schedule that was said by  the engineers.  And there is a mention of that in the paper, but there's no explanation as to why.  I don't know if they didn't do an investigation or if they simply didn't document the result  of their investigation.  But there was a social problem here.  There was a schedule set.  It involved trucks that were pumping water, carrying it off the bridge.  It involved water being stored evenly, hopefully across the pontoons that they were storing  it in.  And for some reason, crews couldn't stick to that schedule.  We don't know why.  And so we don't truly understand what happened here.  We know that the water was higher in some pontoons and lower in others, which was causing  torque on the bridge, which helped open those cracks.  We know that the trucking crews were behind.  There could have been not enough trucks.  There could have been not enough drivers.  There could have been a strike.  There could have been pay problems.  Any number of things could have gone into this.  And we're never going to truly understand.  So as thorough as this was, as really good as the true investigation was, and as different  as that is from the soundbite that made us all laugh, we still don't truly understand  what happened here, which is just such a good representation to me of the things that it's  easy to miss in a retrospective.  So we all laughed.  He says, you fill a bridge full of water at six.  But no one does things that they think are going to blow up the world.  It's easy to look at that and be like, oh, you stored water in the bridge.  Oh, my God.  But they stored water in the bridge very carefully with a very firm plan for how they were going  to continue to have this be OK, and it still didn't work.  So the lesson here is remember that even as you have that feeling of, like, you know,  face palm, here we go again, it's probably more complicated than that.  Asking about what a person knew and what the conditions were at the time that they made  a decision will give you good insight for effective change.  So let's do better than that.  Now, this is going to be a crash course.  I condensed, like, double the amount of content into this time.  So we're going to run fast.  If you have more questions, come find me afterwards.  Facilitation, we're going to talk about how to have a good meeting.  Other ways to think of this, servant leadership, psychological safety.  As a facilitator, your job is to keep the conversation blame-free, not let people make  bad jokes, and to run a pretty decent meeting that people are willing to sit in instead  of checking out of.  So let's talk about how we talk about all of this.  First thing, foundational Miller's law.  In order to understand what another person is saying, you must assume that it is true  and try to decide what it could be true of.  You're being asked to imagine what it is like to be another person, probably at another  time, at another place, in another context.  This is the only way to truly understand a system or a person or an environment or a  happenstance that was not one that you personally lived through, or even one that somebody else  lived through next to you, because no two people ever have the exact same experience,  even in the exact same moments.  I learned about this from Suzette Hiddon-Elgin in the general art of verbal self-defense,  which is a fairly old book at this point.  I do recommend it.  It talks a lot about language and how we talk to each other and the ways that we can create  conflict and de-escalate conflict.  This is a critical tool to use when you're conducting a retrospective.  Keep this in mind.  Started here.  Everything else is going to build on this.  Let's talk about blame.  English is a pretty blamey language.  I might say to myself or to another person, you knocked over that face.  Why did you knock over that face?  Those are both statements that are about blame and fault.  You.  Yeah, you.  I'm talking to you.  Don't just walk away from me.  That feels awful, right?  That's terrible.  Starting a sentence with you draws a line between yourself and the person you're speaking  to.  So does using it too often.  It creates an oppositional conversation.  There's the person saying you and the person who's being youed at.  Why?  Why is another part of blame?  Why did you do that?  Why did you do it like that?  That also feels terrible.  Asking a question that starts with why is a request for a justification.  It immediately puts the person you're addressing on the defensive.  Why questions get answers in a gentile language because I knocked it over?  The face is broken because I knocked it over.  A gentile language has blame in the very grammar.  It talks about a person doing a thing.  It's strongly remembered by English speakers.  So then after you've asked this question and you've gotten this answer, you've ensured  that everyone who heard it remembers that I knocked over the face.  Other things not to say to people, especially in a retrospective, but often at other times  too.  You always.  You never.  Every time we do this, you should.  You should just.  You should only do it like this.  There's more, but these are the least wanted.  If you have a sibling, anything your sibling ever shouted at you when they were mad that  made you see red goes on this list.  Anything that your child, if you have a child, has said to you that just made you want to  completely blow your top because kids are really good at that, goes on this list.  You might, if you're talking about an API or an RFC, use these words.  Especially like RFCs have this very prescriptive language, right?  So sometimes we can't avoid it.  But remember that when you describe the behavior of a system, you are describing the decisions  that some people made when they built it, right?  It's more upsetting if those people didn't know the consequences of the decisions that  they were making, which they didn't.  No one can be perfectly prescient.  So you are saying things about decisions that people made that they probably didn't even  understand that they were completely making.  Why didn't you just fix it the last time this happened?  How many times have we heard this, right?  This is terrible.  I'm going to take this apart because this is not compassionate and it's not effective  at helping us understand the problem.  This construct assumes that it's happened before, maybe more than once, that it could  have been fixed the last time it happened.  Easily, that's the just.  That you was the right one to fix it and that there was no good reason not to fix it.  That's a lot of assumptions in one very short sentence.  Here's some better things to say.  How could you or how could we?  What happened?  What do you know?  What did you know?  What if?  What if we tried this?  What if we tried that?  Could we next time try this?  What do you think about this proposal that I have?  What would you have wanted to know in the moment where you did that and you made that decision?  All of these are designed to get more complex answers when used in a question.  When you're asking a question, think about what you can say that will get you the longest  answer.  It's a habit that a lot of us get into in business communication.  We try to be short and declarative and stick to the point, especially when you're sending  emails to an executive who only reads three sentences.  Sometimes that's your manager.  It's not my manager, but sometimes.  We get into this habit of being really concise.  That is an extremely useful habit in a lot of circumstances.  In a retrospective, you want to ask complicated questions.  They get complicated answers.  This is a creative process, both for you and the other people who are participating.  One of the things that all of you are trying to do together is imagine a better world.  When you ask these better questions, what you're looking for is called a remediation  item, an idea for a change that we can make to code, to documentation, to playbooks, to  processes that will give a different outcome in the future.  At Heroku, we call this contributing factor discovery because complex systems have complex  failures, as we saw with the bridge.  There's no one specific thing that caused that bridge to fail.  If we talk about a single root cause, that constrains our thinking.  It keeps us considering that there's only one reason that something broke, which is  never the case.  We don't deal with spherical cows on frictionless inclined planes.  We deal with the real world, and it's very complex.  We often hear that something went wrong because of human error, but human error isn't where  you should stop your investigation.  It's where you should start it.  People make the best decisions that they can with the information they have at the time.  How did the human make the error?  What allowed the error to happen?  How did the error take the system down?  How long did it take the human to notice the error?  The answers to those questions can give you things that you can change so that next time  if the exact same thing happens, you'll get a different result.  Fallacy, now that we know about this kind of mistake, we can just not do it again.  That doesn't work.  Who's going to get more sleep next week?  We all think we're going to, are we?  Probably not.  Depending on humans to avoid errors is unreasonable.  People can't be perfectly vigilant.  They can't be better arrested.  They can't just not have a screaming baby next time.  They can't have a cat who's not trying to climb on them, and that's what he always  does every night.  The human you had today is the human you're going to have tomorrow and the human you're  going to have next week, and you have to plan for that human, not an idealized human.  Okay.  Let's talk about how to have a great meeting.  Who's with me?  All right.  If you're facilitating, have an agenda.  Stick to it.  Stay on time.  Stay on topic.  It helps keep you from missing things by running out of time.  It lets participants know what you'll cover and when.  It helps participants stay with the group because people who have fallen behind are  self-conscious and are less likely to participate, and you won't be able to hear what they had  to say about the incident and their perspective on it.  Digressions can be boring for the group or can inhibit group participation, especially  if one person is filibustering or has that hobby horse that they drag out every time.  That discourages other people from sharing.  Staying on track helps make sure everyone stays engaged.  If you're a participant, your job is to help the group stay on track.  Before you share something in a retrospective, ask yourself if what you have to say is on  topic and needs the entire group's attention.  If one of those things is not true, save it for a different context.  Find a different way to bring it up.  Try to work with the facilitator to keep everything moving forward.  Keep your eye on the agenda.  Stay engaged.  Try not to respond to people slacking you.  It's really hard, but try.  Even if you have nothing to contribute right now, because if you're paying attention to  someone else and what they say, you might have a response that moves the conversation  further along.  It's not just about what each particular person says.  It's about the interaction between the participants and the conclusions that you can all come  to together.  Finally, note-taking.  I don't have time to go into my very long rant about note-taking.  Try to make sure it's fair.  Try to rotate it.  Pitch in if someone else is taking notes in a Google Doc or some other kind of shared  format.  Try to make sure that it's not always on one person, and it should never be on the person  who's facilitating, because you can't pay attention to everyone else in the room and  be taking notes at the same time.  It's not a possible task.  As a facilitator, you're going to need to practice interrupting.  It's necessary when people start to rant, complain, get off topic.  It can be super, super uncomfortable.  Remind yourself that you can't achieve the goals of their retrospective if behaviors that  take the meeting off track go unchecked.  The best way to practice this is to think about a time that you experienced when someone  needed to be interrupted and they weren't.  Remember how unpleasant and frustrating the result was.  Everyone's just sort of sitting there, sitting.  You're pretty sure everyone else around you is really unhappy too.  We're not getting anywhere.  This person is taking out all the air out of the room, and we're not coming to any useful  conclusions here.  Remember that feeling wherever it feels in you.  Then imagine yourself in that moment having said, thanks, teammate, but we're really far  afield and we need to come back to the topic so that we can finish on time.  Or everyone working this incident did the best they could, and we're here to figure  out what we need to know to help things go better next time.  Or even, wait a minute, please, if you need time to think.  Do this gently but firmly.  The person you're interrupting may even come to you and thank you later.  That happened to me with a large billing outage that we had a few years ago.  The senior exec for billing was really upset and I had to shut him down.  He came to me after I was in set.  I was really angry because I didn't realize that you were actually on it.  I thought that because everyone was so calm and so good at this, it meant that you didn't  care and I was wrong.  So thank you for telling me.  It's not always going to be like that.  But sometimes it is.  As a facilitator and as a participant, look at who's talking and who isn't.  One of your jobs as a facilitator is to make sure everyone gets a chance to speak even if  they choose to decline that chance because they don't have anything to say.  Don't assume that just because a person is silent, it means they have nothing to contribute.  They might be shy, they might be having a hard time interjecting, they might be having  a bad day.  Even eye on the participants to see who might want to speak and be afraid to interrupt.  Things to watch for.  Weakling, leaning forward, looking lips, biting lips, waving their hand a little bit.  Lips pursed as if they're ready to speak, frowning, nodding, other really engaged behaviors.  Sometimes those are the behaviors of someone who is excited to say something next.  Sometimes it's active listening and you ask them if they have something to contribute  and they don't in that moment, that's also fine.  Try to take time at least twice during your retrospectives to stop the discussion and  invite anyone who hasn't spoken to speak up.  When you do that, you need to wait.  Wait until it gets uncomfortable and then wait a little longer.  Most times when people ask, do you have any, anyone have any questions, anyone have anything  to say, they wait, okay, let's go.  That's too quick.  It's too quick for someone who needs time to marshal their thoughts or to marshal their  courage or whatever it is that they need internally in order to be able to speak, you need to  give them time for that.  Let's talk about humor.  Or we could not talk about humor, but let's talk about humor.  It's really easy to mess up when you're making jokes in retrospective.  You might have heard the saying comedy is tragedy plus time.  In a retrospective, there has not been enough time.  Furthermore, someone, maybe everyone in that retrospective feels like they screwed up and  they cost the company money.  They cost their teammates time and maybe sleep and they made everyone present have yet another  meeting and maybe even their manager is there.  So it's critical to avoid making people uncomfortable by making jokes that are inappropriate in  this context.  Jokes you should not make.  Anything that makes anyone in the room feel even a little bit uncomfortable, even if they  don't really notice it or they won't share it with you.  A lot of times when people are uncomfortable, they're not ready to share that they're uncomfortable  with the person who is the proximate cause of that feeling.  Bad jokes.  Anything your parents say when they're mad or they think you haven't been home recently  enough, anything you say back, anything a manager or other employee ever said to you  that made you feel bad.  Anything you ever said to a co-worker that made them laugh uncomfortably and maybe once  a little bit.  Jokes about getting fired or firing people.  Really a bad plan.  Here's some things I shouldn't have to mention and I'm going to say them anyway.  Race, gender, fat jokes, jokes about disabilities, any joke that contains the word crazy, jokes  about any one person or their decisions or their wisdom or their plans or their face.  You can probably make jokes about Murphy's law, entropy, computers being terrible.  You can be really serious about silly things.  One of my favorite examples is Las Vegas, colloquially known as fabulous.  Maybe you can make a joke about something they are proud of about themselves.  This is tricky.  If you knew I liked making jokes about myself as the girl with her hair on fire, which I  do, you might be able to make that joke.  It might be okay.  The issue with that joke is if somebody else in the room doesn't know that I think that's  fun, it can make that person uncomfortable.  I would usually still try to avoid it in a retrospective.  If you didn't know I made that joke about myself, don't make it.  If you can't make jokes, how do you listen to the mood?  You can be kind, thoughtful, caring, call out successes.  You can thank someone for their honesty and contributions.  You can be warm and welcoming.  What about when things go wrong?  What do you do when other people blame or make bad jokes?  What about when you make a mistake as a participant or a facilitator?  If you mess up, you can tell you've made someone uncomfortable, you made a bad joke, you said  something, blame me by accident, apologize, correct yourself and move on.  It's important to be sincere, but matter of fact.  Don't take time to wallow.  Don't be performative.  Just say, I'm sorry.  Let me rephrase that.  Let me ask you what you would have liked to know at the time.  That's it.  If you take time to feel shame and embarrassment and guilt in the retrospective, you're taking  everyone else's time to feel that with you.  It takes away from the mood of gentle inquiry and curiosity that we're trying to have.  It's not effective if you're apologizing in almost any context.  To put your guilt out to the person who's asked for your apology or to the person to  whom you are apologizing.  It's much better to process that yourself with close friends, with your manager, with  your mentor, whatever and keep the meeting moving.  Sometimes people will blame themselves and you might need to intervene.  This is particularly hard.  You just need a short request that they stop and then you need to move on.  If the person is embarrassed or wants to explain, if they're having that moment where they want  to have their guilt with everyone, give them just a moment, thank them sincerely and then  move on again.  Try things like this is a blameless retrospective and that means not blaming anyone, including  yourself.  We're here to make sure no one gets blamed, including you.  Please don't blame yourself.  That's not what we're here for.  Critically you don't have to be witty.  You don't have to be clever.  You don't have to be on top of your game.  You just need something very simple that asks someone to do something different than what  they just did.  Don't try to win.  Don't try to one up anybody.  Your goal is to remind people of the ground rules and keep things moving.  If someone asks a blaming question, remind them that we're trying to avoid blames.  This is a blameless retrospective.  Can I help you rephrase that?  Here's how I would suggest that you say what I think that you meant to say, does that sound  right to you?  And keep it going.  Someone makes a bad joke.  This is all you've got to say.  Please don't make jokes like that in a retrospective.  Very simple.  Move on.  You don't have to define it.  You don't have to have a conversation about what the boundaries and the edges are and  like was this inappropriate or was that inappropriate.  All you have to say is please don't make jokes like that in a retrospective.  During the retro is not the time to have a teachable moment or to try to stage an intervention.  You don't need to convince anyone of anyone.  You just need them to stop as quickly as possible so everyone else can get back to business.  If their joke is something like I fat fingered it, again with a self-blame, you can gently  say one of the things I said before or you can just ask them to not make a joke like  that in a retrospective.  That's also reasonable.  Once you've gotten them to stop and the group has moved on, don't bring it up again.  It's done.  As far as you're concerned, it's finished.  We never need to talk about this again.  If they come to you and ask why it was inappropriate, you can choose to explain if you feel like  you're in a place where you can have that conversation or you cannot.  You can just reiterate that it's not appropriate for a retrospective and ask them to please  not do it again.  All right.  So what's the point?  Your retrospectives are part of the communication structures of the organization which produces  the systems that you're running.  It's turtles all the way down.  Many people in this room are familiar with it, but I didn't know that Melvin Conway  is still alive.  I didn't know that he coined this in 1968.  This is just such a brilliant insight to me that you can look at a service or an application  and you can see the fault lines and where people didn't know how to talk to each other  and where they weren't able to figure out what the UI should look like compared to what  the API looks like before it all went out the door.  Results of your retrospectives will be reflected in the product that you produce.  Let's go make bigger, more interesting mistakes.  Some acknowledgments.  I won't read them all out.  I am indebted to a lot of people for a lot of the sort of linguists and social concepts  in here, some of the technical concepts, but also in how to teach and how to address  people when I'm trying to show them and not just tell them what I'm trying to do.  Thank you all for coming.  It looks like I have a little bit of time for questions.  Okay, so the question is how do you approach it when someone is senior and they are blaming  or making uncomfortable junior folks in a retrospective?  That sounds to me like the kind of cultural problem where you need probably an executive  sponsor.  You need someone near that person's level who is on board with the idea of a blame with  social perspective who can talk about it and who can sort of help you spread that word  to those folks.  Just depending on the relationship that you have with the person who is doing the blaming,  you might be able to take them aside afterwards and talk to them yourself.  There are some people in my company where I think I have a good enough relationship  that I would be able to do that, but it still always helps to have someone at the high level  who is on board with what you're doing and can help evangelize.  That is super hard.  Sometimes, oh, I apologize.  Yes, how do you interrupt someone who does not take a pause between their sentences?  It's super hard.  What I will do sometimes, all of our retrospectives are on hangouts because we're a remote company.  I will try to break in a couple of times.  In some ways, I have a slight advantage in that I'm usually the only woman in the meeting.  If I'm speaking as the facilitator, I don't have to remind them why I'm speaking.  The sound of my voice reminds them.  As a facilitator, you might have to remind them.  If you try and break in three times, you can't get in.  I have not yet gone to the point of non-consensual muting, but that might be a thing.  In a physical environment, I might even stand up and physically take the floor and say,  the thing that you need to do, please save that for later.  The question is, where do I find the knowledge base?  I am not completely sure.  If you come up to me afterwards, I can give you my card and I can give you the slides  for the longer version of this, which has at least some of that, and that offers for  anybody in the room.  I don't know, but I also haven't been watching what other people have written.  There might be more out there that I'm not aware of.  Recommendations for a manager book club.  The book that I recommended before is the Gentle Art of Verbal Self-Defense.  It's a long title.  There's a bunch of different versions of that book, which were written for specifically  different contexts.  I think that the original is actually quite good.  I didn't think that the others added a lot in terms of contextuality.  I always recommend that book with a little asterisk, which is that the author wrote  it, and she is a pacifist.  She was an absolute pacifist, and so she believed that it was always better to redirect, reframe,  to take the hurt herself than to defend.  And so you can see that in the way that she recommends handling some of these things.  I don't always agree as an adult with her tactics, but I think that it's a really excellent  way to start thinking about language.  Other books.  I feel like there's some, I need to, I will try to collect a list of books and tweet them.  Yeah, okay.  So the question is, what do you do when things get a little bit off track?  They're still related to what you're talking about, but too in depth.  And the shorthand we use at Heroku is you're off in the weeds, let's come back to the point,  let's schedule a follow up if we need to talk about the technical details of this further.  Most companies where I've talked to my peers, they have some shorthand like that, you're  really involved in the details that are not appropriate for this moment.  So I would look for that shorthand within your company if it exists.  Thank you.
RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_0_59.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_60_155.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_156_275.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_276_407.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_408_1967.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_1968_4475.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_4476_4643.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_4644_5411.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_5412_5903.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_5904_6119.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_6120_6251.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_6252_6599.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_6600_6647.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_6648_7055.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_7056_7355.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_7356_7835.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_7836_8735.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_8736_8807.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_8808_8951.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_8952_9131.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_9132_9419.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_9420_9527.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_9528_10295.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_10296_15227.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_15228_17147.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_17148_18683.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_18684_19151.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_19152_23651.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_23652_25415.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_25416_27263.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_27264_36119.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_36120_37199.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_37200_38423.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_38424_40439.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_40440_40715.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_40716_42203.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_42204_43163.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_43164_44075.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_44076_45695.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_45696_47219.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_47220_48263.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_48264_50003.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_50004_50063.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_50064_50543.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_50544_50639.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_50640_53507.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_53508_53675.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_53676_53723.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_53724_53831.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_53832_53879.png RubyConf_2018_-_Retrospectives_for_Humans_by_Court_s7R7V5wC0wA_53880_53997.png