How My Team Does Agile, 2014 Edition

I’ve spent a lot of time and energy over the past few years trying to get my team doing agile software development in a way that feels good to me. We’ve really come a long way, and we’re really getting close to where I want us to be.

Where We Started

When I first joined the team, I was really unhappy with our agile practices. We were running two-week sprints. Before a sprint started, we’d have two meetings: pre-planning and sprint planning. In pre-planning, we’d have 12 developers on their laptops and phones as we went through their assignments person by person. Nobody was invested in anything that anybody else was working on, and so they didn’t bother to pay attention. Everybody would leave with their individual assignments, and they’d come up with the tasks they’d work on to email to the team lead before the sprint planning meeting.

Sprint planning was even worse. We would literally watch the team lead copy/paste tasks from emails into a spreadsheet to be inserted into TFS. There’d be no input or feedback from the team on the tasks, and everybody would just sit and wait for their turn to read their tasks as they were being entered into the spreadsheet. It sounds bad, but it got worse. The cell phone use and not-paying-attention lead to a ban on cell phones and laptops, so you’d just have to sit there and try not to fall asleep.

Coming out of sprint planning, you’d have a list of tasks that you came up with that nobody paid any attention to. There was no accountability. You could probably submit the same list of tasks two sprints in a row without being questioned. But that’s not even the worst part!

The biggest problem that I saw was what I describe as a “culture of failure.” Nobody was completing their tasks in the sprint, and nobody cared. At the end of the sprint, we’d just close ’em out and make new ones, no questions asked. To this day, I can’t wrap my head around how an entire team of developers can be responsible for coming up with their own tasks with their own estimates with no questions asked and not complete them all EVER! (Deep breaths, self… Writing about the past is conjuring some bad juju, and I’m getting angry.)

Where We Are Now

So, yea. That was where we were, and I’m happy to say that we’ve come a long way. I believe we’re experiencing a lot of success today primarily because of a few key changes. We transformed a large team of INDIVIDUALS into lean execution TEAMS, we shortened our sprints from two weeks to just one, we started to focus on our backlog, and we stopped accepting work into sprints unless we believed it could be completed.

Converting our single large team into three smaller execution teams was a big challenge. We had to look at our developers and identify who might and might not work well together. I think we did a pretty good job with that since it’s been about a year, and we’ve only made one or two “trades” between the teams. In order to build the team mentality, we’re assigning work to the team instead of the individuals. The teams are responsible for determining how work is divided, and we really don’t care how it gets done as long as it gets done. Each of our three teams operates a little differently, and each of them is more functional than the big glob we had before.

But the small teams weren’t enough. We were still having problems with planning enough work to get into a sprint. The result is that halfway through, we’d have a lot of items that were blocked or no longer needed. This is mostly because we were stretching to scrape up enough work to fill the sprint, so a lot of what made it in wasn’t ready. That meant a lot of time spent working on things that we didn’t plan for or possibly not working on anything! Additionally, we’d have distractions coming up constantly that couldn’t wait for the next sprint–so that’s more items being pushed out or not worked on. Shortening sprints to one week addressed a lot of those issues. We don’t need as big of a backlog since we only need a week’s worth of work at a time. Distractions are less of a problem because we’re never more than a week away from the next sprint; it’s much easier to tell a customer than you can’t do something for a few days than a few weeks.

With shorter sprints implemented, we could focus on our backlog and ensuring that we have enough work ready to go with each sprint. This was a huge shift. Instead of asking developers what they were working on, we were giving them assignments based on project needs and priorities. If there was any question about the complete-ability of an item, we’d pull it out of the sprint and either add a task to improve its complete-ability or replace it with something else entirely.

So let’s review what we’ve got now: teams that are invested in what their members are working on and short sprints filled with items that can actually be completed. We’re still not completing 100% of our sprint work each week, but we’re having more success than we’ve ever had before.

What Comes Next

The team’s in a good place, but we’ve still got a lot to improve on. We don’t do a great job of story-writing. Our backlog has a lot of “do X” stories that don’t provide much context. Why are we doing that? What else do we need to get where we’re going? Because of this approach, we have a lot of new work that pops up at the end of the sprint as we realize that we now have to “do Y” and “do Z” before we’re done with a certain feature.

So my next focus will be on making sure we write quality stories. Let’s have non-functional stories to create the system functionality needed to complete bigger functional stories. Let’s make sure our stories have valid descriptions and clear completion criteria. Let’s scope stories so we can confidently fit them into a single sprint. Let’s identify the functional stories needed to complete a project so we can have a clear picture of what “done” means before we begin, sharpening our focus on what we’re trying to accomplish while simultaneously building a strong backlog. Yes, the future will be good!

Testing Code Paths vs. Testing Behavior

I have a colleague that’s my equal in terms of unit testing enthusiasm, but we have very different philosophies. He tends to write methods first, then test the hell out of them to ensure that all code paths have been covered and that there are no holes. I tend to code using more of a TDD workflow, writing tests for each behavior that I expect from a method and not worrying about anything else that may or may not being going on.

Both approaches are valid. As we code, we both think about things that could go wrong with our code, and we both account for those things and make sure they’re tested. At the end of the day, we both end up with relatively bug free solutions that work well. Both methods produce high levels of code coverage. although focusing test writing on code paths will likely result is slightly higher coverage since the tests.

Yes, there’s a lot that’s similar about these two different approaches, but the differences are very important. The TDD mantra is “red, green, refactor.” The idea is that you write a failing test, add code to make the test pass, and then refactor the solution to clean up and optimize. This workflow is made for behavior-based testing. You expect a certain result from the method being tested. Once it’s producing that result, it shouldn’t stop producing it due to refactoring or optimizations.

The same statement can be made for tests written based on code paths: an expected result should continue to be produced after code is optimized. I’m venturing to say that optimizations are less likely to occur with the code-first approach, though. When you write code first, you don’t write tests until you’re done. And, since you’re writing tests based on the “finished” code, it’s less likely that you’ll discover flaws. Refactoring also seems less likely for the same reason. If refactoring does occur–which it should–then there’s a different problem: code paths that were once different may now be the same. You may have unknowingly written duplicate tests! (That’s not to say that the duplicate or redundant tests are bad, but you’ll have spent time writing code that, in the end, didn’t need to be written.)

Every developer I’ve ever met has learned to code before they’ve learned to write unit tests. Unit tests are generally written in code, so it’s hard to imagine learning them in any other order. Because we learn these two things in that order, we generally learn to write unit tests by following code paths. If you’re one of those code-first-and-write-tests-later types, I urge you to step out of your comfort zone and start writing behavior-based tests FIRST. You’ll code with purpose and write meaningful tests. You’ll be able to refactor with confidence, knowing that your code’s behavior has been unaffected by your chances. Like any skill, it takes some time to get used to, but I strongly believe you’ll produce higher quality code more efficiently once you become proficient.

Make the Invisible More Visible

A co-worker sent me a link to an article from the Embedded Linux Conference (ELC) where the author spoke with Robert Rose of SpaceX about lessons learned, primarily related to software development. The article references an essay titled Make the Invisible More Visible, from the book 97 Things Every Programmer Should Know. I really like some of the points the essay makes about the danger of invisibility in the development process. I particularly like these two points:

  • If you’re 90% done and endlessly stuck trying to debug your way through the last 10% then you’re not 90% done
  • Lack of visible progress is synonymous with lack of progress

These two statements really go hand in hand. In my experience, developers will often say they’re 80 or 90 percent done when they feel like they’re close to done (or that they should be close to done). In many cases, “90% done” is an arbitrary measure of progress, manufactured to give the illusion that things are on track. Typically in these situations, we’re looking at projects with loosely-defined requirements and insufficient planning. The developer was given a task and hit the ground coding without thinking about the big picture.

Without understanding what’s left, developers can’t be held accountable. When I ask a developer whose been working on a project for two weeks how far along they are, it seems pretty good when they report 80 percent. When I come back a week later and ask for another update, maybe they’re at 90 percent. “Progress has slowed down,” I think to myself, “but at least we should be done next week.” Next week comes along, and now we’re up to 95 percent. This is a perfect illustration of the points above.

In this scenario, the developer reported being at 80% when clearly they weren’t. There was no visibility to what they had accomplished or had yet to accomplish, so the measure was based on feelings—which proved to be completely inaccurate. But how can this be avoided? Here’s my advice: know what you’re coding before you start coding. I’m not talking about classes and methods, though; I’m talking about features and requirements. If you follow agile methodologies, you should be familiar with the concept of user stories. Before you start working on a project, take the time to understand and document the stories required to complete the project. This brings visibility to the project as well as giving you a calculation-based measure of completion percentage. There’s no question about what’s been accomplished and what’s left. As long as you keep completing stories, consistent progress is all but guaranteed.

But what if it’s just a lot of stuff that comes up during testing? If you’re writing good stories with solid completion criteria, you shouldn’t be finding a lot during testing. Additionally, you should be creating automated unit tests to verify what you’re doing and what you’ve done. The automated tests ensure that functionality added early in the development process isn’t affected by functionality added later.

Other features keep popping up; how do I manage those? This depends on the source of the new features. If it’s something that you missed, you just need to be aware of this and improve over time. With practice, you’ll get better and learn what types of things you commonly miss. Peer review can really help with this, too. Sit down with a buddy and walk them through what you’ve got, or work together to come up with the requirements from the beginning. With two of you, you’ll be less likely to miss something. If the source is external—like bosses and customers—your best bet is open, honest communication. This is textbook scope creep, so let them know what the impact of their request will be. “Sure, we can do that, but it’s going to take us an extra two weeks.” If it’s not in the budget, you have to push back or barter. “I’m sorry, but that’s out of scope, and we don’t have the capacity to add that requirement” or, “We can only implement that functionality if we leave out these other two things.”

“A late change in requirements…”

A late change in requirements is a competitive advantage
–Mary Poppendieck

My team works almost exclusively on small, custom projects. Each project begins with high-level requirements gathering, an estimation of effort, and a cost proposal. If the customer accepts the proposal, a contract is signed, and my team works with the customer to create a more detailed requirements document before beginning development.

The problem with this process is that it isn’t very agile, and, even though we complete the project by converting the requirements document into stories and banging them out in sprints, we sometimes fall into the same old waterfall-pitfalls. Requirements written at the beginning are not always the right requirements. Something that seems very important at the start might not make sense by the end. Or, there might be aspects that weren’t considered during requirements gathering, which can lead to important requirements that went undocumented.

At the end of any project, what matters most is that the customer feels good about the business value they’re getting from what they bought. If a project satisfies all of its requirements but ultimately provides no value to the customer, the project is a failure. Conversely, if a project meets only a subset of its requirements but delights its customer, it can be considered a success.

Realizing that requirements will change–and expecting them to–is an important strength of agile methodologies. As features are completed, review them with the customer and re-evaluate what comes next. Sometimes, you might find a feature is no longer needed. More likely, you’ll uncover a feature that was missed but will provide much greater value than what was originally proposed. Shifting requirements late in the game is how you can take advantage of a newly found feature like this, and that’s part of what makes agile so powerful.

Lean Development Teams


I’ve long been a believer of the idea that having only a single developer on a project is a recipe for disaster. First and foremost, you’ve got a code-red, emergency-level, bus factor of 1. (Very bad.) The next problem is that you have no buffer for the individual’s personal strengths and weaknesses. For example, if the assigned developer happens to test only base cases and doesn’t check any edge cases for abnormal behavior, the application quality is likely to reflect that.

I was the lone developer earlier in my career, and I didn’t like it. I wanted to be part of a team. I went from one extreme to the other, though, by joining one of the largest development teams in the company.

I soon grew to learn that very large teams have pretty much the same problems as teams-of-one. In my case, we had a team that was so large, nobody could keep track of what anybody else was doing. Or rather, nobody cared. It was too much. Sprint planning was–and continues to be–a challenge because team members can’t stay focused and engaged as we discuss 10 projects that aren’t related to them. Stand-ups are the same: give my update and zone-out for the rest. We were a big team, but we were a team of individuals. And with that came all the same lone-wolf issues. Specialized knowledge was retained by certain people. As new team members joined, they would be given their own projects. Everybody else would be too busy with their own projects and not available to give the amount of attention and detail required and warranted by a new teammate. And, perhaps most concerning of all, the quality of a customer’s project was largely dependent on which developer was assigned to work on it.

So how can this be fixed?

We’re in the process of restructuring the team into smaller, virtual teams. At the same time, we’re working on building and maintaining a true, prioritized backlog.

As we begin the transition, developers will bring their individual projects with them as they join the virtual team. Our prioritization effort included all of the currently active projects, so the teams are essentially pre-loaded with their own mini-backlogs. The teams will be responsible for reviewing these backlogs, re-estimating the amount of effort required to achieve project closure, and executing. Teams will be able to plan and hold themselves accountable. When the team backlog is clear, it’s time to get the next item from the greater-team’s backlog.

That’s the plan, at least. Making our big team more agile is something that I’ve been trying to focus on for the past year and a half. We’ve had some successes and some less-than-successes, but we’re committed to improving. I think this will be a welcomed change, and I’m optimistic that it will energize the team. Developers will be able to work closely with each other in a much more collaborative environment. At the same time, knowledge sharing will occur naturally, and individuals’ strengths and weaknesses will offset each other a bit.

To quote a former co-worker, “I’m feeling pretty jacked right now.” This is a change that I’m passionate about, and I really believe it’s going to help take my team to the next level. I’m sure I’ll post again with an update on our progress, but in the meantime, have you been through a similar experience? I’d love to hear lessons-learned, tips, or advice. Do share!

Traditional vs. Agile Development

I was poking around the Microsoft Patterns & Practices site on MSDN, and I found a terrific comic that illustrates traditional versus agile development.

Just today, I was doing project planning with co-worker where we were discussing “just doing all the data access at once” versus “building basic (but testable!) end-to-end functionality and enhancing as we progress.” This comic happens to do a great job of comparing these two approaches and highlighting the risks associating with building individual components separately without an appropriate test framework. If only Dr. Frankenstein had been more agile…

Agile Story Points

User stories are one of the core concepts in agile software development. You need to build and maintain a prioritized backlog of estimated stories. Stories are accepted into an iteration from the top of the backlog during iteration planning, and the assigned resource becomes responsible and is held accountable to complete the story within the iteration.

One of the challenges with this process is estimating stories to ensure that they are scoped correctly to safely fit into and be completed within a single iteration. This is where story points enter the equation.

Story points are intended to provide a relative scale of effort for stories, but the unit of a story point is very open to interpretation and seems to vary throughout the community. To me, the most obvious unit for story points is hours. Others have suggested that you use a single, perfect day as the basis, or use a half-day. The problem I have with these suggestions is that there is an implicit conversion that occurs. And what about a story that will only take an hour or two? Days-as-points is too coarse.

Another suggestion I’ve heard is to use the simplest story you can think of as your points baseline. I like this idea. The problem is that I’d estimate the simplest story I can think of to take one hour, and then I’m right back to hours. Alternative units don’t seem to be alternative at all. Instead, I’m just doing extra math to abstract the numbers into something other than hours. At the end of the day, though, everything is still hours-based.

That’s how I felt about story points for a long time. I recently started thinking about story points in a slightly different way, and it’s working for me. Instead of converting to a different unit, I’ve been thinking about story points as an hours-based estimate that is equal parts effort, uncertainty, and complexity.

There are stories that I know will ultimately be fixed by a line or two of code, but they need some figuring-it-out time. The final effort required to complete the story is minimal since it’s probably just going to be a couple of lines. Assigning story points based on minimal effort that’s been adjusted to account for risk due to uncertainty and complexity works great. The story points estimate for this story will be comparable to a high-effort-but-simple, brute-force-type of story.

At the end of the day, the story points scale you and your team uses doesn’t matter as long as it works for you. You want your stories to have a story points estimate that indicates their total required effort, taking into account risk due to uncertainty and complexity. The team should feel good about accepting a story into the iteration and getting it done. You should be able to use the story point estimates from completed stories to determine your team’s velocity and predict future results.

Have you been down a similar path of enlightenment? What have you found that works or doesn’t work?

The Art of Agile Story Writing

Prioritizing user stories

Some of the core concepts of agile software development methodology revolve around converting business requirements into stories that can be tasked out and scheduled into iterations. Without good stories, development efforts can be severely hindered, and the development team’s performance will be less predictable. When the team consistently fails to complete stories and deliver on commitments, a cultural problem is presented.

I’m making the claim that complete, well-written stories are the backbone of a successful agile team. Coming up with these stories is something that teams I’ve been a part of have struggled with for many years, and I think the trouble has primarily come from two things: not understanding what constitutes a good story and not enough design time. We would have poorly written stories with no acceptance criteria handed off prematurely to developers. The developers are then faced with an uphill struggle. Assumptions and guesses are made and the lack of acceptance criteria gives no safety net.

In order to write a good story, you must first understand what the story must provide. Here are the key characteristics of a good story:

  • Describes the business need or value provided
  • Verifiable acceptance criteria
  • Complete-able within a single iteration/sprint

Describing the business need is important for several reasons. First, talking in terms of the business instead of specific implementation details or developer-speak will help when communicating with the non-developer stakeholders. It also keeps you from locking into a specific implementation. I was recently on a requirements gathering trip with my manager. My developer mind instantly jumped into how we’ll do it instead of what we’re doing. The next day, I came up with a better way of accomplishing the what. If we’d gone forward and made the implementation details part of the requirements, it would be a bigger hassle to change those details later in the development process. It was this moment that really opened my eyes to the importance of focusing on the business requirements when gathering requirements and resisting the urge to document implementation details as requirements.

Verifiable acceptance criteria is, perhaps, the most important aspect of a good story. With a complete list of acceptance criteria that can be verified, a developer will be able to overcome any other story shortcomings. The criteria must not be subjective and not open to interpretation. When I mentioned not enough design time previously, this is where that design time was missing. Our team wasn’t spending enough time coming up with the acceptance criteria and defining what it means to be done. Our open-ended stories were doomed to drag on through several sprints and become dumping grounds for miscellaneous tasks that arose because of missed requirements and afterthoughts. Before scheduling a story for development, ample time must be spent thinking about the measurable goals of the story. Those goals should be documented as acceptance criteria, and they should provide developers with a clear view of what “done” is.

The third point, stories must be complete-able within a single iteration, is about keeping stories scoped appropriately. When a story is too big, it’s bound to linger through many sprints. It needs to be broken into smaller stories. When a story is expected to be completed in a single sprint, it’s easier to hold team members accountable when they don’t. Each time a story is completed within its scheduled sprint, that’s a success. As successes begin to happen consistently, a culture change occurs. Once stories can be reliably completed within a sprint, the team becomes highly predictable. That’s a huge win on many fronts.

The symptoms that my team’s stories were suffering from were really a side-effect of having too many developers on teams without enough story writers to put in the necessary legwork to create a solid story. The result is that stories were turned over hastily in an effort to keep everybody busy and productive, but it came at a cost. The stories had loose requirements that were open to a lot of interpretation. Developers made guesses and assumptions, some of which proved to be incorrect. Those areas where the mark was missed likely resulted in buggy software or unsatisfied customers, which are equally undesirable. If you find yourself in a position like that where there are too many developers for the number of stories, it’s important to come up with a way to change that. Tap some of your senior developers and have them assist with story-writing.

Good stories won’t solve all of a development team’s problems, but it will sharpen execution. Before putting a story in the development backlog, review it. Does it describe the business need or value? Does it have well-written, complete, verifiable acceptance criteria? Can it be completed in a single sprint or iteration? If the answer to all three of these questions is yes, you’ve probably got a decent story. Go ahead and put it in the queue with confidence!

The Art of Agile

I was out on the internet doing some general research for ideas to improve my team’s agile software practices, and I found a terrific resource. More specifically, I was in search of information about how to improve our story-writing, and I found it here. I then stumbled upon an equally interesting article about including slack in a sprint. This is something that I’ve felt is necessary for a long time, but I didn’t know how to introduce it. There are a lot of good ideas; read more here.

If you’re doing agile and struggle with any aspects or just looking for new ideas, I recommend checking out this website: James Shore: The Art of Agile

Thanks to James Shore for making this wonderful resource available!