I have a colleague that’s my equal in terms of unit testing enthusiasm, but we have very different philosophies. He tends to write methods first, then test the hell out of them to ensure that all code paths have been covered and that there are no holes. I tend to code using more of a TDD workflow, writing tests for each behavior that I expect from a method and not worrying about anything else that may or may not being going on.
Both approaches are valid. As we code, we both think about things that could go wrong with our code, and we both account for those things and make sure they’re tested. At the end of the day, we both end up with relatively bug free solutions that work well. Both methods produce high levels of code coverage. although focusing test writing on code paths will likely result is slightly higher coverage since the tests.
Yes, there’s a lot that’s similar about these two different approaches, but the differences are very important. The TDD mantra is “red, green, refactor.” The idea is that you write a failing test, add code to make the test pass, and then refactor the solution to clean up and optimize. This workflow is made for behavior-based testing. You expect a certain result from the method being tested. Once it’s producing that result, it shouldn’t stop producing it due to refactoring or optimizations.
The same statement can be made for tests written based on code paths: an expected result should continue to be produced after code is optimized. I’m venturing to say that optimizations are less likely to occur with the code-first approach, though. When you write code first, you don’t write tests until you’re done. And, since you’re writing tests based on the “finished” code, it’s less likely that you’ll discover flaws. Refactoring also seems less likely for the same reason. If refactoring does occur–which it should–then there’s a different problem: code paths that were once different may now be the same. You may have unknowingly written duplicate tests! (That’s not to say that the duplicate or redundant tests are bad, but you’ll have spent time writing code that, in the end, didn’t need to be written.)
Every developer I’ve ever met has learned to code before they’ve learned to write unit tests. Unit tests are generally written in code, so it’s hard to imagine learning them in any other order. Because we learn these two things in that order, we generally learn to write unit tests by following code paths. If you’re one of those code-first-and-write-tests-later types, I urge you to step out of your comfort zone and start writing behavior-based tests FIRST. You’ll code with purpose and write meaningful tests. You’ll be able to refactor with confidence, knowing that your code’s behavior has been unaffected by your chances. Like any skill, it takes some time to get used to, but I strongly believe you’ll produce higher quality code more efficiently once you become proficient.
I’ve been preaching hard about test-driven development and the importance of writing tests first. I can feel the culture beginning to shift as people are slowly starting to buy in, but I had an interesting discovery yesterday.
I was invited to a meeting by some developers that wanted me to walk through how I would’ve used test-driven development to write tests and develop a new project that they had recently started. It was essentially a data validation project that retrieved data from the database, checked it against a set of rules, and recorded any violations. We were reviewing a Controller class that was responsible for orchestrating the operation.
“Okay,” I said, “What is this thing supposed to do?”
The developers told me it retrieves records, validates each record, and saves the validation results. So, without knowing anything more than that, I figured there were at least two external dependencies: an IDataAccess responsible for retrieving records and saving the result and an IValidator that does the data validation/rule-checking. I drew a diagram on the whiteboard to show the relationships between the Controller and these two components.
I explained that since we know the dependencies and how we expect them to be used, we can begin to write tests. We also need to know how our application should react when the dependencies are missing. I started to rattle off some tests:
The group was with me, but they quickly shifted focus to what tests were needed for the DataAccess class. And the tests for its dependencies. And everything else.
“Whoa, whoa, WHOA. None of that matters for this. All we care about is this method,” I say.
“Well, yes, but we want to do test-driven development. We thought the goal was to have all of our tests written first so we can go back and implement them.”
That’s when I had my epiphany. When I’m telling people to write tests first, they think I mean write ALL tests first. This is not the case! It would be virtually impossible to think about every code decision and execution path for an entire method/class/application upfront, and I think that’s where there’s been a disconnect. I can look at the finished code and come up with all the tests, but there is no way I could’ve come up with every single test for every single method before ever writing any code.
I went to another small team of developers and asked them if they also thought I meant “all tests first.” They did. It’s disappointing to know that I was sending the wrong message, but I’m glad I have something to address that will hopefully result in more passengers on the TDD train.
When you’re getting started with test-driven development, don’t try to write every single test first. Don’t even try to write as many tests as you can think of. You just want to write tests as you go. What does this method need to do next? Write a failing test, then write the implementation to make it pass. Red, green, refactor, baby! I’m also exchanging my “tests first” mantra for a new one: “test as you go!”
I’m constantly encouraging developers around me to embrace test-driven development and automated unit testing. However, there’s one very important thing I’ve neglected to include in my evangelizing: a definition for what constitutes a good unit test.
A big problem that I think a lot of developers run into is that they’re writing tests but aren’t realizing any value from them. Development takes longer because they have to write tests, and when requirements change it takes longer because they have to refactor tests. Maintenance and warranty work is slower, too, because of the additional upkeep from failing tests created with every little change.
These problems mostly exist because of bad unit tests that are written due to insufficient knowledge of what makes a good test. Here’s a list of properties for a good unit test, taken from The Art of Unit Testing:
Able to be fully automated
Has full control over all the pieces running (Use mocks or stubs to achieve this isolation when needed)
Can be run in any order if part of many other tests
Runs in memory (no DB or File access, for example)
Consistently returns the same result (You always run the same test, so no random numbers, for example. save those for integration or range tests)
Runs fast
Tests a single logical concept in the system
Readable
Maintainable
Trustworthy (when you see its result, you don’t need to debug the code just to be sure)
This is a great list that you can use to gut-check your unit tests. If a test meets all of these criteria, you’re probably in good shape. If you’re violating some of these, refactoring is probably required in your test or in your design, particularly in the cases of Has full control over all the pieces running, Runs in memory, and Tests a single logical concept in the system.
When I see developers struggling with unit tests and test-driven development, it’s usually due to test “backfilling” on a poorly-designed or too-complex method. They don’t see value because, in their minds, the work is already done and they’re having to do additional work just to get it unit tested. These methods and objects are violating the single responsibility principle and sometimes have many different code paths and dependencies. That complexity makes writing tests hard because you have to do so much work upfront in terms of mocking and state preparation, and it’s really difficult to cover each and every code path. It also makes for fragile tests because testing specific parts of a method rely on a test’s ability so successfully navigate its way through the logic in the complex method; if an earlier part of the method changes, you now have to refactor unrelated tests to re-route them back to the code they’re testing. (Tip: Avoid problems like this by writing tests first!)
Whether you’re just getting with unit tests or a grizzled veteran, you can benefit from using this list of criteria as a quality measuring stick for the tests you write. High-quality tests that verify implemented features will result in a stable application. Designs will become better and more maintainable because you’ll be able to modify specific functionality without affecting the surround system the same way that you’re able to test it. You won’t have to worry about other developers breaking functionality you’ve added, and you won’t have to worry about breaking functionality they’ve added. You’ll be able to make a modification that affects the entire system with a high-level of confidence. I’d venture to say that virtually every aspect of software development is better when you’ve got good tests.
I think it’s safe to say that everybody agrees on the value and necessity of automated unit tests. Writing code without tests is the fastest way to accumulate technical debt. This is one reason why test-driven development (TDD) has become an accepted and preferred technique for developing high-quality, maintainable software. When asked to describe TDD, a typical response might include the red, green, refactor pattern. The idea is that you write a failing test first (red), make it pass (green), and then clean up after yourself (refactor). Easy, right?
Not so fast, my friend.
It breaks my heart to say it, but I think most of the developers on my team don’t write tests first. They follow more of a null, refactor, green pattern. Code is written first. It’s assumed to be pretty good but not verified. Then refactoring occurs to make the code testable as unit tests are written to test the code. I have a few problems with this.
First and foremost, by doing tests last, you’re making it possible to be “done” without proper tests. You should never have somebody say something like, “It’s done, but I didn’t have time to write all the tests.” I’ve seen this happen too many times, and it’s completely unacceptable. You aren’t done until you have tests, and the best way to ensure you have tests is to write them first. Period.
By not writing tests first, you’re also opening yourself to the possibility of writing untestable code. Regardless of whether or not you write tests before or after, you’ll have to figure out how to test the code you are or will be writing. If you write the tests after code, you might find yourself doing some heavy refactoring and restructuring to make the code testable–a step that could have been avoided completely by simply writing the tests first. Conversely, writing tests first empowers you to make smart design decisions that ensure testability before you’ve implemented your solution. Writing tests first forces you to write testable code, and testable code is typically more well-designed.
Tests-first help from a requirements perspective, too. You’re obviously trying to accomplish something by writing code, so what is it? Write failing or inconclusive tests to represent your requirements, and you’ll be less likely to accidentally miss a requirement in the heat of your fervorous coding. If you don’t know the requirements, you probably shouldn’t be coding anything. That should be a huge red flag. Stop coding immediately, and go figure out the requirements!
I don’t think I’ve presented anything terribly controversial here, so why aren’t people writing tests first? In my opinion, it’s primarily a mental block. You have to commit to it. Executing can be tricky–especially in the beginning–but making the commitment to write tests first and sticking to it is really the hard part. Execution will come with time and practice. But you still need to start somewhere, right? So, I’ve whipped up a simple workflow to help get you started. Note that a lot of this is situational, but the basic idea is to start with stubs and do only enough of the actual implementation to complete your test.
Write a test method with a descriptive name.
If I’m going to be writing a Save method, my first test might be named SaveSavesTheRecordToTheDatabase or just SaveSaves.
Implement the test with a single line of code: Assert.Fail() or Assert.Inconclusive().
Note that the Save method might not exist at this point! I’m just spelling out what I want to accomplish by writing test methods.
Repeat for each requirement.
Begin implementing the test.
This will be different depending on the nature of what’s being tested. It might be defining an expected output for a given input. For my example Save method that will save the record to a database, the first thing I might do is create a mock IDatabase and write an assertion to verify that it was used to save the record.
Go as far as you can without implementing your actual change.
Leave your Assert.Fail() or Assert.Inconclusive() at the end of the test if there’s more work to be done.
Begin implementing your changes.
The goal should be to make the test code that you’ve written so far pass or to free a roadblock that prevents you from writing more tests. How will my Save method get access to an IDatabase? Will it be a method argument? Passed to the constructor? I can address these implementation details without implementing the Save method.
Identify and write more tests as you go! For example, did you add a new method argument that will never be null? Add a test to verify that an ArgumentNullException is thrown if it is. Are you making an external service call? Write a test to verify that exceptions will be handled. Write the new tests before you implement the behavior that addresses them!
Go as far as you need to complete your test code.
Use stub methods that throw NotImplementedException if you need a new method that doesn’t have tests.
Finish implementing the test.
Add mocks, add assertions, and implement the rest of your test.
Finish implementing your changes.
Now, with a complete test, finish the rest of your implementation.
Review tests and changes.
Add additional test cases to verify alternate scenarios.
Look for edge cases and ensure they’re covered. (Null checks, exception handling, etc.)
Refactor!
Don’t forget to cleanup after yourself!
Now that you have a solid set of tests, you can refactor your code to make it shiny and clean. The tests will ensure that you don’t lose functionality or break requirements.
This isn’t a perfect workflow, and it won’t work for all scenarios. It’s just meant to help get you over the I-can’t-write-a-test-for-this-yet hump. Remember that writing tests first is a decision you make, above all else. Commit to write tests first. Fight the urge to implement first and test later, and stop coding if you find yourself implementing sans tests. After some time, it will become habit. You’ll learn how to deal with the tricky stuff.
Do you have similar experiences with getting teams to adopt TDD? I’d love to hear about them! Please share your thoughts and ideas related to the topic. What are some things that you’ve done or had done to you to improve adoption of TDD and actually writing tests first?
I was poking around the Microsoft Patterns & Practices site on MSDN, and I found a terrific comic that illustrates traditional versus agile development.
Just today, I was doing project planning with co-worker where we were discussing “just doing all the data access at once” versus “building basic (but testable!) end-to-end functionality and enhancing as we progress.” This comic happens to do a great job of comparing these two approaches and highlighting the risks associating with building individual components separately without an appropriate test framework. If only Dr. Frankenstein had been more agile…
Unit tests are one of my favorite things to write as a developer. I simply love them. It’s like a little game I get to play with myself to ensure I’ve covered all possible scenarios and verify that my code is absolutely bulletproof. If I’m doing maintenance work on a project without tests, you better believe it won’t be without tests by the time I’m finished. And when I review another developer’s code and find issues that I know would’ve been caught by even the simplest of unit tests, I feel sad.
There’s been a lot of buzz about unit tests for several years now, but it’s still a challenge getting everybody to buy in. Where’s the real benefit with these tests? Isn’t it going to take me a lot longer if I have to write tests for everything? Aren’t I just making more work for myself when I make the changes that I’ll inevitably be making? No, no, NO!
The benefits of unit tests and test-driven development are so great and so many that whether or not to do it shouldn’t even be a discussion. That said, I realize there’s a path to enlightenment that takes time to travel, and I want to help you get there. So let’s explore some of these endless benefits that I’m speak of…
SOLID Design
Code that is difficult to test is often a result of poor design. If you follow the SOLID design principles, you will find that your code generally lends itself well to unit tests. Writing tests first can help identify a bad design before it’s too late. SOLID principles make testing easier. So, by embracing unit tests and test-driven development, your coding practices will naturally evolve to follow these principles.
Single responsibility principle – Testing and verifying an object with a single, clear, defined purpose will be easier than testing a class with multiple, vague responsibilities. Know your class’s purpose, and you’ll have a better idea what tests to write. It’s important to apply this principle to methods, as well. A method that does one thing is easy to test. A method that does 100 things, not so much.
Open/closed principle – Making classes extensible will make them flexible in unit test scenarios. One of my favorite techniques is using partial mocks with RhinoMocks, which rely heavily on this principle.
Liskov substitution principle – Embracing this principle will help you keep base classes and interfaces simple. This will make consumers of your objects easier to test because less time can be spent repeating unnecessary mocking. Don’t make an IMyDataProvider that has implementation-specific public methods (e.g., CreateDatabaseConnection(), ExecuteStoredProcedure(), etc.). Instead, create a method that does what’s needed, like GetData(). The result is that a MockMyDataProvider can be provided, and you need only mock the GetData method every time instead of each implementation-specific method. Still write tests for those implementation-specific methods, but write them in the tests for your data provider implementation!
Interface segregation principle – Interfaces are, in my opinion, the single most important aspect of unit testing layer interactions. Mock objects built from interfaces are absolutely essential for testing most non-trivial scenarios.
Dependency inversion principle – Creating mocks, stubs, and fakes is meaningless without a way to inject them into the object being tested. Use constructor injection, property injection, or a dependency injection framework like Unity to provide your object with mocks to remove outside dependencies and test specific functionality.
Fewer Surprises
There is nothing more frustrating than spending hours upon hours troubleshooting an issue only to arrive at a piece of code that has clearly never been run. Write tests, and check your code coverage. If there are lines of code that haven’t been covered, you can evaluate them to determine whether or not additional tests should be written. Play the what-else-could-possibly-go-wrong-here game with yourself. Enforce your own assumptions with exceptions, and test those. If you expect an argument to be not-null, write a test that expects an ArgumentNullException to be thrown; that will be significantly easier to troubleshoot than a null reference exception.
Improved Maintainability
If you’ve spent any significant time maintaining the work of others, you know this is a big one. It’s unsettling to find yourself in the middle of someone else’s 1000-line function with a block of commented code that looks like it does exactly what’s been reported to not be happening. Clearly, removing it was an intentional move, but what will you be breaking by uncommenting it? If a test is written (MammothMethod_ShouldNotCallDatabaseDirectly()), and you uncomment the code that calls the database directly causing that test to fail, you’ll have received two yellow flags warning you that maybe this is a bad approach. Without the test, a developer shrugs, uncomments the code, feels good about it, and re-introduces a bug that was likely fixed previously by another developer. (Didn’t we fix that two releases ago?) Writing tests for your fixes ensures that you aren’t on the other end of that scenario where somebody undoes your fixes. (If they still undo your fix and break or refactor your test in the process, be sure to yell at them.)
Unit tests are also helpful in terms of code management when complex branching strategies are employed. Let’s say you have 20 developers working in different branches. When the branches are brought back together, how can you confirm that changes were not lost? With unit tests in place, you simply need to run the tests. Since that should be happening automatically with each build anyway, all you need to do is verify that the test are there. This really beats the alternative of reviewing code to see if a particular change is included. Plus you feel good as a code-mover knowing that the code is verifying itself with each move.
Well-Tested, High Quality Solutions
At the end of the day, what you’re really trying to produce is a quality solution. You want your deliverables to work correctly when your assumptions are met and handle unexpected scenarios gracefully. You want to tell your users how your application will handle any situation they can come up with. You want your users to feel great about you, whether it’s your company, your product, or you specifically. Unit tests can help you do all this. What happens if the web service isn’t available? There’s a unit test to verify an email is sent. How can I audit queries to the database? There’s a unit test to verify logging. What if an expected record is missing? There’s a test for that, too. If you can think of it, make sure you’ve tested it. Write the tests as you go, and you’ll be sure not to affect features implemented earlier. At the end of development, your code will have a collection of tests self-verifying every feature and enforcing all assumptions.
Did it take more code? Yes.
Did it take more time? Arguably.
Will the final product have higher quality, require less maintenance, and result in a net decrease in total development effort over time? Definitely.
adamprescott.net 