Performance Profiling for Unit Tests

When I first got my hands on Visual Studio 2012, I was trying out every feature I could, new and old alike. I love the new Test Explorer. Code coverage is SO much better than it was before. Code analysis for code clones is a terrific, too. The only thing I wasn’t happy about was performance analysis.

The reason I wasn’t happy with performance analysis is that I couldn’t use it on unit tests. Luckily, Microsoft fixed that with the release of Visual Studio 2012 Update 1. Now you can right-click a test and choose Profile Test to run performance analysis on a single unit test, and it is awesome!

When you choose to profile a test, the test runs as usual followed by the analysis. When analysis completes, you’re presented with a summary view that shows you CPU usage over time, the “hot path” call tree—a tree of the most active function calls where most of the work was performed—and a list of functions responsible for doing the most individual work.

You can find out more and investigate resource spikes by selecting a time range on the graph and filtering. That’s all good and well, but what really blew me away was that you can click on the functions with the most individual work to drill into them. Drill into them? Yea—you’re taken to a view that shows you the selected function, its callers, and the methods it calls. There are percentages that show how much time was taken in each of the three areas (callers, current method, and callees), and you can click into any of the methods displayed to navigate up or down the call stack. The actual code for the current method is also displayed. The only thing that seemed sub-optimal is that I couldn’t edit the code directly; there’s a link to the actual file, though, so you’re only a click away from the editable code file.

There are other, sortable views you can look at, too. You can view a call tree or breakdown by module, and you can get to the same function details view described above from each of those views. It’s a really useful, powerful experience.

Here’s where it gets really nuts, though: add SpecFlow to the mix. SpecFlow lets you write feature-based scenarios that are capable of serving as automated integration tests. The scenarios run like normal unit tests. You can right-click them in the Test Explorer to run performance analysis on them. This means that you can do targeted performance analysis on specific features of your application! To test this out, I sorted my unit tests by execution duration and analyzed the slowest. I was able to find a bottleneck with a caching algorithm used by nearly all of the other modules in the application. Execution time of the 350 unit tests in the project went from 50 seconds to 20. That’s a HUGE improvement from fixing one flaw found from running analysis on one function picked only because it was the most time-consuming in the slowest test.

Good tests are supposed to run quickly, and performance analysis is an invaluable tool to help you triage valuable-but-poor-performing tests. Also, since you’ve got automated tests, you can refactor and optimize the performance of your application with high confidence. If you haven’t used performance analysis before—more specifically, performance analysis for unit tests—give it a shot; I’d be blown away if you didn’t find incredible value.

Author: Adam Prescott

I'm enthusiastic and passionate about creating intuitive, great-looking software. I strive to find the simplest solutions to complex problems, and I embrace agile principles and test-driven development. View all posts by Adam Prescott