A Better Table.CreateInstance() for SpecFlow

SpecFlow remains my top choice for automated integration testing. I love writing cute little cukes and putting them together in different ways to create different test scenarios. And one of my favorite cuke tricks is using tables to define an object, which I wrote about some time ago.

The CreateInstance<T> extension method provides an easy way to convert that table data into objects for testing, but I really think there should be a better way to populate child properties of complex objects.

Consider this example:

public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public Address Address { get; set; }
}

public class Address
{
    public string Street { get; set; }
    public string City { get; set; }
    public string State { get; set; }
    public string Zip { get; set; }
}

It would be nice if you could write a cuke like this:

Given a person
	| field          | value         |
	| firstName      | adam          |
	| lastName       | prescott      |
	| address.street | 123 number ln |
	| address.city   | anytown       |
	| address.state  | ny            |
	| address.zip    | 10000         |

And then convert it to a person like this:

[Given(@"a person")]
public void GivenAPerson(Table table)
{
    var person = table.CreateInstance<Person>();
    ScenarioContext.Current.Set(person);
}

But you can’t. Well, you can, but the address property won’t be populated. I didn’t like that, so I decided to grow my own. It works by identifying the child properties in the table and creating sub-tables, then using reflection to find and set the property on the result object. It works recursively, too, so you could even go n properties deep (i.e., address.state.abbreviation).

At its core, it’s just using CreateInstance<T> so you get all the niceties that go along with that. Note also that it only works with the 2-column, vertical tables with field and value columns. I called my extension method BuildInstance since CreateInstance was already taken. Here it is… Enjoy!

using System;
using System.Linq;
using System.Reflection;
using System.Text.RegularExpressions;
using TechTalk.SpecFlow;
using TechTalk.SpecFlow.Assist;

public static class TableExtensions
{
    public static T BuildInstance<T>(this Table table)
    {
        T result = table.CreateInstance<T>();

        // find sub-properties by looking for "."
        var propNames = table.Rows.OfType<TableRow>()
            .Where(x => x[0].Contains("."))
            .Select(x => Regex.Replace(x[0], @"^(.+?)\..+$", "$1"));

        foreach (var propName in propNames)
        {
            // look for matching property in result object
            var prop = typeof(T).GetProperty(
                propName,
                BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase);

            if (prop != null)
            {
                // create sub-table with relevant rows (field == propName.something)
                var subTable = new Table("field", "value");
                var re = new Regex(string.Format(@"^{0}\.([^\.]*)$", propName), RegexOptions.IgnoreCase);
                table.Rows.OfType<TableRow>().Where(x => re.IsMatch(x[0]))
                    .Select(x => new[] { re.Replace(x[0], "$1"), x[1] })
                    .ToList()
                    .ForEach(x => subTable.AddRow(x));

                // make recursive call to create child object
                var createInstance = typeof(TableExtensions)
                    .GetMethod(
                        "BuildInstance",
                        BindingFlags.Public | BindingFlags.Static,
                        null,
                        CallingConventions.Any,
                        new Type[] { typeof(Table) },
                        null);
                createInstance = createInstance.MakeGenericMethod(prop.PropertyType);
                object propValue = createInstance.Invoke(null, new object[] { subTable });

                // assign child object to result
                prop.SetValue(result, propValue);
            }
        }

        return result;
    }
}

Database SpecFlow Scenarios and TransactionScope

My team has been using SpecFlow to verify that our data access components are working correctly. In an effort to write repeatable tests that won’t pollute the database, we’ve decided to wrap each scenario in a transaction.

Creating and rolling back the transactions is very simple to do with SpecFlow’s BeforeScenario & AfterScenario hooks.

using System.Transactions;

[Binding]
public class TransactionHooks
{
    private TransactionScope _transactionScope;

    [BeforeScenario]
    public void BeforeScenario()
    {
        _transactionScope = new TransactionScope();
    }

    [AfterScenario]
    public void AfterScenario()
    {
        _transactionScope.Dispose();
    }
}

By including these hooks, you’re placing your test scenarios in the loving embrace of a transaction scope. The act of instantiating the TransactionScope will update the ambient transaction that will be used by all subsequent code unless you explicitly tell it to do otherwise.

If you’re testing code that uses TransactionScope itself, using TransactionScopeOption.Required will allow the code to use the ambient transaction if one exists. Note that this is the default value, so it’s what you’re using if you’re not explicitly specifying an option. However, the other TransactionScopeOption values will cause code to execute outside your test scenario’s ambient transaction by either creating a new/different transaction (RequiredNew) or executing outside the transaction (Suppress).

Performance Profiling for Unit Tests

When I first got my hands on Visual Studio 2012, I was trying out every feature I could, new and old alike. I love the new Test Explorer. Code coverage is SO much better than it was before. Code analysis for code clones is a terrific, too. The only thing I wasn’t happy about was performance analysis.

The reason I wasn’t happy with performance analysis is that I couldn’t use it on unit tests. Luckily, Microsoft fixed that with the release of Visual Studio 2012 Update 1. Now you can right-click a test and choose Profile Test to run performance analysis on a single unit test, and it is awesome!

ProfileTest_ContextMenu

When you choose to profile a test, the test runs as usual followed by the analysis. When analysis completes, you’re presented with a summary view that shows you CPU usage over time, the “hot path” call tree—a tree of the most active function calls where most of the work was performed—and a list of functions responsible for doing the most individual work.

ProfileTest_SummaryView

You can find out more and investigate resource spikes by selecting a time range on the graph and filtering. That’s all good and well, but what really blew me away was that you can click on the functions with the most individual work to drill into them. Drill into them? Yea—you’re taken to a view that shows you the selected function, its callers, and the methods it calls. There are percentages that show how much time was taken in each of the three areas (callers, current method, and callees), and you can click into any of the methods displayed to navigate up or down the call stack. The actual code for the current method is also displayed. The only thing that seemed sub-optimal is that I couldn’t edit the code directly; there’s a link to the actual file, though, so you’re only a click away from the editable code file.

ProfileTest_FunctionDetails

There are other, sortable views you can look at, too. You can view a call tree or breakdown by module, and you can get to the same function details view described above from each of those views. It’s a really useful, powerful experience.

Here’s where it gets really nuts, though: add SpecFlow to the mix. SpecFlow lets you write feature-based scenarios that are capable of serving as automated integration tests. The scenarios run like normal unit tests. You can right-click them in the Test Explorer to run performance analysis on them. This means that you can do targeted performance analysis on specific features of your application! To test this out, I sorted my unit tests by execution duration and analyzed the slowest. I was able to find a bottleneck with a caching algorithm used by nearly all of the other modules in the application. Execution time of the 350 unit tests in the project went from 50 seconds to 20. That’s a HUGE improvement from fixing one flaw found from running analysis on one function picked only because it was the most time-consuming in the slowest test.

Good tests are supposed to run quickly, and performance analysis is an invaluable tool to help you triage valuable-but-poor-performing tests. Also, since you’ve got automated tests, you can refactor and optimize the performance of your application with high confidence. If you haven’t used performance analysis before—more specifically, performance analysis for unit tests—give it a shot; I’d be blown away if you didn’t find incredible value.

Learning Cukes

Cucumber

I’ve written [several] [posts] [in] [the] [past] about my team’s adoption of SpecFlow and BDD, and I’m still loving it several months later. The large project that we started with has nearly 10,000 lines of code and 93% code coverage. We’ve gone through several large refactors, and each time we walk away with a high level of confidence that no functionality was lost or affected negatively. It’s been a really great experience.

One of the challenges of adoption was just learning the Cucumber step definition syntax, or rather, how to write cukes. For getting started, I recommend taking a look at this page for a good list of descriptions, scenarios, and examples. If you’re using SpecFlow, you may also want to check out their step definition documentation on GitHub.

Once you’ve got the basic syntax down, the hard part begins. My team hasn’t had much formal discussion about Cucumber best practices, and we’re still learning what works and what doesn’t. If you look around online, you can find a few good articles with helpful suggestions, though.

Here’s a great post that I recommend reading. This article offers advice on just about every aspect of creating and managing your cukes, from feature files to tags to running and refactoring.

I also found this post from EggsOnBread to be very helpful. All of the recommended practices are good. This was one of the first articles I read when I was getting started, and it’s served me well. I’ll be honest, though–many of the points didn’t stick during my initial read. It became much more valuable after spending several months working with Cucumber and then re-reading.

Compare Objects from Tables in SpecFlow

Earlier this week, I wrote about how to create and populate objects from tables created in your SpecFlow scenarios. If the only reason you’re populating an object is to do a comparison, there’s an easier way: the CompareToInstance<T> method.

Similar to the previously discussed method, CreateInstance<T>, an object will be created using the properties contained in a table of values. The object properties will be matched ignoring case and whitespace. CompareToInstance allows you to provide an instance of the same type and does a property-by-property comparison. If any of the property values are different, a TechTalk.SpecFlow.Assist.ComparisonException is thrown with a message containing all differences that were found.

Here’s an example!

Scenario: Update a person name
	Given I have an existing person
	And I change the name to the following
		| first | middle | last  |
		| Ftest | Mtest  | Ltest |
	When I save the person
	And I retrieve person
	Then the name was changed to the following
		| first | middle | last  |
		| Ftest | Mtest  | Ltest |
// don't forget!
// using TechTalk.SpecFlow.Assist;

[Given(@"I change the name to the following")]
public void GivenIChangeTheNameToTheFollowing(Table table)
{
    var p = ScenarioContext.Current.Get<Person>();
    p.Name = table.CreateInstance<NameType>();
}

[Then(@"the name was changed to the following")]
public void ThenTheNameWasChangedToTheFollowing(Table table)
{
    var p = ScenarioContext.Current.Get<Person>();
    table.CompareToInstance<NameType>(p.Name);
}

Calling Steps From Steps in SpecFlow

Steps are the building blocks of SpecFlow. Each Given/When/Then line in a SpecFlow scenario represents a step, and steps should be reused across features and scenarios to test your application from different angles. When you’re building a low-level scenario, you may want to use very specific steps. In a higher-level feature, you may want to perform the same tasks but in a less granular fashion. Wouldn’t it be nice if you could create a “super-step” that just calls the necessary sub-steps?

Well, guess what? You can, and it’s really easy to do. First, let’s build some fake code to work with. I created a simple PersonRepository that lets me add and save Person objects. Here are the classes and my initial SpecFlow test.

Person.cs

using System;

namespace samples.SpecFlowDemo
{
    public class Person
    {
        public int Id { get; set; }

        public string Name { get; set; }

        public DateTime? DateOfBirth { get; set; }
    }
}

PersonRepository.cs

using System;
using System.Collections.Generic;

namespace samples.SpecFlowDemo
{
    public static class PersonRepository
    {
        private static readonly Dictionary<int, Person> Persons;

        static PersonRepository()
        {
            Persons = new Dictionary<int, Person>();
        }

        public static Person Get(int id)
        {
            if (!Persons.ContainsKey(id))
            {
                return null;
            }
            return Persons[id];
        }

        public static bool Save(Person person)
        {
            if (person == null)
            {
                return false;
            }
            if (!Persons.ContainsKey(person.Id))
            {
                Persons.Add(person.Id, person);
            }
            Persons[person.Id] = person;
            return true;
        }
    }
}

Person_Add.feature

Feature: Person_Add
	In order track person records
	As a records manager
	I need to add new persons

Scenario: Add a person
	Given I have a new person record with the following properties
		| id  | name   | date of birth |
		| 100 | Rodney | 2/20/1950     |
	When I save the person
	Then the person is saved successfully
	And I can retrieve the person by ID

Person_AddSteps.cs

using Microsoft.VisualStudio.TestTools.UnitTesting;
using TechTalk.SpecFlow;
using TechTalk.SpecFlow.Assist;

namespace samples.SpecFlowDemo.SpecFlow
{
    [Binding]
    public class Person_AddSteps
    {
        [Given(@"I have a new person record with the following properties")]
        public void GivenIHaveANewPersonRecordWithTheFollowingProperties(Table table)
        {
            ScenarioContext.Current.Set<Person>(
                table.CreateInstance<Person>());
        }
        
        [When(@"I save the person")]
        public void WhenISaveThePerson()
        {
            var r = PersonRepository.Save(
                ScenarioContext.Current.Get<Person>());
            ScenarioContext.Current.Set<bool>(r, "SaveResult");
        }
        
        [Then(@"the person is saved successfully")]
        public void ThenThePersonIsSavedSuccessfully()
        {
            Assert.IsTrue(
                ScenarioContext.Current.Get<bool>("SaveResult"),
                "SaveResult");
        }
        
        [Then(@"I can retrieve the person by ID")]
        public void ThenICanRetrieveThePersonByID()
        {
            var expected = ScenarioContext.Current.Get<Person>();
            var actual = PersonRepository.Get(expected.Id);
            Assert.AreSame(expected, actual);
        }
    }
}

Now let’s say I want to test the PersonRepository’s ability to update records. In order to update a record, the record needs to exist. I could reuse the add feature’s “I have a person” step with a provided table of properties and its “I save the person” step, but it would be nice if I didn’t have to call both of those steps and provide data each time I needed to do something with an existing record.

I can avoid the repetition by calling those steps from within a new step that I’ll create called “I have an existing person record.” Here’s the code for my update feature.

Person_Update.feature

Feature: Person_Update
	In order track person records
	As a records manager
	I need to update existing persons

Scenario: Update a person
	Given I have an existing person record
	When I change the person name to "Rocko"
	And I save the person
	Then the person is saved successfully
	And I can retrieve the person by ID
	And the person name was saved as "Rocko"

Person_UpdateSteps.cs (Note the highlighted lines. The only “gotcha” that I ran into is that the steps class must inherit from SpecFlow’s Steps class in order to access the Given/When/Then functions.)

using Microsoft.VisualStudio.TestTools.UnitTesting;
using TechTalk.SpecFlow;

namespace samples.SpecFlowDemo.SpecFlow
{
    [Binding]
    public class Person_UpdateSteps : Steps
    {
        [Given(@"I have an existing person record")]
        public void GivenIHaveAnExistingPersonRecord()
        {
            var header = new[] { "Field", "Value" };
            var t = new Table(header);
            t.AddRow("id", "100");
            t.AddRow("name", "Fred");
            t.AddRow("date of birth", "12/15/1990");

            Given("I have a new person record with the following properties", t);
            When("I save the person");
        }
        
        [When(@"I change the person name to ""(.*)""")]
        public void WhenIChangeThePersonNameTo(string p0)
        {
            var p = ScenarioContext.Current.Get<Person>();
            p.Name = p0;
        }
                
        [Then(@"the person name was saved as ""(.*)""")]
        public void ThenThePersonNameWasSavedAs(string p0)
        {
            var p = ScenarioContext.Current.Get<Person>();
            Assert.AreEqual(p0, p.Name);
        }
    }
}

You can read more about calling steps from step definitions in the SpecFlow documentation.

You can also download the code for this example from GitHub.

Create and Populate Objects in SpecFlow Scenarios

SpecFlow is definitely my favorite new tool for 2012. If you haven’t checked it out yet, you should. If you’ve checked it out and still don’t get it, come talk to me. I love writing scenarios using business language to help guide my test-driven development. The integration-level tests are proving to be an awesome supplement to more traditional isolated unit tests.

Sometimes it’s hard to come up with a good scenario step because you need to specify several details. Don’t worry, though–SpecFlow’s got ya covered with its Assist Helpers. Let’s say I’m working on some code that has a configuration object that contains my application’s settings. When I’m writing a feature scenario, I might want to provide these settings. I can do just that by putting the details into a table in my scenario.

Configuration.cs

public class Configuration
{
    public string OutputDir { get; set; }
    public bool Enabled { get; set; }
}

Feature1.feature

Scenario: CreateInstance demo
    Given I have the following configuration
        | output dir | enabled |
        | c:\blah    | true    |
    When I do something
    Then what I expected to happen happened

In my step definition, I can import the TechTalk.SpecFlow.Assist namespace to use Assist Helpers like the CreateInstance&lt;T&gt; extension method. When called on a table, CreateInstance will match columns to properties to create and populate an object. Here’s how I use it with the above example:

[Given(@"I have the following configuration")]
public void GivenIHaveTheFollowingConfiguration(Table table)
{
    var config = table.CreateInstance();
}

When creating the object, case-sensitivity and whitespace are ignored, allowing you to create a human-friendly table of your object’s property values. You can also use a vertical table with properties listed as rows. This table will produce the same result as above:

Scenario: CreateInstance demo
    Given I have the following configuration
        | Field      | Value   |
        | output dir | c:\blah |
        | enabled    | true    |
    When I do something
    Then what I expected to happen happened

This technique allows you to create complex objects without having to write any code. You can create highly re-usable steps that rely on the scenario author to provide necessary details in the scenario itself. Simply awesome!