Unit Test Debug Wonkiness in Visual Studio 2012

Our team has started rolling out Visual Studio 2012, and it’s been largely issue-free. In fact, the only issue that we’ve run into is a weird one: the debugger gets screwed up while debugging unit tests. What I mean by “messed up” is that you step over a line, and it jumps down several lines, variables don’t get set; you know, wonkiness!

Luckily, we were able to find this MSDN forum post, which describes our problem and how to solve it.

The problem seems to be related to the solution test settings file. The solutions presented in the forum are to remove the old test settings file or to remove the AgentRule section from the file. We tried these, and it worked for us, too.

Looking a little deeper, we found that the issue occurred when debugging unit tests–running them worked fine–in VS2012 for a solution created in VS2010 with code coverage enabled. Even with all that, the issue did not occur regularly. It worked fine on my machine, but not on my colleague’s. We removed the old test settings file but re-enabled code coverage 2010-style from within 2012 to maintain backward compatibility.

From the same forum post, there is a link to the issue on Microsoft Connect, but the issue has been “deferred” by Microsoft’s development team.

Xsd2Code in Visual Studio 2012

At the time of writing this, Xsd2Code does not install nicely in VS2012. It’s easy enough to “copy” your install from VS2010, though. Here’s a summarized version of adrianot75’s solution.

  1. Copy Xsd2Code.Addin.Addin from the VS2010 Addins directory to the VS2012 Addins
    • From: %UserProfile%\My Documents\Visual Studio 2010\Addins
    • To: %UserProfile%\My Documents\Visual Studio 2012\Addins
  2. Edit the Addin file to include an extra element for version 11.0
    <HostApplication>
      <Name>Microsoft Visual Studio</Name>
      <Version>11.0</Version>
    </HostApplication>
    

Compare Objects from Tables in SpecFlow

Earlier this week, I wrote about how to create and populate objects from tables created in your SpecFlow scenarios. If the only reason you’re populating an object is to do a comparison, there’s an easier way: the CompareToInstance<T> method.

Similar to the previously discussed method, CreateInstance<T>, an object will be created using the properties contained in a table of values. The object properties will be matched ignoring case and whitespace. CompareToInstance allows you to provide an instance of the same type and does a property-by-property comparison. If any of the property values are different, a TechTalk.SpecFlow.Assist.ComparisonException is thrown with a message containing all differences that were found.

Here’s an example!

Scenario: Update a person name
	Given I have an existing person
	And I change the name to the following
		| first | middle | last  |
		| Ftest | Mtest  | Ltest |
	When I save the person
	And I retrieve person
	Then the name was changed to the following
		| first | middle | last  |
		| Ftest | Mtest  | Ltest |
// don't forget!
// using TechTalk.SpecFlow.Assist;

[Given(@"I change the name to the following")]
public void GivenIChangeTheNameToTheFollowing(Table table)
{
    var p = ScenarioContext.Current.Get<Person>();
    p.Name = table.CreateInstance<NameType>();
}

[Then(@"the name was changed to the following")]
public void ThenTheNameWasChangedToTheFollowing(Table table)
{
    var p = ScenarioContext.Current.Get<Person>();
    table.CompareToInstance<NameType>(p.Name);
}

Calling Steps From Steps in SpecFlow

Steps are the building blocks of SpecFlow. Each Given/When/Then line in a SpecFlow scenario represents a step, and steps should be reused across features and scenarios to test your application from different angles. When you’re building a low-level scenario, you may want to use very specific steps. In a higher-level feature, you may want to perform the same tasks but in a less granular fashion. Wouldn’t it be nice if you could create a “super-step” that just calls the necessary sub-steps?

Well, guess what? You can, and it’s really easy to do. First, let’s build some fake code to work with. I created a simple PersonRepository that lets me add and save Person objects. Here are the classes and my initial SpecFlow test.

Person.cs

using System;

namespace samples.SpecFlowDemo
{
    public class Person
    {
        public int Id { get; set; }

        public string Name { get; set; }

        public DateTime? DateOfBirth { get; set; }
    }
}

PersonRepository.cs

using System;
using System.Collections.Generic;

namespace samples.SpecFlowDemo
{
    public static class PersonRepository
    {
        private static readonly Dictionary<int, Person> Persons;

        static PersonRepository()
        {
            Persons = new Dictionary<int, Person>();
        }

        public static Person Get(int id)
        {
            if (!Persons.ContainsKey(id))
            {
                return null;
            }
            return Persons[id];
        }

        public static bool Save(Person person)
        {
            if (person == null)
            {
                return false;
            }
            if (!Persons.ContainsKey(person.Id))
            {
                Persons.Add(person.Id, person);
            }
            Persons[person.Id] = person;
            return true;
        }
    }
}

Person_Add.feature

Feature: Person_Add
	In order track person records
	As a records manager
	I need to add new persons

Scenario: Add a person
	Given I have a new person record with the following properties
		| id  | name   | date of birth |
		| 100 | Rodney | 2/20/1950     |
	When I save the person
	Then the person is saved successfully
	And I can retrieve the person by ID

Person_AddSteps.cs

using Microsoft.VisualStudio.TestTools.UnitTesting;
using TechTalk.SpecFlow;
using TechTalk.SpecFlow.Assist;

namespace samples.SpecFlowDemo.SpecFlow
{
    [Binding]
    public class Person_AddSteps
    {
        [Given(@"I have a new person record with the following properties")]
        public void GivenIHaveANewPersonRecordWithTheFollowingProperties(Table table)
        {
            ScenarioContext.Current.Set<Person>(
                table.CreateInstance<Person>());
        }
        
        [When(@"I save the person")]
        public void WhenISaveThePerson()
        {
            var r = PersonRepository.Save(
                ScenarioContext.Current.Get<Person>());
            ScenarioContext.Current.Set<bool>(r, "SaveResult");
        }
        
        [Then(@"the person is saved successfully")]
        public void ThenThePersonIsSavedSuccessfully()
        {
            Assert.IsTrue(
                ScenarioContext.Current.Get<bool>("SaveResult"),
                "SaveResult");
        }
        
        [Then(@"I can retrieve the person by ID")]
        public void ThenICanRetrieveThePersonByID()
        {
            var expected = ScenarioContext.Current.Get<Person>();
            var actual = PersonRepository.Get(expected.Id);
            Assert.AreSame(expected, actual);
        }
    }
}

Now let’s say I want to test the PersonRepository’s ability to update records. In order to update a record, the record needs to exist. I could reuse the add feature’s “I have a person” step with a provided table of properties and its “I save the person” step, but it would be nice if I didn’t have to call both of those steps and provide data each time I needed to do something with an existing record.

I can avoid the repetition by calling those steps from within a new step that I’ll create called “I have an existing person record.” Here’s the code for my update feature.

Person_Update.feature

Feature: Person_Update
	In order track person records
	As a records manager
	I need to update existing persons

Scenario: Update a person
	Given I have an existing person record
	When I change the person name to "Rocko"
	And I save the person
	Then the person is saved successfully
	And I can retrieve the person by ID
	And the person name was saved as "Rocko"

Person_UpdateSteps.cs (Note the highlighted lines. The only “gotcha” that I ran into is that the steps class must inherit from SpecFlow’s Steps class in order to access the Given/When/Then functions.)

using Microsoft.VisualStudio.TestTools.UnitTesting;
using TechTalk.SpecFlow;

namespace samples.SpecFlowDemo.SpecFlow
{
    [Binding]
    public class Person_UpdateSteps : Steps
    {
        [Given(@"I have an existing person record")]
        public void GivenIHaveAnExistingPersonRecord()
        {
            var header = new[] { "Field", "Value" };
            var t = new Table(header);
            t.AddRow("id", "100");
            t.AddRow("name", "Fred");
            t.AddRow("date of birth", "12/15/1990");

            Given("I have a new person record with the following properties", t);
            When("I save the person");
        }
        
        [When(@"I change the person name to ""(.*)""")]
        public void WhenIChangeThePersonNameTo(string p0)
        {
            var p = ScenarioContext.Current.Get<Person>();
            p.Name = p0;
        }
                
        [Then(@"the person name was saved as ""(.*)""")]
        public void ThenThePersonNameWasSavedAs(string p0)
        {
            var p = ScenarioContext.Current.Get<Person>();
            Assert.AreEqual(p0, p.Name);
        }
    }
}

You can read more about calling steps from step definitions in the SpecFlow documentation.

You can also download the code for this example from GitHub.

Abstract Classes and Generic Methods

I ran into a fun scenario where I passing an object that derived from an abstract base class into a generic static method where it would be passed on to another generic method for some special, type-specific processing. Everything worked great when I passed in an object declared as one of the derived types. However, when the object was declared as the abstract base type, it all fell apart because the generic method wanted to treat the object as the base type and not the actual, derived type!

Problem

Consider this re-invented version of the scenario. I have a static PatientRouter class that accepts injured Creatures. If the Creature is a Human, it will be routed to a HumanHospital. If it’s an Animal, it will be routed to an AnimalHospital. Note, however, that the code fails if a Creature object is received, even if the Creature is actually a Human or Animal. We need to do something so this Creature can be correctly cared for!

public static class PatientRouter
{
    public static void Route<T>(T creature) 
        where T : Creature
    {
        SendToHospital(creature);
    }

    public static void SendToHospital<T>(T creature)
        where T : Creature
    {
        if (typeof(T) == typeof(Creature))
        {
            throw new Exception("Unacceptable!");
        }
        if (typeof(T) == typeof(Human))
        {
            var h = new HumanHospital();
            h.CareFor(creature as Human);
        }
        if (typeof(T) == typeof(Animal))
        {
            var h = new AnimalHospital();
            h.CareFor(creature as Animal);
        }
    }
}

Solution

There are two options that I found for dealing with this scenario. The first is to use the dynamic keyword introduced in .NET Framework 4.0. Here’s what that might look like:

public static void Route<T>(T creature)
    where T : Creature
{
    dynamic d = creature;
    SendToHospital(d);
}

Unfortunately, .NET 4.0 wasn’t an option for me, though. I was, however, able to come up with an acceptable solution using reflection. I don’t love using reflection to execute a method like this, but it gets the job done–so I’m content to use it in a scenario like this until .NET 4.0 becomes available.

public static void Route<T>(T creature) 
    where T : Creature
{
    // using reflection
    typeof(PatientRouter)
        .GetMethod("SendToHospital")
        .MakeGenericMethod(creature.GetType())
        .Invoke(null, new[] { creature });
}

For reference, here’s the complete example with both solutions:

namespace samples.BaseClassGenericMethod
{
    using System;

    public abstract class Creature
    {
        public bool IsInjured { get; set; }
    }

    public class Human : Creature
    {
        /* human stuff */
    }

    public class Animal : Creature
    {
        /* animal stuff */
    }

    public interface IHospital<T> where T : Creature
    {
        void CareFor(T patient);
    }

    public class HumanHospital : IHospital<Human>
    {
        public void CareFor(Human patient)
        {
            Console.WriteLine("Caring for human!");
        }
    }

    public class AnimalHospital : IHospital<Animal>
    {
        public void CareFor(Animal patient)
        {
            Console.WriteLine("Caring for animal!");
        }
    }

    public static class PatientRouter
    {
        public static void Route<T>(T creature) 
            where T : Creature
        {
            // base case
            SendToHospital(creature);

            // using dynamic (.NET 4.0)
            //dynamic d = creature;
            //SendToHospital(d);

            // using reflection
            //var h = typeof(PatientRouter)
            //    .GetMethod("SendToHospital")
            //    .MakeGenericMethod(creature.GetType())
            //    .Invoke(null, new[] { creature });
        }

        public static void SendToHospital<T>(T creature)
            where T : Creature
        {
            if (typeof(T) == typeof(Creature))
            {
                throw new Exception("Unacceptable!");
            }
            if (typeof(T) == typeof(Human))
            {
                var h = new HumanHospital();
                h.CareFor(creature as Human);
            }
            if (typeof(T) == typeof(Animal))
            {
                var h = new AnimalHospital();
                h.CareFor(creature as Animal);
            }
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            var h = new Human();
            PatientRouter.Route(h);

            var a = new Animal();
            PatientRouter.Route(a);

            Creature c = new Human();
            PatientRouter.Route(c);

            Console.ReadLine();
        }
    }
}

Create and Populate Objects in SpecFlow Scenarios

SpecFlow is definitely my favorite new tool for 2012. If you haven’t checked it out yet, you should. If you’ve checked it out and still don’t get it, come talk to me. I love writing scenarios using business language to help guide my test-driven development. The integration-level tests are proving to be an awesome supplement to more traditional isolated unit tests.

Sometimes it’s hard to come up with a good scenario step because you need to specify several details. Don’t worry, though–SpecFlow’s got ya covered with its Assist Helpers. Let’s say I’m working on some code that has a configuration object that contains my application’s settings. When I’m writing a feature scenario, I might want to provide these settings. I can do just that by putting the details into a table in my scenario.

Configuration.cs

public class Configuration
{
    public string OutputDir { get; set; }
    public bool Enabled { get; set; }
}

Feature1.feature

Scenario: CreateInstance demo
    Given I have the following configuration
        | output dir | enabled |
        | c:\blah    | true    |
    When I do something
    Then what I expected to happen happened

In my step definition, I can import the TechTalk.SpecFlow.Assist namespace to use Assist Helpers like the CreateInstance&lt;T&gt; extension method. When called on a table, CreateInstance will match columns to properties to create and populate an object. Here’s how I use it with the above example:

[Given(@"I have the following configuration")]
public void GivenIHaveTheFollowingConfiguration(Table table)
{
    var config = table.CreateInstance();
}

When creating the object, case-sensitivity and whitespace are ignored, allowing you to create a human-friendly table of your object’s property values. You can also use a vertical table with properties listed as rows. This table will produce the same result as above:

Scenario: CreateInstance demo
    Given I have the following configuration
        | Field      | Value   |
        | output dir | c:\blah |
        | enabled    | true    |
    When I do something
    Then what I expected to happen happened

This technique allows you to create complex objects without having to write any code. You can create highly re-usable steps that rely on the scenario author to provide necessary details in the scenario itself. Simply awesome!

GitHubbin’

GitHub is a “social coding” service that offers web-based source control using the Git revision system. One of the appealing features is that you can host unlimited public repositories with a free account, which is perfect for my needs. I’ve wanted to look into it for a while as a way to host and share code that I write for many of the various projects that I write.

Getting Started

So I was finally ready to start using GitHub. I went to github.com, created a new free account, and created a new repository. It was all intuitive and seemingly very easy. But then I got stuck. I was looking all over for some sort of “upload code” link that would let me pick some files to be uploaded to the new repository. It didn’t exist. I was stuck.

The problem is that I didn’t take time to understand exactly what GitHub is or how it worked. I was trying to work through the bootcamp tutorial, and most of the steps were entering commands into a Unix-like prompt. That’s the point when I started to read more carefully and found that I needed to install Git locally, create a local repository, and then commit changes to the web repository. I was still just blindly entering commands into the prompt, not really understanding what I was doing, though.

I was struggling, but I somehow managed to stumble upon GitHub’s own Windows offering: GitHub for Windows. It got a lot easier at that point. GitHub for Windows is a polished, intuitive application. Using it, I was able to create my local repository, push it to GitHub, commit changes locally, and sync to the web. It was incredibly easy!

Tutorial

Using GitHub is really easy once you know what you’re doing, but it wasn’t quite as intuitive to figure out as I would’ve expected. Now that I have it (somewhat) figured out, I wanted to create a straightforward, barebones tutorial to help guide the masses. Note that this tutorial is based on my own experience in Windows.

  1. Create a new GitHub account at github.com
  2. Download and install GitHub for Windows from windows.github.com
  3. Start GitHub for Windows
  4. Create a new local repository
      Note the “Push to GitHub” option. If you select this, the repository will be created using the credentials you provide. If you don’t select this option, you can push it to GitHub later by clicking a button on the repository details page.
  5. Put some files in your local repository
      Just copy files into the local repository directory!
  6. Commit changes locally
      Double-click the local repository or click the arrow next to the repository to open it
  7. Add a commit message and click COMMIT
  8. Publish changes to the server
    • When you click the Publish button, your files will be uploaded to the web repository
    • After the initial publish, the button will change to “Sync” which can be used to upload additional local changes or retrieve changes made by others

That’s all there is to it. After your changes have been published, the code will be accessible through GitHub at https://github.com/%5Busername%5D/%5Brepositoryname%5D. You can get to your repository from GitHub for Windows by right-clicking the repository and choosing “view on github.”

Stubs in Microsoft Fakes

A few weeks back, I wrote an article about using shims in Microsoft Fakes. In addition to shims, Microsoft Fakes also has support for stubs. Stubs in Microsoft Fakes works a lot like other mocking frameworks such as Rhino Mocks, and it comes with a lot of the same limitations (must be visible, must be virtual, etc.). Let’s look at an example of what we can do using stubs in Microsoft Fakes. (Note: Microsoft Fakes requires Visual Studio 2012.)

Before we can use Microsoft Fakes, we need to do a little setup. Here are the steps that need to be performed:

  1. Create a new solution/project; I’m using a C# Console Application named “FakesSample”
  2. Add a new Unit Test Project to the solution; mine is named “FakesSample.Tests”
  3. Add a reference to the project to be tested in the test project
  4. In Solution Explorer, right-click the newly created reference and choose Add Fakes Assembly

And we’ll need some code to test. This is what I’ll be using:

namespace adamprescott.net.FakesSample
{
    using System;

    public class Mammal
    {
        public string Name { get; set; }
    }

    public interface IDataAccess
    {
        Mammal Get(int id);
    }

    public class Program
    {
        static void Main(string[] args)
        {
            var p = new Program();
            p.Run();
        }

        public IDataAccess DataAccess { get; set; }

        public void Run()
        {
            var m = DataAccess.Get(1);
            Console.WriteLine(m.Name);
        }
    }
}

Create a stub

The core functionality of a stub in unit tests is to remove external dependencies and/or provide makeshift functionality. In our code above, there is an IDataAccess that’s used to retrieve Mammal objects by ID. The Microsoft Fakes stub is perfect for this. In code, we create a Mammal object to be returned, and we return it using a lambda in our stub:

var m = new Mammal { Name = "Terry" };
var stubDataAccess = new StubIDataAccess
{
    GetInt32 = x =&gt; m
};

Verify a call

The stub gives us our desired behavior, but how do we know the stub was used? We can add verification by setting a flag in our stub implementation and asserting the value after the test is executed.

var m = new Mammal { Name = "Terry" };
var wasCalled = false;
var stubDataAccess = new StubIDataAccess
{
    GetInt32 = x =&gt; {
        wasCalled = true;
        return m;
    }
};
// ...
Assert.IsTrue(wasCalled);

I like that Microsoft Fakes makes this capability available without the need of a third party, but I’ll probably be sticking with Rhino Mocks for the time being. I haven’t found anything that I couldn’t do in MS Fakes that I could do with Rhino Mocks, but it often takes a lot more code. Maybe it’s because I’m less familiar with Fakes, or maybe it’s because Rhino Mocks is a more mature mocking framework–I’m not sure.

Here’s the complete unit test example:

namespace adamprescott.net.FakesSample.Tests
{
    using adamprescott.net.FakesSample.Fakes;
    using Microsoft.VisualStudio.TestTools.UnitTesting;

    [TestClass]
    public class ProgramTest
    {
        [TestMethod]
        public void RunWritesNameToConsole()
        {
            // Arrange
            var target = new Program();

            var m = new Mammal { Name = "Terry" };
            var wasCalled = false;
            var stubDataAccess = new StubIDataAccess
            {
                GetInt32 = x =&gt;
                {
                    wasCalled = true;
                    return m;
                }
            };
            target.DataAccess = stubDataAccess;

            // Act
            target.Run();

            // Assert
            Assert.IsTrue(wasCalled);
        }
    }
}

Test-first Development Cartoons

Lately, I’ve gotten into the habit of keeping a ton of can’t-forget-to-read-this articles open in separate tabs in my browser. It’s gotten to the point where I have so many tabs open that I can’t remember why or how I got to the tab in the first place. It’s kinda fun, though. Going back through the tabs feels like browsing new content that’s all be hand-picked by me for me. Thanks, me of the past!

I was flipping through some of the older tabs and found this Microsoft Patterns & Practices article about unit testing. This article is a bit long, but it serves as a good introduction to unit testing, touching on a variety of topics related to unit tests and automated system testing. There were a couple of cartoons about test-first development that I really liked in this article.

Writing unit tests before you write the code—test-first development—is recommended by most developers who have seriously tried it. Writing the tests for a method or interface makes you think through exactly what you want it to do. It also helps you discuss with your colleagues what is required of this particular unit. Think of it as discussing samples of how your code will be used.

For example, Mort, a developer at Fabrikam, has taken on the task of writing a method deep in the implementation of an ice-cream vending website. This method is a utility, likely to be called from several other parts of the application. Julia and Lars will be writing some of those other components. They don’t care very much how Mort’s method works, so long as it produces the right results in a reasonable time.

In the comic, Mort has been given some requirements and is left to come up with implementation details on his own. He writes tests to show an example of how his method will be used as well as to build in some assumptions. The point of test-first development is that you come up with a way to test before you begin coding. In practice, this is easier said than done, but it’s a good habit to get into. Writing tests first ensures that you write good, testable code. When you write the solution first, it’s much easier to end up with poorly tested or untested (gasp!) code.

The second comic introduces a new piece of functionality. Note that no code had been written at this point, but a new test is added to verify a different behavior of the same code. At the end of the comic, Mort points out that all that’s left to do now that the tests are in place is to write some code that makes the tests pass. Time is spent writing tests and thinking through requirements and scenarios up-front. Then, when you move on to the “real” development, it’s much easier because you’re just writing code to make the tests pass. (Way better than running code and repeating manual tests over and over again!)