Unit Test with Exception

There was a developer came to me, asking me how to perform unit test that check for exception. If the expected exception is thrown, it should consider the test as a success.

We can use ExpectedException attribute to assert exception.

So, here are the sample code:

[TestMethod(), ExpectedException(typeof(ApplicationException),

    "Something is wrong, no exception had been thrown.")]

public void TestMethod1()

{

    //write some code here which will throw ApplicationException

    throw new ApplicationException("Test assert exception");

}

Debugging Problem with Visual Studio 2012 Unit Test

Today I found a very annoying problem while debugging my unit test with Visual Studio 2012. In a usual case scenario, when you are debugging your assembly code, at any breakpoint you can hit F10 or F11 to step over or step into the code line by line. However, today I find it behave differently, when I step over my code, the execution is skip jumping a few lines or more.

I would like to share the steps how I troubleshoot this problem and found the root cause and resolution.

First, check all your projects are not running in "Optimize Code" by opening up project properties and look for Build section then locate the "Optimize Code" checkbox is not checked.

Then, run your projects in debug mode, open up the Module window from the Debug menu, and then check all your assemblies are loaded and identify all the symbol paths are correct.

The loaded symbol paths look fishy. When you go to the project bin folder, you would notice that there are extra pdb files in the folder. For example:

<Assembly Name>.dll
<Assembly Name>.pdb
<Assembly Name>.instr.pdb

And then, in your Module window, it shows that the loaded symbols are actually <Assembly Name>.instr.pdb.

Well, when we enable Code Coverage in unit test, the instrumented assembly will be generated into another set of *.pdb files which are named as <Assembly Name>.instr.pdb. I never notice the *.instr.pdb would actually get loaded instead.

After disabling the Code Coverage, Test Impact, Intelli Trace, everything from my unit test setting, then do a solution cleaning and rebuild, no more *.instr.pdb file is generated. While debugging the code, only normal *.pdb files get loaded and no more code jumping around after hitting F10.

However, I am a little bit curious what is the actual root cause, so I go and enable back the Code Coverage. Surprisingly, *.instr.pdb files are not generated. So, I go and enable Test Impact, now I see *.instr.pdb files are generated, and I replicated the same problem occur earlier again.

So, is it a bug in Visual Studio 2012? :P

Data Driven Unit Testing with XML

I had been stopped writing blog for the past 3 months, it is really tough for me to keep up the momentum in writing blog in a timely manner. Well, until recently I join a new company and I suppose I will encounter some unfamiliar issue and new experience to share here more oftenly.

Back to the topic, I would like to share how to write data driven unit test with XML.

Do you ever encounter a scenario that you wish to test one test case with multiple different set of test data?

I used to experience in writing data driven unit test with MbUnit. But, today I want to share about doing the same thing with Visual Studio unit test.
Here is one of the solution with XML.

        [DeploymentItem("\\TestData\\TimeOffTestData.xml"), TestMethod()]
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML",
            "|DataDirectory|\\TestData\\TimeOffTestData.xml", 
            "TimeOffTest_ApplyTimeOffTest", DataAccessMethod.Sequential)]
        public void ApplyTimeOffTest()
        {
            TimeOff timeOff = new TimeOff();
            timeOff.UserId = int.Parse(TestContext.DataRow["UserID"].ToString());
            timeOff.StartDate = DateTime.Parse(TestContext.DataRow["StartDate"].ToString());
            timeOff.EndDate = DateTime.Parse(TestContext.DataRow["EndDate"].ToString());

            TimeOffComponent component = new TimeOffComponent();

            try
            {
                component.Apply(timeOff);
            }
            catch (Exception ex)
            {
                if (!Convert.ToBoolean(TestContext.DataRow["ExpectException"]))
                {
                    throw ex;
                }
                else
                {
                    Assert.AreEqual(typeof(InvalidTimeOffException), ex.GetType(), 
                        "This exception type was unexpected.");
                }
            }
        }

Here is the content of my test data XML:

  

<?xml version="1.0" encoding="utf-8" ?>
<TestData>
  <TimeOffTest_ApplyTimeOffTest>
    <UserID>1</UserID>
    <StartDate>2012-09-10</StartDate>
    <EndDate>2012-09-13</EndDate>
    <ExpectException>true</ExpectException>
  </TimeOffTest_ApplyTimeOffTest>
  <TimeOffTest_ApplyTimeOffTest>
    <UserID>1</UserID>
    <StartDate>2012-11-13</StartDate>
    <EndDate>2012-11-10</EndDate>
    <ExpectException>true</ExpectException>
  </TimeOffTest_ApplyTimeOffTest>
  <TimeOffTest_ApplyTimeOffTest>
    <UserID>1</UserID>
    <StartDate>2012-11-12</StartDate>
    <EndDate>2012-11-13</EndDate>
    <ExpectException>true</ExpectException>
  </TimeOffTest_ApplyTimeOffTest>
  <TimeOffTest_ApplyTimeOffTest>
    <UserID>1</UserID>
    <StartDate>2012-11-15</StartDate>
    <EndDate>2012-11-20</EndDate>
    <ExpectException>true</ExpectException>
  </TimeOffTest_ApplyTimeOffTest>
</TestData>

What's does Microsoft.VisualStudio.TestTools.DataSource.XML do at the background is above XML will be parsed into a table, hence <TimeOffTest_ApplyTimeOffTest> is a table name then its child elements are columns name then the element content are the row data. Therefore above XML describe 4 set of test data and the differences of the test data.

When you perform test run, it will perform 4 runs with different set of test data accordingly to the defined XML above.

Here is how the test result look like:

Common Problem

After you have finished coding, and you could not get the same result like above screenshot. Following are the common issues that you may encounter and here is the solution:

Known Issue: The unit test adapter failed to connect to the data source or read the data.

Resolution:

Go to your Solution Explorer, add new item to your solution, then select "Test Settings" from the left menu, then add a new test setting.

In the "Test Settings" window, select "Deployment" from the left menu, then check the "Enable Deployment" checkbox.

If problem still persist, go to Solution Explorer, locate your test data XML file and then open the file properties. Make sure the "Copy to Output Directory" property is NOT set to "Do not copy", make it either "Copy always" or "Copy if newer".

If problem still persist and you are using Visual Studio 2012, move your XML file to your project root directory. The problem is the |Data Directory| is not returning correct path. I notice some odd behavior between Visual Studio 2010 and 2012.

I have a folder call "TestData" that keep all my XML files in my test project well organized.

With Visual Studio 2010, no issue in detecting my XML file in the folder.
With Visual Studio 2012, I will get the error. However, there is a workaround for it, which is not to use |Data Directory| when specifying your test data file location. Instead, use real physical path. For example:

        [DeploymentItem("\\TestData\\TimeOffTestData.xml"), TestMethod()]
        [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML",
            "D:\\Projects\\DataDrivenUnitTest\\DataDrivenUnitTest.Test\\TestData\\TimeOffTestData.xml", 
            "TimeOffTest_ApplyTimeOffTest", DataAccessMethod.Sequential)]
        public void ApplyTimeOffTest()

There is one more workaround for solving the problem with |DataDirectory| in Visual Studio 2012. You can set Copy to Output Directory value to "Do not copy" for the XML file. Then, open the unit test project properties, then set the post-build event command line: copy "$(ProjectDir)TestData" "$(ProjectDir)$(OutDir)"

So that, every time you build your unit test project, the command line above copy all the files in the TestData folder to your output directory without the TestData folder automatically.



You may download my unit test project to play around: Download Here