Sometime in our tests, we need to control the test execution order. This mostly happens in our integration test and not as often as our unit tests. Some people might even argue that controlling the execution order of unit test is bad practice. You can read more about it here. The main reason to not order execution for unit tests are that they’re suppose to be independent from each other.

So when I’m talking about ordering the test execution I’m talking about integration tests. In this post I’m going to show how we can run both test classes and test method in order.

Control the Test Class Order By Using ITestCollectionOrderer

Before we go ahead with changing the test class execution order we need to know a little thing about test collections. In short test collections helps us to group test classes together for various purposes. You can read more about it here.

The important thing is if the Collection attribute is not specified, the test collection will be its fully qualified name. Which means if we order the execution of these test collections, they’re going to get ordered by their name if the collection attribute is not present. Let’s say we have these test classes and we want to change their execution order.

This particular sample is from Brad Wilson sample repository. As I said, we can remove the collections here and classes with their full name are going to be considered as a collection. Let’s say we have these classes and we want to run them alphabetically. What we can do is to create a class that is responsible for telling the xUnit framework how to order its test collections.

Here we created a class and implemented the ITestCollectionOrderer interface. As you can see I return an IEnumerable<ITestCollection> that is sorted by display name. So that’s going to be the order of execution.

Next step is to apply the TestCollectionOrderer attribute. We also need to stop ruining tests in parallel because we want to have control over order of the execution.

We can include these in AssemblyInfo.cs class or if not present we can include it at the beginning of our file. Now if we run the tests, the order will be TestsInCollection2, TestsInCollection3, TestsInCollection1.

Run Test Methods in Sequence By Using ITestCaseOrderer

Another scenario for ordering test is when we want to order test case methods and not test classes. We can do that by implementing ITestCaseOrderer to tell xUnit framework in what order our test cases should be running.

For example here we implemented the ITestCaseOrderer interface and sorted the test cases by their name and return them. Here’s how we use this order-er.

Let also see a more sophisticated scenario. We want to order our test cases in a custom manner and pass a priority number inside some kind of attribute. We want to use that attribute like this.

So let create a attribute.

Now we need to create a test case orderer that uses that attribute and return the test cases.

We first extract the priority value of the test case and store it in a dictionary with the test case. Because there can be multiple cases with the same priority, we select the test cases by priority. Again we compare the tests with the same priority by name, and execute them by the name if they have the same priority. Finally we yield return the test cases with the same priority if any. Now the test cases are going to be executed based on the priority number that we passed it. You can see the full repository of these examples here.

Summary

In this post we saw how we can change the execution order of our tests. We first saw how we can do this for different test classes and then we saw how to order test methods.