One of the hallmark of a good unit test is its maintainability. The test that is not maintainable is a pain and there is a higher tendency for it to be ignored. One of the mistakes in software testing is Over-specification of tests. This problem often comes with testing object interaction with a mock object when in fact value-based testing is enough. In this post I’m going to discuss why interaction based testing makes our test brittle  and leads to less maintainable test. I also present some argument against interaction based testing when it is not necessary.

Mock vs Stub

Let’s first see what mock object is and how is it different from stubs. There is a seminal article by Martin Fowler that discusses this difference called Mocks Aren’t Stubs. In short mocks record the interaction between objects and then verify that interaction has happened in expected way. In other words mock measure interaction and can fail the test on its own. Stubs on the other hand only substitute dependency and are like predetermined behavior with specific data. They are designed to make the SUT behave in specific way or change the flow of our program.

Different Kinds of Unit Test

Now let’s see how unit tests can differ in the way they perform their task. We have three option when we want to test something.

1. Value Based Testing: We test to see if the expected value is returned from a function
2. State Based Testing: We test to see if the data and behavior of our system has changed
3. Interaction Based Testing: We test to see if particular object interacted with another in an specific way using mock objects

Now we have enough vocabulary to start discussing which one of these approaches leads to a more maintainable test.

Why Interaction Based Tests Leads To Over-specification

Take the above action method for example, at first glance there are five things which needs testing in this code.

1. If id not supplied for this action, it returns BadRequest.
2. It correctly passes the id into PortfolioByIdQuery
3. If PortfolioByIdQuery result was null it return NotFound.
4. It passes the correct model into Map method
5. It returns the correct view and view model

For the number two, I could write a test like the following.

Here we test to see if what passed into PortfolioByIdQuery has indeed the same PortfolioId as what we’ve passed into our action method. But is that really necessary? I mean think about it, here we’re testing Detail action and we isolate PortfolioByIdQuery and the call to Map. But is number 2 and number 4 really falls into our specification for this action method? I don’t think so, here’s a good article that discusses this. If we really think about it we understand that number 2 and 4 doesn’t really influence what the detail actually does because as I’ve said we’ve isolated them. So what I end up with for this action is four tests.

By doing that I have less test to write. Also I made sure that if in the future I changed how case number 2 and 4 works, I have less unit test to change, and less failed tests. That’s partly because mocks can fail the test and we do the assertion on them, but stubs do cannot fail our tests (not on their own anyway). That of course doesn’t mean that we should avoid mocks, but here our method has a return type that we can use for testing. Maybe if our method was void and didn’t produce any measurable side effect in our system, then interaction based testing was necessary. It all comes down to our specification, but it can be avoided in most cases.

Summary

In this post, I’ve explained what is the difference between mock and stub and described different method of going about testing a piece of code. I also discussed why interaction based testing and lead to Over-specification and less maintainable code as a result.