One of the hallmark of a good unit test is its maintainability. The test that is not maintainable is a pain and there is a higher tendency for it to be ignored. One of the mistakes in software testing is Over-specification of tests. This problem often comes with testing object interaction with a mock object when in fact value-based testing is enough. In this post I’m going to discuss why interaction based testing makes our test brittle and leads to less maintainable test. I also present some argument against interaction based testing when it is not necessary.
Mock vs Stub
Let’s first see what mock object is and how is it different from stubs. There is a seminal article by Martin Fowler that discusses this difference called Mocks Aren’t Stubs. In short mocks record the interaction between objects and then verify that interaction has happened in expected way. In other words mock measure interaction and can fail the test on its own. Stubs on the other hand only substitute dependency and are like predetermined behavior with specific data. They are designed to make the SUT behave in specific way or change the flow of our program.
Different Kinds of Unit Test
Now let’s see how unit tests can differ in the way they perform their task. We have three option when we want to test something.
- Value Based Testing: We test to see if the expected value is returned from a function
- State Based Testing: We test to see if the data and behavior of our system has changed
- Interaction Based Testing: We test to see if particular object interacted with another in an specific way using mock objects
Now we have enough vocabulary to start discussing which one of these approaches leads to a more maintainable test.
Why Interaction Based Tests Leads To Over-specification
Take the above action method for example, at first glance there are five things which needs testing in this code.
idnot supplied for this action, it returns
- It correctly passes the
PortfolioByIdQueryresult was null it return
- It passes the correct model into Map method
- It returns the correct view and view model
For the number two, I could write a test like the following.
Here we test to see if what passed into
PortfolioByIdQuery has indeed the same
PortfolioId as what we’ve passed into our action method. But is that really necessary? I mean think about it, here we’re testing Detail action and we isolate
PortfolioByIdQuery and the call to
Map. But is number 2 and number 4 really falls into our specification for this action method? I don’t think so, here’s a good article that discusses this. If we really think about it we understand that number 2 and 4 doesn’t really influence what the detail actually does because as I’ve said we’ve isolated them. So what I end up with for this action is four tests.
By doing that I have less test to write. Also I made sure that if in the future I changed how case number 2 and 4 works, I have less unit test to change, and less failed tests. That’s partly because mocks can fail the test and we do the assertion on them, but stubs do cannot fail our tests (not on their own anyway). That of course doesn’t mean that we should avoid mocks, but here our method has a return type that we can use for testing. Maybe if our method was void and didn’t produce any measurable side effect in our system, then interaction based testing was necessary. It all comes down to our specification, but it can be avoided in most cases.
In this post, I’ve explained what is the difference between mock and stub and described different method of going about testing a piece of code. I also discussed why interaction based testing and lead to Over-specification and less maintainable code as a result.