21 Testing Mistakes - Part 2

6. Overusing mock frameworks

Let me start off by saying, I'm not a big fan of mock frameworks because, they increase the test suite run-time and usually decrease re-usability by repeating the same mock set-up instead of creating one re-usable test double.

This repeated mock framework boilerplate, sooner or later, will break in a lot of places when something changes (e.g. its contract). This is the exact opposite of what we do in production code (keeping things DRY). However, having a re-usable test double follows that good practices, because we don't repeat ourselves.

These handwritten test doubles are written by us, and can be extended to suit our needs. For example, we could have additional functions which we can use for verification avoiding some boilerplate and protect ourselves from additional changes:

class FakeNavigator : Navigator {

  var commands: List<NavigationCommand> = emptyList()
  
  override fun navigate(command: NavigationCommand) {
    commands.add(command)
  }

  fun assertLatestCommandEquals(item: Item) {
    assertEquals(item, commands.last())
  }
}

Another thing with mocking frameworks is that they are also easy to misuse by testing implementation details of the SUT instead of its behavior / outcome. This in turn leads to brittle test suites that break frequently when changing some implementation details even though the behavior stays the same. I will elaborate on this more in the next point.

I do see the benefit of mocking frameworks and use them sometimes, however in my opinions they have more downsides than upsides. If you want to keep using mocking frameworks, go ahead. My only suggestion would be that if you have classes / interfaces which are mocked frequently, create a Mock Wrapper or a Test Double by hand, it will keep your test code DRY and encourage more re-usability.

7. Testing implementation details instead of the outcome

Before delving into the details, I want to point out that the popular "mock" represents all form of test doubles: Fakes, Stubs, Spies etc. Because of this, it's hard to differentiate between different types of test doubles. In most cases, a Stub or a Fake should cover most of the needs, however with mock frameworks, usually everything becomes a mock.