21 Testing Mistakes - Part 2

The second part of my Testing Mistakes series which is only available to Members / Subscribers
This part focuses on
6. Overusing mock frameworks
7. Testing implementation details instead of the outcome
8. Comparing whole complex data structures instead of only the tested fields
9. Not using Soft assertions
10. Repeating the SUT Algorithm in assertions instead of an explicit value

6. Overusing mock frameworks

Let me start off by saying, I'm not a big fan of mock frameworks because, they increase the test suite run-time and usually decrease re-usability by repeating the same mock set-up instead of creating one re-usable test double.

This repeated mock framework boilerplate, sooner or later, will break in a lot of places when something changes (e.g. its contract). This is the exact opposite of what we do in production code (keeping things DRY). However, having a re-usable test double follows that good practices, because we don't repeat ourselves.

These handwritten test doubles are written by us, and can be extended to suit our needs. For example, we could have additional functions which we can use for verification avoiding some boilerplate and protect ourselves from additional changes:

class FakeNavigator : Navigator {

  var commands: List<NavigationCommand> = emptyList()
  override fun navigate(command: NavigationCommand) {

  fun assertLatestCommandEquals(item: Item) {
    assertEquals(item, commands.last())

Another thing with mocking frameworks is that they are also easy to misuse by testing implementation details of the SUT instead of its behavior / outcome. This in turn leads to brittle test suites that break frequently when changing some implementation details even though the behavior stays the same. I will elaborate on this more in the next point.

I do see the benefit of mocking frameworks and use them sometimes, however in my opinions they have more downsides than upsides. If you want to keep using mocking frameworks, go ahead. My only suggestion would be that if you have classes / interfaces which are mocked frequently, create a Mock Wrapper or a Test Double by hand, it will keep your test code DRY and encourage more re-usability.

Mocking is not practical — Use fakes
This article talks about the benefits fakes provide over mocks in testing software. Fakes lead to better API and readable/robust tests.
Replacing Mocks :: Ryan Harter
After reading a recent post by Sam Edwards detailing how he wraps mock objects for tests, I had a good discussion with him and others about the alternative: simply making Fake implementations.

7. Testing implementation details instead of the outcome

Before delving into the details, I want to point out that the popular "mock" represents all form of test doubles: Fakes, Stubs, Spies etc. Because of this, it's hard to differentiate between different types of test doubles. In most cases, a Stub or a Fake should cover most of the needs, however with mock frameworks, usually everything becomes a mock.

This post is for subscribers only

Sign up to read this post and all other free member only posts.
You've successfully subscribed to AKJAW
Great! Now you have full access to all members content.
Error! Could not sign up. invalid link.
Welcome back! You've successfully signed in.
Error! Could not sign in. Please try again.
Success! Your account is fully activated, you now have access to all content.
Error! Stripe checkout failed.
Success! Your billing info is updated.
Error! Billing info update failed.