21 Testing Mistakes - Part 3

11. Not focusing on boundary values

Sometimes when our System Under Test might be using numbers in some conditions. For example, checking if a number is greater or equal:

class SomeKindOfCalculator {

    fun isValueOverTheLimit(number: Int): Boolean {
        return number >= 5
    }
}

We could write a passing tests using values which are far above or below the "limit":

@Test
fun sevenIsOverTheLimit() {
    val result = systemUnderTest.isValueOverTheLimit(7)
    
    assertEquals(true, result)
}

@Test
fun twoIsUnderTheLimit() {
    val result = systemUnderTest.isValueOverTheLimit(2)

    assertEquals(false, result)
}

But such tests will still pass even if we change the limit number, which is probably not a desired outcome. The number 5 is an important part of the SUT, so the tests should probably fail when it is changed by accident:

@Test
fun fiveIsOverTheLimit() {
    val result = systemUnderTest.isValueOverTheLimit(5)

    assertEquals(true, result)
}

@Test
fun fourIsUnderTheLimit() {
    val result = systemUnderTest.isValueOverTheLimit(4)

    assertEquals(false, result)
}

The first iteration of the tests can produce a False Negative (Test passes even though production code is broken). These types of tests give us a false comfort of security, because the regressions might not be caught by our automated test suite. It will be either need to be caught by manual testing, or worse it will land production and cause us some problems.

Focusing on boundary values, helps us avoid this issue altogether, giving us more confidence when releasing and shortening the development loop. Because if some issue is caught by a Manual test, we will probably be focusing on something different at the moment, but we will need to go back to this issue, change context, fix it and hope it will pass.

12. Using "private" functions in tests

Private functions are private for a reason, they are implementation details of the system under test and should not be exposed to the outside just for testing.