Kotlin Multiplatform Parameterized Tests and Grouping Using The Standard Kotlin Testing Framework

I covered the Kotlin Multiplatform Testing ecosystem before. But I'll try to summarize it briefly here.

Coming from Android, we're a little spoiled when it comes to testing frameworks. The default is JUnit 4, but we also have JUnit 5 which are very mature frameworks. On Kotlin Multiplatform, we can't leverage JUnit features like parameterized tests and nested test classes. By default, Kotlin Multiplatorm uses the kotlin.test framework, which unfortunately is not as feature rich as JUnit.

There are alternatives frameworks like the Kotest, which does offer parameterized tests, but has some limitations with Kotlin Multiplatform like the Kotest plugin not being able to run tests from commonTest, so they always need to be executed from the command line.

However, in this article I'd like to focus on the standard testing framework, because it comes out of the box with Kotlin and has good IDE support, and also has a similar API as JUnit.

Parameterized tests

I tried to keep the test examples simple and self-explanatory, the last one is more complicated, but I hope it's still easy to follow.

Parsing a string to an Enum

We have an Enum class representing Card (Suite), and an extension function that parses a string and returns the corresponding card or null.

enum class Card {
    HEARTS,
    DIAMONDS,
    SPADES,
    CLUBS
}

fun String.parseToCard(): Card? =
    when (this) {
        "Hearts" -> Card.HEARTS
        "Diamonds" -> Card.DIAMONDS
        "Spades" -> Card.SPADES
        "Clubs" -> Card.CLUBS
        else -> null
    }

class CardParsingTest {

    @Test
    fun `'Hearts' is a card`() = validTestCase("Hearts", Card.HEARTS)

    @Test
    fun `'Diamonds' is a card`() = validTestCase("Diamonds", Card.DIAMONDS)

    @Test
    fun `'Spades' is a card`() = validTestCase("Spades", Card.SPADES)

    @Test
    fun `'Clubs' is a card`() = validTestCase("Clubs", Card.CLUBS)

    @Test
    fun `'Heart' is not a card`() = invalidTestCase("Heart")

    @Test
    fun `'Diamons' is not a card`() = invalidTestCase("Diamons")

    @Test
    fun `'Spade' is not a card`() = invalidTestCase("Spade")

    @Test
    fun `'Club' is not a card`() = invalidTestCase("Club")

    private fun validTestCase(stringCard: String, expectedCard: Card) {
        assertEquals(actual = stringCard.parseToCard(), expected = expectedCard)
    }

    private fun invalidTestCase(stringCard: String) {
        assertNull(actual = stringCard.parseToCard())
    }
}

There are two helper functions validTestCase and invalidTestCase, depending on the expected outcome the respective "TestCase" is called. Having one function could also work in this case, but I personally think that having this distinction makes the test class more readable.

Formatting names

We have a Person class which we want to format depending on the available data.

data class Person(
    val firstName: String,
    val lastName: String,
    val secondName: String? = null,
    val secondLastName: String? = null
)

fun Person.formatToString(): String {
    val secondName = if (secondName == null) "" else " $secondName"
    val secondLastName = if (secondLastName == null) "" else "-$secondLastName"
   return "$firstName$secondName $lastName$secondLastName"
}

class PersonFormattingTest {

    @Test
    fun `Full name is correctly formatted`() = testCase(
        Person(firstName = "John", lastName = "Doe"),
        "John Doe"
    )

    @Test
    fun `Full name with second name is correctly formatted`() = testCase(
        Person(firstName = "John", secondName = "Bob", lastName = "Doe"),
        "John Bob Doe"
    )

    @Test
    fun `Full name with second last name is correctly formatted`() = testCase(
        Person(firstName = "John", lastName = "Doe", secondLastName = "Dilly"),
        "John Doe-Dilly"
    )

    @Test
    fun `Full name with second name and last name is correctly formatted`() =
        testCase(
            Person(
                firstName = "John",
                secondName = "Bob",
                lastName = "Doe",
                secondLastName = "Dilly"
            ),
            "John Bob Doe-Dilly"
        )

    private fun testCase(person: Person, expectedString: String) {
        assertEquals(actual = person.formatToString(), expected = expectedString)
    }
}

Searching through keywords

We have a UseCase which filters available keywords for a given query. The keywords come from a repository, which is replaced with a test double during testing.

interface KeywordRepository {
    suspend fun getKeywords(): List<String>
}

class FakeDelayingKeywordRepository : KeywordRepository {
    var keywords: List<String> = emptyList()

    override suspend fun getKeywords(): List<String> {
        delay(500) // Simulate some I/O operation
        return keywords
    }
}

class SearchForKeyword(
    private val keywordRepository: KeywordRepository,
    private val dispatcher: CoroutineDispatcher,
) {

    sealed class SearchResult {
        object Empty : SearchResult()
        object InvalidQuery : SearchResult()
        object Error : SearchResult()
        data class Success(val keywords: List<String>) : SearchResult()
    }

    suspend fun execute(query: String): SearchResult = withContext(dispatcher) {
        if (query.count() < 3) return@withContext InvalidQuery

        val keywords = keywordRepository.getKeywords()

        if (keywords.isEmpty()) return@withContext Error

        val matches = keywords.filter { keyword -> keyword.contains(query) }
        if (matches.isEmpty()) {
            Empty
        } else {
            Success(matches)
        }
    }
}

class SearchTest {

    private lateinit var fakeKeywordRepository: FakeDelayingKeywordRepository
    private lateinit var dispatcher: TestDispatcher
    private lateinit var systemUnderTest: SearchForKeyword

    @BeforeTest
    fun setUp() {
        fakeKeywordRepository = FakeDelayingKeywordRepository()
        dispatcher = UnconfinedTestDispatcher()
        systemUnderTest = SearchForKeyword(fakeKeywordRepository, dispatcher)
    }

    @Test
    fun `When query is empty then InvalidQuery is returned`() =
        testCase(query = "", expectedResult = InvalidQuery)

    @Test
    fun `When query has 2 characters then InvalidQuery is returned`() =
        testCase(query = "fi", expectedResult = InvalidQuery)

    @Test
    fun `When query valid the expected Success result is returned`() =
        testCase(
            query = "fir",
            keywords = listOf("first", "second", "Fire", "sound"),
            expectedResult = Success(listOf("first")),
        )

    @Test
    fun `When query valid but does not match any keywords then Empty is returned`() =
        testCase(
            query = "asd",
            keywords = listOf("second", "Fire", "sound"),
            expectedResult = Empty,
        )

    @Test
    fun `When query valid but keyword repository is empty then Error is returned`() =
        testCase(
            query = "asd",
            keywords = emptyList(),
            expectedResult = Error,
        )

    private fun testCase(
        query: String,
        expectedResult: SearchForKeyword.SearchResult,
        keywords: List<String> = emptyList()
    ) =
        runTest(dispatcher) {
            fakeKeywordRepository.keywords = keywords

            val result = systemUnderTest.execute(query)

            assertEquals(actual = result, expected = expectedResult)
        }
}

This test is more complicated compared to the last ones, because it requires some Arrangement, before the test Action and Assertions. It also resembles more of a "real" thing that happens during app development.

The added benefit of having a "TestCase" function is that the systemUnderTest could also be created inside of it, without repeating the same boilerplate in every test. This can be helpful if a Test Double or System Under Test take in different constructor parameters depending on the test.

Grouping tests in Kotlin Multiplatform

Another topic I talked about in the introduction was JUnit 5 nested classes used for grouping test cases. This is not available in the standard Kotlin testing library, but there is another way of grouping test cases.

Instead of nested classes, they could be normal classes in different files. The problem with this is that the System Under Test creation has to be repeated in every test class.

To illustrate this problem, I refactored the Search UseCase to be more complicated by extracting out some classes:

class QueryValidator {

    fun isValid(query: String): Boolean = query.count() < 3
}

class KeywordFilter {

    fun execute(query: String, keywords: List<String>): List<String> {
        return keywords.filter { keyword -> keyword.contains(query) }
    }
}

The constructor of the UseCase is now the following:

class SearchForKeyword(
    private val keywordRepository: KeywordRepository,
    private val queryValidator: QueryValidator,
    private val keywordFilter: KeywordFilter,
    private val dispatcher: CoroutineDispatcher,
)

System Under Test creation

The instantiation of the system under test is now more involved:

private lateinit var keywordRepository: FakeDelayingKeywordRepository
private lateinit var dispatcher: TestDispatcher
private lateinit var systemUnderTest: SearchForKeyword

@BeforeTest
fun setUp() {
    keywordRepository = FakeDelayingKeywordRepository()
    dispatcher = UnconfinedTestDispatcher()
    systemUnderTest = SearchForKeyword(
        keywordRepository,
        QueryValidator(),
        KeywordFilter(),
        dispatcher
    )
}

Repeating this in every test class will cause the test class to be more complex and harder to maintain every time a constructor parameters changes. To help with that, a helper Object Mother function can be created:

fun createSearchForKeyword(
    dispatcher: CoroutineDispatcher,
    keywordRepository: KeywordRepository = FakeDelayingKeywordRepository(),
): SearchForKeyword {
    return SearchForKeyword(
        keywordRepository,
        QueryValidator(),
        KeywordFilter(),
        dispatcher
    )
}

The QueryValidator and KeywordFilter are implementation details from the perspective of the test, so their creation is hidden here. However, the dispatcher and repository are used inside the test class, so they are created in the test class and passed in here.

Different Test classes

The test cases for the Search UseCase remain the same because the behavior did not change, however the test class will be split into two: SearchInvalidKeywordTest and SearchValidKeywordTest

class SearchInvalidKeywordTest {

    private lateinit var dispatcher: TestDispatcher
    private lateinit var systemUnderTest: SearchForKeyword

    @BeforeTest
    fun setUp() {
        dispatcher = UnconfinedTestDispatcher()
        systemUnderTest = createSearchForKeyword(dispatcher)
    }

    @Test
    fun `When query is empty then InvalidQuery is returned`() =
        testCase(query = "")

    @Test
    fun `When query has 2 characters then InvalidQuery is returned`() =
        testCase(query = "fi")

    private fun testCase(query: String) =
        runTest(dispatcher) {
            val result = systemUnderTest.execute(query)

            assertEquals(actual = result, expected = InvalidQuery)
        }
}

The keyword repository is removed from here, because it is not used by the tests. The default parameter in createSearchForKeyword function creates the repository.

class SearchValidKeywordTest {

    private lateinit var fakeKeywordRepository: FakeDelayingKeywordRepository
    private lateinit var dispatcher: TestDispatcher
    private lateinit var systemUnderTest: SearchForKeyword

    @BeforeTest
    fun setUp() {
        fakeKeywordRepository = FakeDelayingKeywordRepository()
        dispatcher = UnconfinedTestDispatcher()
        systemUnderTest = createSearchForKeyword(dispatcher, fakeKeywordRepository)
    }

    @Test
    fun `When query valid the expected Success result is returned`() =
        testCase(
            query = "fir",
            expectedResult = Success(listOf("first")),
            keywords = listOf("first", "second", "Fire", "sound")
        )

    @Test
    fun `When query valid but does not match any keywords then Empty is returned`() =
        testCase(
            query = "asd",
            expectedResult = Empty,
            keywords = listOf("second", "Fire", "sound")
        )

    @Test
    fun `When query valid but keyword repository is empty then Error is returned`() =
        testCase(
            query = "asd",
            expectedResult = Error,
            keywords = emptyList()
        )

    private fun testCase(
        query: String,
        expectedResult: SearchForKeyword.SearchResult,
        keywords: List<String>
    ) =
        runTest(dispatcher) {
            fakeKeywordRepository.keywords = keywords

            val result = systemUnderTest.execute(query)

            assertEquals(actual = result, expected = expectedResult)
        }
}

In this test class, the keyword repository is a property because it is used to Arrange the correct return value before calling the System Under Test.

This method of grouping tests, is more verbose and complicated which for simple cases might be an overkill, but when the test cases keep growing (like when there are a lot of edge cases), this grouping is definitely better than having one 600+ line test class.

Summary

Although the standard Kotlin testing framework lacks some JUnit features, we are able to implement them by hand. They are more verbose, but we have to work with what we got.

If you have your own strategies for writing Kotlin Multiplatform tests, feel free to share them in the comments 😁