Mocking framework for tests

[bitPogo]

Description

Since this will be at some point a topic in terms of testing and has some implications - are we going for mocks?

Currently, there 3 Frameworks (I know of):

Mockative -> not sure if this is still active
MocKMP
KMock -> my baby/you can play with it here

Additional Information

A few words to KMock - I am currently preparing my next iteration of, which will bring, aside from some dependencies updates, some new stuff. But I will need a bit time for that.
However, KMock is a bit different to mockk or Mockito if you are used to it, so I recommend trying it upfront. The big plus is no worries about the setup since the KMock's gradle plugin will do it. It is tightly tested so most limitation are known this far.

[crocsandcoffee]

Awesome topic - might be better have thrown this under General Discussion first to reduce some noise/discussions, but all good! Lets see what others think and then we bring this in

I am most familiar with https://mockk.io/ but don't know if there are any implications with use in a KMP project. We should also write up some documentation/best practices in this repo with guidelines on when to use fakes vs mocks. It's a discussion that happens often in the community and this repo could serve as a great source of documentation and knowledge for best practices etc.

[bitPogo]

Mockk only works on jvm/android...so no KMP/you can only run tests on those platforms.
Besides, I think we need to clarify the terminology first (I definitely spend too much time on that topic) - I go by the definitions of M. Fowler. I am not sure what you mean by fakes, since these are in my experience the most expensive one.

MocKMP feels closest to mockk at first. But (cautious I am opinionated) has its issues might still break from case to case (but it got better). MocKMP is like mockk intrusive and provides only mocks.

Mockative is semi intrusive, but super opinionated about its usage.

KMock is also semi intrusive, since it produces Stubs, but you can also use it like mocks (with runs). Also, KMock can provide spies* which the others do not have so far I am aware of.

And I totally agree, this repo can be source of documentation as real life product.

[evant]

Not really a fan of mocking libs personally. I should really write a blog post about it, but https://www.billjings.com/posts/title/fakes-are-great-but-mocks-i-hate/ covers some problems pretty well.

[thebino]

I agree here, Use interfaces for abstraction and create Fakes instead of mocking.

[bitPogo]

Honestly, I cannot agree with that, after maintaining some KMP projects, with either handwritten stubs or fakes, which had to evolve over time. It takes time to adjust them, which could be spent otherwise, since fakes are intrusive as mocks. Also, assertions or verifications (if needed) have to be reinvented with each stub or fake (more or less).
Since we are on KMP we talk in any only about interfaces - but I would really love to understand the core of the problem.

[racka98]

I think fakes would be the best option for KMP

[crocsandcoffee]

So I don't think the question is mocks OR fakes. I think we will definitely want to decide on some mocking framework for tests. There are reasonable use cases for using one method over the other. If it's not clear how that distinction is made - then we have a great opportunity here to write up something as documentation with examples/guide for both, or links to existing articles etc that shed light on when to use one over the other.

There rarely is a single solution that covers all cases and this certainly isn't an exception.

Let me know your thoughts

[crocsandcoffee]

Also copied over a message from Alex in slack that I think is a useful take:

IMO you should fake by default, and mock when you'd end up reimplementing logic for verification.

Mock boundaries, fake data.

[evant]

this gets super annoying (in my opinion) once you start to refactor stuff or the code base gets a certain size

I've had that exact same problem with a codebase that used mocks heavily. I think it's more about where you chose the api boundries then how you implement them. I'm a fan of only faking what you need to (ex: api calls, db queries, etc) and use real classes for everything else to keep those boundaries well-defined.

[evant]

There are reasonable use cases for using one method over the other.

I agree this is true but I'm just not sure it's worth the downsides. In most places you should be verifying the state of your application instead of it's behavior. Mocks do shine when you need to actually verify behavior, a classic example being analytics.

val mockAnalytics = mock()
...
verify { mockAnalytics.log("action", mapOf("arg1" to "value1")) }

but I don't find implementing that manually that big of a deal and it prevents you to have to think about what to use

class FakeAnalytics : Analytics {
   val logs = mutableListOf<Pair<String, Map<String, String>>>
   
   override fun log(action: String, args: Map<String, String>) {
      logs.add(action to args)
   }
}

val fakeAnalytics = FakeAnlaytics()
...
assertEquals(fakeAnalytics.logs.last(), "action" to mapOf("arg1" to "value1"))

that being said, I've only every worked in codebases that used mocks that did them poorly so I'd be open to some smarter usage if we feel we can keep it reasonable.

[bitPogo]

I've had that exact same problem with a codebase that used mocks heavily. I think it's more about where you chose the api boundries then how you implement them. I'm a fan of only faking what you need to (ex: api calls, db queries, etc) and use real classes for everything else to keep those boundaries well-defined.

Well, that is really a question of the school of mind you are in - if you are more London School or Detroit School. Also, I do not think we need to replay the arguments for both sides (unless somebody wants it). There are a lot of articles and so out there.
However, the more interesting question is - how to incorporate both?

that being said, I've only every worked in codebases that used mocks that did them poorly so I'd be open to some smarter usage if we feel we can keep it reasonable.

I really hope we manage to do that, and better. Also, I hope the codebases were not that bad.

[kpgalligan]

We use KMock (in places where we auto-gen mocks, which isn't everywhere). We did a relatively extensive internal review of a few libraries and wound up using KMock. It was 7/8 months ago, though. Anyway, just my 2 cents. The other KMP options we tried out didn't have anything significantly different that was missing in KMock.

[thebino]

⛔️ This discussion is closed. Please vote or check for results