←back to thread

An overly aggressive mock can work fine, but break much later

(nedbatchelder.com)
67 points ingve | 3 comments | 16 Nov 25 22:36 UTC | HN request time: 0.4s | source
Show context
lelandbatey ◴[17 Nov 25 00:41 UTC] No.45949894[source]▶
I feel like the #1 reason mocks break looks nothing like this and instead looks like: you change the internal behaviors of a function/method and now the mocks interact differently with the underlying code, forcing you to change the mocks. Which highlights how awful mocking as a concept is; it is of truly limited usefulness for anything but the most brittle of tests.

Don't test the wrong things; if you care about some precondition, that should be an input. If you need to measure a side effect, that should be an output. Don't tweak global state to do your testing.

replies(3): >>45950085 #>>45951034 #>>45955029 #
dpark ◴[17 Nov 25 05:22 UTC] No.45951034[source]▶
> you change the internal behaviors of a function/method and now the mocks interact differently with the underlying code, forcing you to change the mocks

Rarely should a mock be “interacting with the underlying code”, because it should be a dead end that returns canned data and makes no other calls.

If your mock is calling back into other code you’ve probably not got a mock but some other kind of “test double”. Maybe a “fake” in Martin Fowler’s terminology.

If you have test doubles that are involved in a bunch of calls back and forth between different pieces of code then there’s a good chance you have poorly factored code and your doubles are complex because of that.

Now, I won’t pretend changes don’t regularly break test doubles, but for mocks it’s usually method changes or additions and the fix is mechanical (though annoying). If your mocks are duplicating a bunch of logic, though, then something else is going on.

replies(1): >>45956320 #
1. lelandbatey ◴[17 Nov 25 18:22 UTC] No.45956320[source]▶
To fully show my hand: What I do not like are mocks which have you outline a series of behaviors/method calls which are expected to be called, typically with specific input parameters, often with methods in order. In python, this looks like setting up Mocks and then using `assert_called` methods. In Go, this means using Gomock-generated structs which have an EXPECT.

A test should not fail when the outputs do not change. In pursuit of this ideal I often end up with fakes (to use Martin Fowler's terms) of varying levels of complexity, but not "mocks" as many folks refer to them.

[0] - https://docs.python.org/3/library/unittest.mock.html#unittes...

[1] - https://pkg.go.dev/github.com/golang/mock/gomock

replies(2): >>45956468 #>>45967865 #
2. dpark ◴[17 Nov 25 18:35 UTC] No.45956468[source]▶
So this I agree with. Mocks are often used to over-constrain and end up validating that the implementation does not change rather than does not regress.

There are some specific cases, such as validating that caching is working as expected, where it can make sense to fully validate every call. Most of the time, though, this is a pointless exercise that serves mostly to make it difficult to maintain tests.

It can sometimes also be useful as part of writing new code, because it can help validate that your mental model for the code is correct. But it’s a nightmare for maintenance and committing over-constrained tests just creates future burden.

In Fowler terminology I think I tend to use Stubs rather than Mocks for most cases.

3. bccdee ◴[18 Nov 25 15:50 UTC] No.45967865[source]▶
Yeah I think true mocks often lead to fragile, hard-to-read tests. A text which expects a bunch of specific interactions with a mock is white-box, whereas a test which tests the state of a fake after the operation is complete is mostly black-box. Then, if your dependency gets updated, only the fake breaks, rather than all of your test expectations.

A particularly complex fake can even be unit-tested, if need be. Of course, if you're writing huge fakes, there's probably something wrong with your architecture, but I feel like good testing practices should give you options even when you're working with poorly architected code.