When to use Test Doubles
Test doubles are tools to simplify our tests, they can also be used to design code not yet written. To define granular units that are more…
Test doubles are tools to simplify our tests, they can also be used to design code not yet written. To define granular units that are more or less independent of others. But they are not the production code, they are a double.
Test doubles are tools to simplify our tests, usually to use them you need to adapt your code. You need to introduce collaborators that help you to extract some logic, the one that you want to test independently to introduce a double for the upper logic that use your collaborator.
Some definitions extracted from Martin’s Fowler blog:
Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.
Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an InMemoryTestDatabase is a good example).
Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test.
Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent.
Mocks are pre-programmed with expectations which form a specification of the calls they are expected to receive. They can throw an exception if they receive a call they don’t expect and are checked during verification to ensure they got all the calls they were expecting.
As we said before, we have to adapt our code to use them. My service under test (SUT) will do fewer things, it will have fewer responsibilities. I’m delegating to the production implementation represented by my test double some responsibilities.
If my code is basically playing with a third party library, framework, or anything that I cannot easily change, to generate any kind of side effect, what I really want to check is how good is my code using that library.
In this case my code is so coupled to the library that in fact is just using it, in that case I don’t think it is a good idea to change the library by a double.
Only mock what you own, because mocking others is rude.
https://twitter.com/hynek/status/1478384282766913548
I would say as a general rule, don’t change libraries or frameworks by any kind of test doubles.
Let’s imagine that I want to call a rest endpoint outside my organization and I want to create a test for that thin layer that’s doing the final work.
I know that I cannot call the real rest endpoint in my test, so I need some kind of double at some point.
Following the rule described above, I need to use the real library to call the rest endpoint, so the only thing I can do is to fake the server. And this is good, because I will test that I really use correctly the library, that I really call the correct endpoint that I can manage errors coming from the library in any situation. Basically, I’m testing my integration with that library.
That fake server is just a set of predefined rest json answers to a set of requests, but will force our code to interact correctly with the library and all the situations we want to test.
These assumptions are just one part of the contract, yes test-doubles define a contract between the client, the service under test (SUT), and our server (the fake server in this case).
Server side contract
You could say what happens if the server changes, then the contract is broken, there are two situations here:
You control the server, if the server in the example is being developed by other team in your organization, and there is good communication between teams. In that case, you can use contract tests to check that the server cannot break the contract (for example, with Pact).
If you don’t control the server, then you only can be sure that things are changing monitoring the server. If you have access to a staging environment you can monitor that environment to be sure that things don’t change (base of what your integration needs), if not then monitor in production the server, to have early feedback on what’s going on and react as fast as you can.
There is no much value trying to check the integration between the client and the server through e2e tests executed in the pipeline if you don’t control the server. The e2e tests tend to test a lot of different aspects:
The frontend part of your application.
The data in your database.
The integration with the server.
The logic of the whole application.
A test that fails with a big scope that cannot tell you where is failing is hard to verify, devs don’t like to repeat investigations once and again.
Much easier and effective is to monitor what is happening in the integration between both systems.
Input and output
Please, don’t use doubles for input data and output results in general (apart from dummies). This is specially bad if you are creating them because those inputs and outputs are hard to create. This is the same symptom why I don’t like the builder pattern, it usually means that we are not honoring the Interface Segregation Principle of SOLID.
Those values are probably not the thing we are interested on, we are just using them because they had been created in the past, and we are lazy to create new abstractions.
The risk of doubles
Test doubles come with a risk (in software, everything is a trade-off). When you introduce a test double in your tests, you are taking assumptions of how the production code will behave.
As I said, If you control the production code you are changing by the test double when you create it, you have to force the production code to behave the same as the double for the inputs described in the test that use the double.
The risk then is changing the production code in the future without changing your double, then our tests will pass, but our code will be broken in production.
I usually have this risk in mind so to minimize it, I try to use doubles just in the borders between layouts, and in special cases inside a layout if they pay the bill (the logic is so complicated that I reduce the complexity of my tests).
To use them only in the borders between layouts help me to identify them, in some sense to define the layouts, to design.
There is also the possibility as described before to use contract tests to validate that the contract is not lost between doubles and your production code.
When to use them
Use test doubles to delegate decisions for the future, if you yet don’t know what db to use or how the final rest-endpoint will be. Doubles will help you to create wrappers over those concepts that isolate you (at a level) from all the specific decisions that are not important to build your business logic.
Use doubles to reduce the complexity of your production code, so to reduce the number of tests you have to create in your SUT.
There are devs that try to avoid the usage of test doubles, they try to test everything with real libraries and infrastructure. I don’t like that approach because the effects of having a lot of integration or e2e tests are Low Feedback Loops (very slow pipelines). People tend to create big batches when the test feedback loop is slow.