Like many tech companies, at Moonpig.com we have a spider’s web of internal services communicating over HTTP. Services call other services. The website calls services. Our iOS and Android apps use these services. The services are owned by a number of different teams, and the apps are owned by entirely separate teams.
As our technological landscape evolves, the number of services we have increases. As the number of services increases, so does the number of dependencies between them. As the complexity increases, how do we know what service 1 requires of service 2? And how can we be sure that changes we make to introduce service 3 don’t break service 1?
Enter Consumer-Driven Contracts. This a pattern that seeks to reduce the risk of maintaining services by explicitly documenting and testing the dependencies between them. In short, the user of a service (the Consumer) defines a set of tests (a Pact) that specify exactly how the service they are calling (the Provider) should act when a given request is sent to it.
The Pact runs as unit tests against the classes in the Consumer that call the Provider. In effect the Pact is used to set up a mocked version of the Provider: we assert that the Consumer sends the expected request to the mock Provider, and we assert that the Consumer can handle the response from the mock Provider.
Now here’s the clever bit. The Pact, which was written by the team developing the Consumer, can also be used to test the Provider, which might be written by an entirely different team. When running against the Provider the Pact sets up a mocked Consumer to send the requests to the Provider.
A Pact can be versioned. As the services evolve, multiple versions of a Pact can be published to document each versioned dependency. We can store each version of a Pact, and run all of them against the Provider in order to assert that it can still support all versions of a Consumer. When we kill off old versions of a Consumer, we can delete their Pacts.
We adopted consumer-driven contracts around a year ago, after the technique appeared on the ThoughtWorks Technology Radar.
There’s a number of different frameworks out there that make it easier to adopt consumer-driven contract methodologies: Pact and Pacto are two of the more common options. Our services are .net based, so we use pact.net for testing the contracts between our .net services. Pact.net is, unsurprisingly, the .net implementation of the Pact protocol and it provides the framework for building and running Pacts against the Consumer and Provider.
Our build pipeline for the Consumer runs the Pact tests along side all the other Consumer unit tests; when these Pact tests pass we publish the Pact to a central repository of Pacts. When our build pipeline for the Provider runs, it pulls the relevant Pacts down from the repository and runs each of them against the Provider.
We use Pact Broker as our Pact repository, running in a docker container in Azure. Pact Broker is a ruby-based service which provides a nice RESTful API to publish and retrieve Pacts, and also provides some pretty diagrams which let us easily view the dependencies between services…
The greatest pain we’ve experienced in using Pact is when running them against the Provider. A little background…. when the Pacts run against the Consumer or Provider, the framework makes actual HTTP requests. When running against the Consumer, the Consumer code under test sends the HTTP requests to a mock Provider hosted by the Pact framework; conversely, when running against the Provider, the Pact framework sets up a mock consumer which sends the HTTP requests to wherever we tell it the provider is.
Because of this, we made the decision to run our Provider tests against against the full deployment of the Provider on our integration environment; after all, we already had this environment in place, and the Pacts are testing integration with the Consumer, so it seemed like a natural fit. However, this has lead to considerable pain in setting up the context for the Provider; we’re having to inject test data into the database each time the tests run in order to guarantee that the Provider returns the correct data to the mock Consumer. As anyone who has ever had to write tests that hit a database, this is painful. In addition, if the Provider depends on another service, we might have to also set up test data for *that* service.
To address this, we are currently reworking our implementation to run the provider tests against the Provider hosted in memory using the OWIN TestServer. This gives us the ability to focus only on the controller classes, and mock out anything called by the controllers, thus eradicating the need to touch the database. This makes more sense; after all, the Pact is only testing the interactions between the Consumer and Provider, not the entire Provider functionality. It also much more in line with how we run the tests directly against the classes in the consumer that send the HTTP requests; as is often the case in nature and science, symmetry is a beautiful thing. This approach also allows us to run the provider tests earlier in our build pipeline, alongside the Provider’s unit tests. It’s early days but so far it seems a big improvement in usability.
I think we haven’t yet felt the full benefit of our Pacts; until now, most of our Pacts are covering API endpoints in which the Consumer and Provider have been written by the same team. Furthermore, these APIs have not changed since the initial Pacts were written. So, whilst it has been useful to document the service dependencies, and know that our Consumer and Provider can talk to each other at the moment of creation, we haven’t fully tested how useful this all is in asserting the compatibility of multiple versions of Consumers and Providers as the APIs evolve. I expect this to change in the coming months, as we develop more services that will be used by our mobile apps.