The steps to reaching Pact Nirvana
This is a technical document that provides a roadmap for setting up consumer driven contract (CDC) testing using Pact. By the end of the document, you will understand how to create a release pipeline that allows you to independently deploy any application with confidence that it will work correctly with the other applications in its environment - without having to run a suite of end to end tests.
Before you read this document, you should:
This document can be used both for greenfields projects, and retrofitting Pact into an existing project. Pick and choose the steps that apply to your team, as each integration is different. Technology; development, testing and deployment processes; history; organisational culture; all these things vary from team to team, and will impact what Pact workflow makes sense for you. You may implement only the first few steps described below, and just use Pact as a precursor to your standard integration tests; or you may throw away your integration tests altogether and reach “Pact Nirvana”.
As Pact has been implemented in many different languages, this document will outline the theory for each step. You will need to consult the documentation of your chosen language to learn the relevant syntax for each task.
Contracts are not a replacement for good communication between or within teams. In fact, contracts require collaboration and communication. One could make the argument that this is one of the main reasons to leverage Pact and enforce communication pathways in large internal and external development organizations. Contracts are not a magical silver bullet that will allow you to hide in your developer caves and toss built artifacts at each other until everything passes. You need all teams to be invested in the process. Collaborate about the problems, collaborate over the design, and keep the communication channels open.
If you have not used Pact before, consider trying it out on some “spike” applications first. Using the technology stacks of your consumer and provider respectively, write a consumer test for a single endpoint (eg “GET /helloworld”). Generate the pact. Send it to the provider team - nothing fancy, just email the file. Get them to implement the provider, and verify the pact. Next, you could try adding provider states and matchers. Make sure you understand how the pact generation and verification steps work.
Now you have two different sets of tests in two different codebases. The artifacts that tie these tests together are the pact, and the verification results. The Pact Broker is a service that allows your projects to exchange pacts and verification results in an automated way. While you can use Pact without a Pact Broker, using one allows you to get the most out Pact. The Pact Broker is designed to operate within your test frameworks and CI system, and as such, has integration points in each of the Pact implementations to publish and retrieve contracts and results. The Pact Broker and its clients are open source tools (though you can get your own hosted instance of the Broker at pact.dius.com.au).
You now have a workflow where:
One of the complications introduced by the CDC workflow is that if you are using your contracts to drive out the requirements for your provider, the new expectations are usually added to the contract before the functionality has been implemented in the provider. If you publish a contract with new expectations in it, and you are using the workflow described above, a provider build will be kicked off, and it will (correctly) fail during the verification task. This, unsurprisingly, tends to make provider teams unhappy.
“Tagging” application versions in the broker allows you to introduce new expectations to a pact without breaking your provider builds. It’s an approach similar to git feature branches, where you can keep an unbreaking main line of development, while adding new, breaking interactions on the side.
To achieve this, when a pact is published, its associated pacticipant version is tagged with an identifier that will be used by the provider to differentiate between the “main line” safe pacts (eg. tagged “master”) and the potentially breaking pacts (eg. tagged “feat-new-foobar”)
To keep a green build in your provider’s CI, rather than verifying the latest overall pact, it should verify the pact for the latest version tagged with “master” in the CI.
If you use feature branches for your consumer development, it is recommended to tag the pacticipant version with the name of the branch. If you use feature toggles, the tag could be the name of the feature toggle.
It's all very well knowing that your consumer/provider are compatible with the head versions of each other, but if you want to be able to deploy your consumer and provider independently, you also need to be sure that your provider is compatible with the production version of your consumer.
As well as enabling a change workflow, tagging also allows you to ensure backwards compatibility between production and head versions of your applications by allowing the provider to verify the pact associated with the latest production version as well as the latest master version.
Before you deploy to a production environment, you need to know whether or not your app is compatible with the versions of the other apps that already exist in that environment. The old-fashioned way of managing these dependencies involved deploying sets of pre-tested applications together, creating a bottleneck and meaning that speedy development and testing on one application may be negated by slow development and testing on another.
The Pact way of managing these dependencies is to use the Pact Matrix - this is the matrix created when you create a table of all the consumer and provider versions that have been tested against each other using Pact. You can view the Pact Matrix for any pair of applications by going opening /matrix/provider/PROVIDER/consumer/CONSUMER in your Pact Broker.
One very important thing to note is that a verification is associated with the pact content itself, not to a specific consumer version. This means that if a pact does not change between publications, any previous verifications can automatically be applied to the new pact publication, effectively “pre-verifying” it. (For this reason, it is best not to use any random data in a pact, as this will cause the broker to consider it a new revision of the pact.) Linking a verification to the pact content rather than to the application version also means that we can do a “cartesian join” of pacts/verifications, resulting in many more “compatible” versions than would otherwise be the case.
The way you check if you are safe to deploy is to determine if there is a row in the matrix that contains the version of the application you’re about to deploy and the version of the other application that already exists in that environment. You can do this with the can-i-deploy tool, which will be described in more detail below.
The need for this check increases proportionately with the time that elapses between your pact test execution and your release. If you practice continuous deployment, and you go straight from a test build into a production deployment build, then you can be pretty sure that the version of the other application that is in production probably hasn’t changed in the meantime. If there is a considerable amount of time, however, it is best to do a compatibility check again just before deploying.
For example, if you run a pact verification on Monday that verifies the recommended “master” and “production” pacts, but you don’t deploy the provider artifact until Friday, the version of the consumer that is now in production may have changed.
Alternatively, a pact may have been verified by the “master” version of the provider, but that version of the provider may not yet have been deployed to production. The consumer cannot be deployed to production until the version of the provider that it depends on is in production (unless it has been specifically written to fail gracefully, which is actually the best approach, but is one that isn’t always followed).
The can-i-deploy tool is a CLI that has been written to query the Matrix to ensure that you are safe to deploy.
To really make sure you’ve dotted all the I’s and crossed all the T’s, there’s one last check you may need to do. As mentioned above, before you deploy your consumer to production, you need to know that the production provider has verified the pact from the consumer version you’re about to deploy. However, the production version of the provider may never have run a CI against your candidate pact, because your candidate consumer version may not have existed when the CI for that provider version was run.
To reach Pact Nirvana, the final step is to have a CI build that checks out the code for the production version of your provider, and have it verify the same pacts that the head version does. If you have followed the recommended strategy of tagging provider versions in the broker on deployment to production, you can use the broker to determine which version of the provider is currently in production. Make sure that the provider application version used when publishing your verification can be reverse engineered to a git reference.
pact-broker describe-version --pacticipant PACTICIPANT --latest prod
(describe-version has been released in version 1.14.0 of the pact_broker-client gem, but it has not been documented as of 29 Jan 18 - need to use pact-broker help describe-version)