A few years back I started to specialize in microservice architecture based applications, as I saw this as a possible game changer for testing. In this article, I’d like to highlight some of the challenges and opportunities when testing microservices.
From structuring a software architecture into microservices you would expect more clarity compared to a software monolith. But in real life, things might not be that clear.
Martin Fowler describes microservices as follows:
“…In terms of size, there are no hard and fast rules. Commonly, microservices are of the order of hundreds of lines but can be tens or thousands depending on the responsibility they encapsulate. A good, albeit non-specific, rule of thumb is as small as possible but as big as necessary to represent the domain concept they own.” (Martin Fowler)
In addition to the fuzzy rules around microservices, software is constantly being iterated upon and enriched with additional features, so what might have been a good decision in the original architecture diagram may have turned into a monolith by itself and needs to be broken down further. Also, microservice architectures only make sense with a clear and concise data and domain model on top that describes the business logic and interactions of each service. This provides constant challenges in continuous testing as requirements often change.
But microservices provide also great opportunities for testers when understood right. Let’s look at the 8 main parameters to successful testing of microservices.
Documentation
As a tester, I’m happy about good documentation. But usually, it’s outdated by the time I test a change or bugfix. To be able to understand the software architecture you need a good description of its functionality and interfaces. At Humanitec, all our backend services have an auto-generated swagger documentation implemented, describing all exposed endpoints and its usage. Since it’s being auto-generated from the code, it’s never outdated. Having strong documentation goes a long way not only for testers but also anyone else involved in the development.
Test Strategy
With a test strategy in place, you want to make sure you understand what needs to be tested by gathering requirements and feedback from your stakeholders. In general, make sure you first test everything which can be verified fast and easy so when it comes to the more complex test scenarios you already have your bases covered. As shown in the test pyramid below, the more complex tests have a higher chance of passing after the less complex tests have already covered the basics.
Test pyramid
Defining and customizing a testing pyramid that aligns with the product requirements may help in communicating your testing strategy.
Being a QA Engineer and/or a Test Manager, I might be only accountable for end-to-end testing (automated and manual). Without having a strong foundation of lower level testing types, my end-to-end tests will break early and therefore won’t cover the test scenario they were built for. Highlighting to developers, DevOps engineers and product owners what’s important for successful quality assurance will provide you the support you need.
Consumer-Driven Contract Testing
One concept I would like to highlight in particular is Consumer-Driven Contract Testing. After verifying that each microservice performs as expected with unit and component tests, you need to verify the integration between services. At Humanitec, we use REST APIs as a communication layer between services.
In Consumer Driven Contract Testing you ensure that each API consumer (e.g. a frontend service) documents the endpoints which it consumes from an API provider as well as the expected responses. These API consumer contracts are then stored centrally or with the API provider. When an API provider then updates any of its endpoints or responses it checks if it’s violating any of the stored contracts. Enabling CDC-Testing with tools like pact.io or Postman making it possible to run integration tests on each update of an API provider without having to have a full environment running.
End-to-end Testing
Further up the pyramid, I like to highlight the importance of building a solid foundation of API tests before covering functionality via UI tests. In my opinion, there are two approaches to building API end-to-end tests:
- Coming from the API documentation ensuring coverage of all available endpoints and operations (e.g. GET/POST/PATCH/PUT/DELETE). This way you’ll get a good overview and understanding of how the APIs are working and how they interact.
- Analyzing requests from the frontend by recording the API calls and putting them into direct context with the user flow. This way you can catch specific ways the frontend is using the APIs, ensuring the validity of the test.
When building UI tests, you should take into account previous test coverage by unit and API tests. The validations in your UI tests should be limited to business critical functionality and visuals which have not been covered on other test levels. This should make your rather simple UI tests more stable.
I became a big fan of the Robot Framework as it provides API and UI tests out of the same framework by utilizing the python requests library in robotframework-requests as well as Selenium in the robotframework-seleniumlibrary. Both can be run in combination from the same test suite providing a single html test report.
As a side note, I really enjoyed being at the Robot Framework Conference (RoboCon2019) in January this year in Helsinki. You can watch my talk about “Test metrics in Kibana” here.
Continuous Testing and Continuous Deployment
A well defined Continuous Integration pipeline is crucial to allow Continuous Testing and Continuous Deployment of single microservices. Based on an update of a single microservice, the CI pipeline needs to ensure running all tests applicable for each level of the above-mentioned test pyramid. Test suites for end-to-end tests need to be able to run specific scenarios based on the functionality of the microservice under test. By using the Robot Framework for end-to-end test suites we are able to use its tagging functionality to achieve this.
Running Continuous Deployments of single microservices to production enables us to develop, test and deploy in small and fast iterations. This ensures testability as well as keeping the risk of bugs low. Continuous Deployments into production are also a strong driver for building a solid automation strategy.
Testing in Production
In a Kubernetes environment with a service mesh like Istio, it is possible to test a new instance of a microservice in integration with the production environment. Istio allows routing to the instance of the microservice under testing based on conditional routing rules as well as traffic mirroring. This allows for full integration tests with production code and data.
Monitoring
After you deploy code changes to production, it’s important that you understand that testing does not stop there. You can see monitoring as a form of post-production testing. A solid monitoring setup will enable you to catch any unexpected behavior of your production system as soon as possible. At Humanitec, we are using Prometheus and Grafana to see for example any increase of http 4xx or 5xx responses.
Summary
Quality Assurance has never been just testing and bug reporting. It’s an overarching process with many stakeholders and dependencies. Modern software architectures following the microservice principles provide good opportunities to establish efficient quality assurance processes. Compared to classic testing approaches with centralized QA teams, microservice architectures make it easier for multi-disciplinary teams to build more efficient QA processes with less resources.
Other articles I recommend for further reading on this topic:
https://martinfowler.com/articles/microservice-testing/
https://labs.spotify.com/2018/01/11/testing-of-microservices/
https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16
