Checkout Charlie

The publisher of some of Europe’s largest affiliate marketing portals wasn’t delivering features fast enough. Rather than focussing on code, teams were distracted by complex deployment setups, Kubernetes and dependencies on Ops.

About Checkout Charlie

Checkout Charlie is one of the leading publishers of affiliate marketing sites active across Europe. Headquartered in Berlin, their engineering team has to deliver features to all their sites at neck breaking speed.

Customer is currently looking for developers to join them!

Infrastructure and tooling setup

When Checkout Charlie first engaged with Humanitec they were running a multi-cloud setup, leveraging EKS for some services, Heroku for others. As they were making their shift towards a platform approach, the setup was consolidated to AWS using managed Kubernetes (EKS), Jenkins and GithubActions for CI, Route53 as DNS, S3 for file storage, a variety of databases (AWS RDS, Elastic, Redis) and RabbitMQ for messaging.

Key challenges

Checkout Charlies engineering team is up to speed if it comes to building a setup that serves the needs of developers optimally. When the business scaled engineering had to adapt to the changed requirements in terms of load and global distribution. 

They reached the limits of Heroku’s scalability and decided to consolidate their entire fleet of applications on EKS. However, running Kubernetes at scale overwhelmed the development teams. Requesting resources such as databases, environments or DNS blocked them from delivering.

  • Prevent problems before they hit: engineering management forecasted correctly that with growing complexity they had to invest into the simplification of the deployment setup to keep a true “you build it you run it” culture. 
  • High lead time: while the business kept growing, the team had to fix broken deployment flows and firefight problems.
  • Blocked developers: in the new setup backend engineers waited up to one week for a fresh environment
“Our setup required a huge amount of hand-holding, lots of manual steps and was simply costly. It blocked developers.”
Frederico Infanti, Software Architect

Key improvements

By building their Internal Developer Platform with Humanitec, Checkout Charlie simplified their deployment process to a simple “git-push”. Developers are now able to self-serve the tech they need, operate Kubernetes without fiddling in unstructured scripts and deliver fast.

  • Unlock ops: frustrating, repetitive ticket ops workflows are automated.
  • Unblock developers: engineers can self-serve deployments without worrying about the underlying infrastructure.
  • Go fast: teams can spin up fully provisioned environments on demand.
“I showed a backend developer how to deploy with Humanitec, he had tears in his eyes.”
Vincent Stoltzenberg, Head of Development

Humanitec erased bottlenecks and dependencies, reduced pressure on operations, simplified maintenance and reduced waiting times. Change failure rate dropped significantly.

reduction in manual tasks

by automating requests from developers.

reduction in waiting times

by providing what developers need in real-time.

lower change failure rate

by enforcing parameterized env variables.

Lower lead time

by allowing developers to go fast without breaking things.

Technical deep dive

Infrastructure orchestration before and with Humanitec

Before building their Internal Developer Platform (IDP) with Humanitec, Checkout Charlie’s setup was static. If a developer required a new infrastructure component, they had to request that from a central Ops team or stand it up themselves. This approach led to bottlenecks and frustrated both Ops and developers. After building their IDP with Humanitec, Checkout Charlie’s developers can now independently apply changes to configurations without fiddling with unstructured scripts. They are able to self-serve any tech they require, such as fully provisioned environments, databases, or simply debug by pulling logs and error messages.

“I would say we knew what we were doing before Humanitec, but this is a new level of DevOps.”
Vincent Stoltzenberg, Head of Development

App config management

Before Humanitec the team used kubectl to manage application configurations. This approach didn’t allow them to scale. Developers had a hard time onboarding to the CLI, changes weren’t tracked, roll-backs were time-consuming. They knew they had to enable configuration as code.

Humanitec allowed them to achieve this without overwhelming teams further with Helm charts. Ops can now set baseline templates that contain any default they want to enforce. Developers can apply changes to these templates through the CLI or UI. At deployment time, the platform API creates a fresh set of manifests including the environment specific elements (e.g. databases, file storage, DNS), saves them to the repo in Github and executes them against the EKS API. Manifests are versioned, increasing visibility and allowing for easy rollbacks or diffs.

Ops still left the option for developers to script everything themselves using Helm. 97% of teams followed the “Golden Path”.

“”I’m in love” puts it well.”
Vincent Stoltzenberg, Head of Development

Final setup

Jenkins is still used for CI, although the team is slowly migrating to GithubActions. CI is wired up with the Platform API to notify it once the image is built. The image registry is wired up to the Platform API. The API deals with RBAC, creates application configurations per deployment and calls the correct open source driver at the correct request. Developers self-serve deployments, resources, logs and more through the developer self-service UI or CLI.

Timeline and evaluation

  • POC: 5 weeks
  • Evaluated against Heroku
  • Migration: 6 weeks
  • Onboarding per new developer: 30 minutes