By streamlining application configurations and infrastructure orchestration, an IDP frees up Ops from handling repetitive tasks and eliminates Ops-Devs inefficiencies and bottlenecks. Ops design golden paths that developers follow according to the defined RBAC model. Ops centrally set baseline configuration templates, roles and permissions; developers self-serve the tech they need without ever having to wait on Ops.The three key components of an IDP are a core API layer, open source drivers that the API uses to provision the underlying infrastructure and a self-service UI and CLI.
Fences to platforms
Remember when you could simply throw your code over the proverbial fence and wash your hands of it? How great was that. Some overworked Ops (or System Admin, as they were called), sitting somewhere in the dark basement of your company, would have to deal with it later. They had to make sure your and everyone else’s code could in fact run on servers they had to physically purchase and install. If something broke or a server went down, they were on call for it. Not your problem. True developer freedom. Or was it?
This is what software development and deployment effectively looked like throughout the 90s and early 00s. Then, half way through the decade, things started to change. Virtually every company’s value proposition across all industries became digitized. Whether selling direct to consumer or B2B, users increasingly expected a seamless digital experience. Suddenly, infrastructure had to be deployed globally, across multiple regions and with countless integrations. Amazon Web Services (AWS) was released to the public in 2006 and Cloud Native started gathering momentum. Microservice architectures were the new cool kid on the block. Werner Vogels, the iconic CTO of AWS, popularized the “you build it, you run it” paradigm. It turned out, developer freedom wasn’t about throwing stuff over the fence and forgetting about it, it was actually the opposite. Engineers were now required to understand their setup so they could run their own code and apps, independently. True DevOps, true freedom.
That sounded great in theory, but in practice things started getting quite complex, quite fast. With the infrastructure footprint of engineering organizations doubling every year and an increasingly convoluted tooling landscape, developers now had to gain a full understanding of new technologies or concepts such as Kubernetes or Infrastructure as Code (IaC), just to deploy their code. They had to touch dozens of different tools or scripts only to add a simple database and test a code change they did. True freedom didn’t seem that great anymore.
These challenges were especially felt at fast growing tech companies. When you have to onboard hundreds of new developers per month to an ever-evolving infrastructure and a setup that gets more advanced by the week, requiring everyone to be familiar with kubectl or Terraform scripts makes your hiring plans very, very complicated. That is why top performing engineering teams decided to recentralize Ops functions in Internal Platform teams, who were responsible for building Internal Developer Platforms (IDPs).
Developer self-service with Internal Developer Platforms
Lead tech organizations understood that the key to scale in a world of ever-expanding internal tooling and infrastructures was to design a self-service layer, so their engineers could independently provision the tech and tools they needed to run their applications. This self-service layer has been accurately described by Evan Bottcher (Thoughtworks) as a digital platform that is
“a foundation of self-service APIs, tools, services, knowledge and support, which are arranged as a compelling internal product. Autonomous delivery teams can make use of the platform to deliver product features at a higher pace, with reduced coordination.”
By minimizing complexity and cognitive load, such digital platforms, or Internal Developer Platforms, create “a curated experience for engineers” (Matthew Skelton, co-author of Team Topologies), who are the end users and customers of the internal platform team, responsible for designing and orchestrating the IDP.
Jason Warner (CTO of GitHub) told us that building an IDP was not only a fundamental shift in the way GitHub operated, but an essential element of their aggressive scaling. Aaron Erickson, who built the IDP serving over 8,000 engineers at Salesforce, “cannot understate the importance of introducing self-service capabilities in an engineering organization” and describes the implementation of the IDP as a clear inflection point in the growth trajectory of Salesforce’s engineering setup.
By enabling developers to self-serve what they need to run their own apps and services, IDPs allowed early adopters to massively outperform competition and grant their engineers true developer freedom, in the form of developer self-service.
The 2020 Puppet State of DevOps Report showed a strong correlation between the use of internal platforms and high DevOps evolution.
These results were further confirmed in the latest 2021 report, which shows how IDPs and internal platform teams are the “key to success at scale” and the best way to “lift your organization out of the midfield and into the top performers”.
The impact of introducing an IDP is quite impressive. According to the Accelerate State of DevOps report (2019), top performing engineering teams that operate internal platforms deploy over 200 times more frequently than the average organization. Their lead time from commit to deployment is 106x, while managing to lower both their MTTR (over 2000x faster) and change failure rate (7x better).
The trend is clear. Developer self-service and IDPs are the future of software development, as well as the way we think about and implement DevOps. They enable true “you build it, you run it” and free both Ops and developers from having to deal with menial implementation details. Self-service and IDP adoption are already trickling down from more advanced engineering organizations to the rest of the industry. But what exactly is an Internal Developer Platform and should your team build one? In the following sections we’ll walk through the fundamental components of IDPs and precisely how they improve on the current ways of dealing with application configurations, infrastructure orchestration, access control and other key areas of your development and deployment workflows.
Life before an Internal Developer Platform
Before getting into the details of how an Internal Developer Platform is built, it is important to briefly sum up the main challenges that IDPs solve and how these are addressed today in most engineering setups. Zooming out and looking at the deployment workflow of the average development team, this is what it looks like for an engineer to deploy their code:
While both source code management and the build step have been solved and have established ecosystems with many tools providing very similar solutions, the deployment bit still lacks standardization and clear answers. To actually get an app or set of services up and running, you need three key things: built code, app configurations and the required infrastructure (e.g. database, DNS, etc.). Built code is fine, but what about app config management and infrastructure orchestration? Every team seems to have a different deployment setup which expects developers to touch multiple config files (e.g. YAML, Helm) or IaC scripts (Terraform, Pulumi) to do a simple deployment and provision basic infrastructure. That is precisely what creates a zoo of config files and all sorts of requests for Ops, leading to overheads and bottlenecks, i.e. frustration and waiting times for everyone involved.
Represented graphically, this is what application configuration management looks like in practice for most teams. Everybody fiddles around in unstructured scripts that are hard to maintain, version and understand. Onboarding to these setups takes ages and they are always going to be error-prone at any scale.
The situation is quite similar for infrastructure orchestration. The Ops team is a bottleneck and developers wait for environments and infrastructure. Provisioning of infrastructure is done manually or using IaC that is hard to govern and maintain.
This not only makes your setup an Ops nightmare to maintain today, but it also means it will become harder and harder to scale efficiently as your team and infrastructure grow.
One possible solution that some engineering teams implement is hardcoding every step of the deployment process into a pipeline. That is an improvement in terms of Developer Experience, as it provides an extra layer of abstraction on top of the raw bash scripts and config files, removing unnecessary cognitive load from developers. The issue with this approach however is that your setup becomes extremely static, with rules very much “set in stone”. Every new use case will basically require a whole other pipeline to change the design and composition of the app you want to deploy. This in turn leads once again to lots of requests for Ops and your bottleneck is back. Unsurprisingly, static and hardcoded approaches become increasingly harder to maintain and scale with, as your needs evolve.
Life with an Internal Developer Platform
Unlike these static approaches, an IDP gives developers the freedom to design apps as they like, provisioning the infrastructure they need, within a golden path that is defined by a central Ops team and enforced through a clear Role Based Access Control (RBAC) model.
Teams that leverage Internal Developer Platforms can conveniently streamline application configurations and infrastructure orchestration to let engineers dynamically interact with their setups. It becomes extremely easy to adapt to new use cases or developers’ requirements.
Specifically, Ops can now simply set baseline configuration templates (plain YAMLs) with default configurations for every environment and workload. Developers apply changes to this baseline on their own through a UI, CLI or API, according to their role and permissions. At deployment time, the IDP takes care of generating new manifests that are executed against clusters and infrastructure, and stored in a target repository.
Similarly, the IDP enables developers to self-serve the infrastructure they need to run their apps, fully independently from Ops. In the self-service UI and CLI, they can easily specify what tech they require (a database, Ingress and DNS, file storage, etc.). The IDP makes sure the requested infrastructure is provisioned by open source drivers and serves it back to the developers. This approach also allows Ops to handily integrate the IDP with any pre-existing IaC setup, such as Terraform, Pulumi or any other.
The key components of an IDP
An Internal Developer Platform therefore streamlines application configurations and infrastructure orchestration, combining them with an RBAC permission structure and a self-service layer for developers (through a UI or CLI). The three key components of an IDP are the core lightweight API layer, the open source drivers that communicate with the underlying infrastructure and the self-service UI and CLI.
The core API layer sits after the CI/build setup and is notified about new builds by an HTTP request at the end of the pipeline. It’s also wired up to container registries to consume images for deployments. It is the API that is responsible for serving infrastructure (DBs, DNS, File Storage) to developers by using open source drivers. The API is also what applies the changes that developers specify in the UI or CLI to create new manifests, which it then deploys onto the different clusters via the Kubernetes API. Developers can interact with the core API directly or through the self-service UI or CLI.
Drivers are used by the core API at deployment time. Resources are provisioned based on dependencies defined by application developers. Which driver is used and how it is configured is defined centrally by the Ops team and enforced throughout the organization by the IDP. As an example use case, a developer defines a dependency on a Postgres database on a particular workload. At deployment time, the API identifies which driver to use to provision the database. The driver creates the database, returning connection information such as db-name, host, port, username and password. These are then injected into the workload.
It’s important to point out that the standard developer workflow does not change with an IDP. Developers apply changes to their code, do a git-push and the respective environment is updated by the platform automation in the background. In most cases, developers won’t likely have to touch the infrastructure nor the IDP. However, if developers do want to change configs, add a database, spin up a new environment, debug or see logs, they can do so independently through the UI and CLI self-service layer.
Self-service and beyond
Internal Developer Platforms are the best way to enable developer self-service. They allow Ops teams and platform engineers to design golden paths for the rest of the engineering organization to follow. Within these predefined paths, developers can self-serve on-demand environments, provision any infrastructure, add images from a CI pipeline or container registry and deploy. If they so choose, developers can also go off path and specify their own configurations that they can directly apply using the core IDP API.
After applying the desired configuration changes and deploying, engineers can easily manipulate those deployments by outputting a diff between two manifests or rolling back throughout the versioned deployment history. Within the context of the permissions set through the RBAC model, developers truly achieve full freedom, from code to production.
The benefits of such flexibility are felt across the entire engineering organization. Teams that have implemented an IDP with Humanitec see a 95% decrease in waiting times and a virtual elimination of bottlenecks between the Ops and the dev teams. They deploy 4x more frequently than they used to, without compromising on quality, as Mean Time To Recovery (MTTR) also drops by 60% on average.
For most, this is only the beginning. Using an Internal Developer Platform allows engineering teams to build custom integrations with tooling, system or infrastructure they want to use or add to their setup as they grow. For instance, an IDP can be easily wired up to an APM or security setup, complemented with service catalog functionality or connected to admin systems like Jira and Slack to create all sorts of automations.