The design phase of building an effective Internal Developer Platform (IDP) is hugely time-intensive. Not only do you need to figure out what essential components to include, you also have the daunting task of picking the right tech from a sprawling tool landscape ever-increasing in size. So often even knowing where to start is the first hurdle. But there’s good news. To help speed this process up and simplify things, we’ve now open sourced the reference architecture implementation codes for AWS, Azure and GCP-based platforms. This means anyone using one of the three major cloud providers can easily get started right away.
The best platform engineering initiatives always begin with a Minimum Viable Platform (MVP). This lets you move fast with a representative case that proves value to key stakeholders, is iterated based on feedback, then expands quickly to serve the needs of your developers and platform engineers.
This approach enables you to fast-track the IDP design process and reduce it to days, rather than months. And for further support, we launched this learning path showing you how to master your IDP with a reference architecture setup using Backstage, the Humanitec Platform Orchestrator, Score, and a cloud provider of your choice. It also walks you through the everyday tasks of developers and platform engineers, running applications and modifying the setup in hands-on sessions.
Before we get stuck into the what and the how of the new open sourced reference architectures, let’s take a look at why you even need a reference architecture to get started building your IDP.
The fastest way to design an IDP
IDP design and implementations vary. Until recently there was no proven, standard pattern that was scalable or repeatable for platform teams to follow. How an IDP turns out depends on what technologies you have in place, which you want to keep and get rid of, and what golden paths you want to design. In addition, your org size, developer workflows, and external influences like regulation will also shape the outcome of your IDP. However, no matter your requirements or how much technology changes, or even the multitude of tools available, there are standard patterns to be followed to help you build an IDP fast.
Using hundreds of real-life platform setups of all sizes and industries, a group of platform experts introduced their new reference architecture for IDPs on Amazon Web Services (AWS) and Google Cloud Platform (GCP) during PlatformCon 2023, which condensed the common patterns and best practices they had learned. This inspired us at Humanitec to create our own IDP reference architectures for AWS, Azure, and GCP-based setups, with the aim of helping organizations design, build, and deliver an enterprise-grade IDP in the fastest way possible.
Five key platform components
There are five main planes set out in the reference architecture that make up different areas of the platform. While the reference architecture’s aim is to give you a starting point, your existing estate should also be considered in each organizational context to incorporate the components already in place.
Developer Control Plane
This plane is the primary configuration layer and interaction point for the platform users. It harbors the following components:
- A version control system. GitHub is a prominent example, but this can be any system that contains two types of repositories:
- Application source code
- Platform source code, such as using Terraform
- Workload specifications. The reference architecture uses Score.
- A portal for developers to interact with. It can be the Humanitec Portal, Backstage or any other portal on the market.
Integration and Delivery Plane
This plane is about building and storing the image, creating app and infrastructure configs from the abstractions provided by the developers, and deploying the final state. It’s where the domains of developers and platform engineers meet.
This plane usually contains four different tools:
- A CI pipeline. It can be Github Actions or any CI tooling on the market.
- The image registry holding your container images. Again, this can be any registry on the market.
- An orchestrator, which in our example, is the Humanitec Platform Orchestrator.
- The CD system, which can be the Platform Orchestrator’s deployment pipeline capabilities, an external system triggered by the Orchestrator using a webhook, or a setup in tandem with GitOps operators like Argo CD.
Monitoring and Logging Plane
The integration of monitoring and logging systems varies greatly depending on the system. This plane, however, is not a focus of the reference architecture.
Security Plane
The Security Plane of the reference architecture is focused on the secrets management system. The secrets manager stores configuration information such as database passwords, API keys, or TLS certificates needed by an application at runtime. It allows the Platform Orchestrator to reference the secrets and inject them into the workloads dynamically.
The reference architecture sample implementations use the secrets store attached to the Humanitec SaaS system.
Resource Plane
This plane is where the actual infrastructure exists including clusters, databases, storage, or DNS services. The configuration of the resources is managed by the Platform Orchestrator, which dynamically creates app and infrastructure configurations with every deployment and creates, updates, or deletes dependent Resources as required.
When it comes to the assembly of these planes, integration is more intuitive for some layers than others. For example, platform teams are responsible for wiring individual plane components to each other, as well as binding one plane to the other. The raw end-to-end flows must also be tested and refined to ensure smooth integration and an optimal developer experience (DevEx).
How it works
So now we know what essential components make up an IDP. We also know that you can get to an MVP fast using the open source implementation code. Next let’s look at how it all works.
By using Score, our OSS workload specification, developers can describe how their apps fit together and define what resources they depend on. These requests are then matched by the Humanitec Platform Orchestrator against the baseline configs (which are set by the platform team). The Platform Orchestrator interprets the configs and resources required for a workload to run with every git-push, and creates app and infrastructure configs that are based on rules again defined by the platform team. It then executes them based on the following “Read”-”Match”-”Create”-”Deploy” pattern:
- Read: Interpret workload specification and context.
- Match: Identify the correct configuration baselines to create the application. configurations and identify what resources to resolve or create based on the matching context.
- Create: Create application configurations; if necessary, create (infrastructure) resources, fetch credentials and inject credentials as secrets.
- Deploy: Deploy the workload into the target environment wired up to its dependencies.
Get started building your MVP
So now your organization has a standard, proven, scalable, AND repeatable way of getting started with your MVP and expanding out to build an effective enterprise-grade IDP, fast. These new open source implementation codes for AWS, Azure, and GCP setups are exactly what an organization needs to get started with a platform engineering initiative based on experience and best practices Here’s how to get started:
- Create your Humanitec account.
- Check our reference architectures on AWS, GCP, and on Azure.
- Expand your platform knowledge and explore our learning path.
- Build your MVP.
We’ve helped dozens of organizations of all sizes and industries build successful MVPs that demonstrate value, and quickly begin to expand towards an effective enterprise-grade IDP. Join our MVP program to learn more and get started.
