Hey,
The biggest misconception in platform engineering is that it’s mostly about providing visibility. This is a secondary focus at best. It feels tempting because providing visibility seems like a quick fix and it means your managers can “touch” something. But this rarely provides a return on investment. There are several terrific articles out there like “build your house first, not the front door” or “if you put a pane of glass on a pile of sh’t you observe a pile of sh’it”. So I’ll leave you with that.
This gets us to the question, “what delivers ROI?” In my opinion, you should spend time figuring out how to structure your repositories. In doing so you’ll be able to build a system design that’s easy for developers to operate, and easy for operations to maintain and secure.
When I design platforms I always opt for the method of dynamic configuration management (DCM) and design my repo structure accordingly. To reap: DCM is the approach of dynamically generating app and infrastructure configs with every release. Developers describe their workload and its dependencies in abstract terms using a workload specification like Score. This would for instance express “my workload depends on a DB of type Postgres” and send it through the CI-Pipeline with every git-push. A Platform Orchestrator takes the abstract request and the context (deployment to an environment of type staging) and matches general resource definitions to create or update the DB of type Postgres. It then generates the final config and wires everything up.
The advantage for the developer is that the experience is very smooth and easy. With just one file, the developer describes what they need and pushes it. The platform takes care of the rest. This is a game-changer for security and operations teams. Rather than having to maintain 200 slightly different configs of the same resource, they simply have to maintain one baseline config set. Everything that’s “global” is a golden path. It’s basically supported by the platform team.
Let’s look at how such a repo structure could look in a well designed platform:
On the workload specification level we recognize that the baseline config set is stripped of all app and infrastructure configs. So no Terraform files or Helm charts are needed to provision resources on a workload level:
- Workload source code
- Workload Spec (Score)
- Docker file
- Pipeline configs (assuming you’re not managing them globally)
On the platform level the situation looks as follows:
- Base resource configs/ IaC: Terraform, Crossplane and friends that generally configure resources. You might for instance have 10 different supported Terraform modules for DBs of type Postgres.
- Resource Definitions: Tell the Platform Orchestrator what, when, and how to use the base resource configs. For instance: If the abstract request of the specification indicates that the workload requires a resource type = postgres and the env-type = staging, then forward the following input to a driver and execute this exact Terraform module.
- Workload Profiles: Think of them like base Helm charts. The Platform Orchestrator will use them to create the app-configs.
- Automations/Compliance: Configs for different toolchain components of the platform. In a good platform the platform itself is code and treated as a product.
A few observations on this design:
- Tougher RBAC settings lead to more standardization. Meaning that if your developers can see how a resource gets configured in production but not change it, all your prod resources are configured exactly the same way.
- A “golden path” in this design is the procedure of developers describing what they need in abstract terms and the Platform Orchestrator automatically creating/updating the correct resource.
- Going “off the golden path” literally means forking the resource definition/IaC because you want to tweak something that’s not generally relevant. But only for your specific use-case. And should your platform team observe a lot of specific custom definitions, it can pull them in and offer them globally with an SLA. You’ve essentially created a new golden path!
- This enables you to genuinely “treat your platform as a product”. So rather than spending ages cutting resources you think you may need in the future, you observe what your customers request, and how they behave and react by supporting things globally.
One final observation for why this repo structure is great if you’re a large scale enterprise. I’ve recently worked with a team of 15,000 developers spread across five country organizations, and one global platform team. They wanted to pair global standards with localization. They could just use the platform repo structure as outlined above and the country orgs simply forked the global platform.
Join my webinar “How to design your repository structure for successful platforming” for a deeper dive into sustainable platforming and structuring git repositories the right way.
Look forward to seeing you then!
Kaspar
