Dynamic Configuration Management (DCM) is a methodology used to structure the configuration of compute workloads. Developers create workload specifications, describing everything their workloads need to run successfully. The specification is then used to dynamically create the configuration, to deploy the workload in a specific environment. With DCM, developers do not need to define or maintain any environment-specific configuration for their workloads.
For developers, DCM enables workload-centric development, meaning that workloads can be deployed with their resources and configuration to all environments, using a single workload specification.
For platform engineers, DCM helps define how and where to provision resources and workloads; this helps improve consistency and compliance in an organization. It also makes delivering a self-service Internal Developer Platform (IDP) much more accessible.
The alternative to DCM is a static, infrastructure-centric approach where a developer must maintain separate configuration files per environment and manage infrastructure autonomously, or outsource to other teams (TicketOps).
Advantages of Dynamic Configuration Management
Standardization of configuration
DCM enables workload-centric development, at the center of which sits the workload specification.
A workload specification is a structured file that describes everything a workload needs to run successfully without environment-specific configuration. Platform teams can use it to define how to deploy workloads from standard base configurations, which helps eliminate configuration drift.
Configuration drift happens when teams create new workloads by copying the structure of the configuration from an existing workload, and then customizing it to fit their needs. Over time, these customizations compound, resulting in subtle configuration differences between workloads. This then makes it challenging to perform bulk updates across workloads.
The environment-agnostic nature of workload specifications helps eliminate configuration drift and mitigate cognitive load. A platform team defines how to provision the environment-specific resources and configuration for the workload, and developer teams declare everything they need in a single workload specification that applies to all environments.
Separation of concerns
DCM enforces a strict separation between environment-specific and environment-agnostic configuration. Depending on the engineering organization, the same or different people may be responsible for managing shared resources such as databases, provisioning infrastructure, and running workloads. DCM does not have an opinion on how to organize the engineering team, but it does provide the structure for teams to work better together.
In regulated industries, it's vital to tightly control data access. This typically requires setting up a centralized data or database administrator (DBA) team. In a static approach, there would need to be some interaction between the developer and the DBA team if a workload needs a database. The developer would then have to take custody of the connection string—which would likely contain secret information in the form of credentials. This interaction is needed because otherwise, there is no way for the DBA team to know that a developer would need a new database.
In the DCM world, the workload specification declares the need for a database as a dependency. Automated tooling can notify the DBA team when a new database is needed. In more sophisticated cases, the whole process can be automated—provisioning databases at deployment time, as and when required by workloads.
Databases are also "environment-specific," and the DBA team can fully own them. The developer can configure the connection string in the workload with a placeholder. This way, the developer doesn't need to worry about taking custody of the connection string, and the DBA team can optimize their provisioning processes.
Better developer experience
Implementing a DCM approach reduces developer cognitive load. When paired with a dynamic Internal Developer Platform (IDP), developers only need to maintain the workload specification, since the IDP ensures that the correct resources are provisioned and wired to the workload.
DCM also improves promotion between environments. When promoting a workload with configuration changes, an engineer must figure out changes vs. configuration. This predicament occurs because most static configurations require individual configurations per environment. At best, this is a slow process requiring in-depth knowledge of how the workload is configured. At worst, it can introduce hard-to-debug errors involving environment cross-linking. DCM solves this issue with a strict separation between environment-specific and environment-agnostic configuration.
New possibilities of working
DCM provides most value in situations where tooling is used to manage the dynamic configuration that is generated. This opens up new possibilities such as:
- Provisioning (and cleaning up) infrastructure based on what a workload actually needs
- Setting up ephemeral environments, for example on every PR
- Centralized change of resources that are shared by workloads managed by different teams
Summary
DCM is firmly rooted in the platform engineering movement. The methodology has been adopted across industries from Silicon Valley tech giants to financial institutions in New York, to international retailers. Teams from startups to Fortune 100 companies are adopting the methodology and feeding that experience back into the community. This in turn is helping drive awareness, and enabling other organizations to make more informed decisions when considering DCM, and the business value it can bring.
