In an ambitious undertaking, a government institution in Europe embarked on a journey to modernize its development and deployment processes for high-security applications. With the dual need for air-gapped, secure on-prem environments and the agility offered by cloud platforms, the entity faced a complex challenge. Their mission-critical systems, requiring the highest levels of security, demanded an innovative solution.
Key challenges
The government institution's primary challenges included:
- High-security needs: Essential government systems require air-gapped environments, running OpenShift on Dell/EMC (on-prem) devoid of internet connectivity, to safeguard against cyber threats.
- Hybrid environment integration: There was a strong desire to leverage on-prem solutions of Hyperscalers and their on-prem solutions such as Azure Stack, AWS Outposts, or GCP Sovereign Cloud without the complexities typical of multi/hybrid setups.
- Inefficient development: Existing on-premise high-security setups were sluggish and unscalable for dev environments, putting high cognitive load on developers and security teams, and hindering rapid development and delivery.
- Compliance demand: Established policies, governance, and security processes were needed to ensure the compliance of the operating environment can be applied at all times.
Solution
In response, the entity collaborated with our partner Bechtle Competence Center AVS to construct an Internal Developer Platform (IDP). This platform was meticulously designed to provide a consistent and seamless developer experience across major Hyperscalers, Bechtle Datacenter, and the entity's own data center. The IDP significantly enhanced secure, high-speed delivery capabilities into air-gapped environments, effectively meeting organizational requirements and circumstances.
Platform architecture
Working with such an IDP, developers only need to declare the required type of resources the workload depends on in an open-source workload specification called Score, without specifying dependencies deeper.
Humanitec's Platform Orchestrator then interprets these workload specifications against environment-specific resource definitions set by the platform engineering team. For each deployment, the Platform Orchestrator generates fresh app and infra configuration files in Kubernetes custom resource form, which are stored in a GitLab repository.
From here, the environment-specific deployment process is slightly different. Depending on the network connectivity, there are two cases:
- Deployment into a Kubernetes distribution at a hyperscaler or Bechtle data center. Network connectivity is given. A GitOps operator – like ArgoCD or FluxCD – can observe the GitLab instance and pull in changes directly.
- The environment is air-gapped and there is no network connectivity. The Git commit, representing the deployment, needs to be exported on one side and imported on the other, using standard Git functionality. This way, deployment packaging and execution can be physically separated.
Now the standard GitOps deployment flow is executed. ArgoCD as the GitOps operator pulls in the changeset into the cluster, creating the custom resources, which triggers controllers responsible for handling them.
The Humanitec Operator will now take over and carry out the following deployment steps like generating a resource graph from the custom resources, using Humanitec Drivers to create the respective manifests and observing how all custom resources are transformed into real objects like deployment sets, workloads and infrastructure.
When this process is complete, the actual deployment is also complete. The reason for this complex choreography lies in the separation of concerns.
Without the separation of the process into the different system parts, it would be impossible to run the deployment process distributed into different physical locations. Also, it would be impossible to separate the development tooling from the deployment/runtime tooling, which would make reducing the attack surface (a prerequisite for high-security environments) unachievable.
- Significantly accelerated feature deployment: Developers can prioritize creative innovation over environmental complexities, facilitated by streamlined, automated deployment and release processes that efficiently manage infrastructure complexities.
- Marked improvement in operational efficiency and productivity, coupled with an increase in the organizations value: infrastructure teams are empowered to focus on delivering high-quality services. This is achieved through well-defined interfaces that simplify the packaging of services into autonomously deployable units, enhancing the deployment process's efficiency.
- Elevated automation levels that bolster security measures and substantially reduce manual operations: Security teams can focus on strategic system design and the rigorous verification of service delivery templates. This approach ensures that services provided through automated processes in verified environments adhere to the highest security standards, eliminating the need for repetitive individual service reviews.
- Effective threat detection and incident response: All inventory and security-relevant data can now be pushed to the customer's Security Information and Event Management (SIEM) API as a defined step within the deployment pipeline. This ensures all changes applied to the runtime estate of the customer are immediately reflected in near real-time in the SIEMs understanding of the threat landscape.
The outcome
The platform enhances the Software Development Life Cycle (SDLC) of the government institution, seamlessly integrating high-security protocols from the outset across both the supply and demand sides. It ensures air-gapped deployments, traceability, and compliance are effortlessly managed, negating typical challenges. Moreover, it fosters meaningful collaboration among all teams involved, significantly reducing manual tasks and streamlining daily operations, embodying a transformative approach to secure software development and deployment.
- Significantly accelerated feature deployment: Developers can prioritize creative innovation over environmental complexities, facilitated by streamlined, automated deployment and release processes that efficiently manage infrastructure complexities.
- Marked improvement in operational efficiency and productivity, coupled with an increase in the organizations value: infrastructure teams are empowered to focus on delivering high-quality services. This is achieved through well-defined interfaces that simplify the packaging of services into autonomously deployable units, enhancing the deployment process's efficiency.
- Elevated automation levels that bolster security measures and substantially reduce manual operations: Security teams can focus on strategic system design and the rigorous verification of service delivery templates. This approach ensures that services provided through automated processes in verified environments adhere to the highest security standards, eliminating the need for repetitive individual service reviews.
- Effective threat detection and incident response: All inventory and security-relevant data can now be pushed to the customer's Security Information and Event Management (SIEM) API as a defined step within the deployment pipeline. This ensures all changes applied to the runtime estate of the customer are immediately reflected in near real-time in the SIEMs understanding of the threat landscape.
The outcomeThe platform enhances the Software Development Life Cycle (SDLC) of the government institution, seamlessly integrating high-security protocols from the outset across both the supply and demand sides. It ensures air-gapped deployments, traceability, and compliance are effortlessly managed, negating typical challenges. Moreover, it fosters meaningful collaboration among all teams involved, significantly reducing manual tasks and streamlining daily operations, embodying a transformative approach to secure software development and deployment.
Timeline and evaluation
- Target architecture design: 4 weeks
- Platform MVP delivered: 8 weeks
- Full integration: 5 months
- Entire product ported after 9 months
- Onboarding a new developer: 40 minutes
