Lano is a leading HR software provider that enables any team to manage their contractors smoothly and without admin hiccups. It automates payments to freelancers all over the world and ensures companies stay fully compliant in every country where they contract workers and operate. If Humanitec is the magical glue to your DevOps setup, connecting all components of your infrastructure and automating Ops overhead, Lano provides a similar integration layer for your HR toolchain, automating away talent admin tasks.
As Covid hit last year and many teams shifted to a partially or fully remote setup, Lano’s customer base exploded. Though already an established business at the time, they were suddenly growing their team at double percentage points month over month. Great news of course, but that also brought about a whole new set of challenges, especially for their engineering infrastructure. Despite adding new engineers almost on a weekly basis, Markus, the CTO, was confident their setup could scale to accommodate the new workforce and the tools they requested. They were running their services on a solid infrastructure with Google Compute Engine (GCE) and CloudSQL instances.
An Ops nightmare
Soon however, cracks started to open in the day to day workflows of the development teams. Most of the new hires had issues with the deployment processes that were in place, either due to their lack of experience (in the case of more junior developers) or of familiarity with the evolving toolchain. More tools were being introduced to keep up with the overall growth and demands of the new developers, complicating this even further.
To maintain this increasingly complex setup, the Ops team had to resort to a growing number of custom scripts that functioned as an unstable glue for the CI/CD toolchain. In turn, this meant workflows easily broke if someone from the Ops team wasn’t constantly involved in maintaining them.
Things got so complicated that if an engineer simply wanted to deploy a new service version they had been working on to some test environment, they would have to touch more than 8 different tools and scripts. Because of these extra steps and the constant context switching, the end Developer Experience (DevEx) was inevitably deteriorating.
This extra cognitive load on developers not only meant they were frustrated and often had to wait hours or days to get an environment up and running to deploy their code, but it also highlighted the massive dependencies app development teams had on the Ops team. This, together with the accumulating maintenance overhead (understanding what was running where was increasingly challenging), led to the Ops department coming hugely under pressure and ultimately turning into a bottleneck for the engineering organization and the overall growth of the company.
It’s just an integration, they said
With Marketing and Sales posting another record quarter, it became clear the situation wasn’t sustainable. The entire engineering organization basically came to a halt when a Stripe integration (apparently not fun to QA) completely blocked Staging for about 2 weeks. As QA was done sequentially, there was no way for other teams to go around it and deploy new changes to Production.
This clearly showed what everyone already knew, the current CI/CD setup wasn’t going to scale the way they needed it to. It can be incredibly frustrating to know the direction you want to move into as an organization, but then finding yourself stuck in a dysfunctional infrastructure, due to the lack of specific knowledge required to scale.
To accomplish what they were asked to, Lano’s engineering needed everyone to be comfortable deploying with YAML, HELM and all the tools they had introduced, but that just wasn’t going to happen as the team was growing. However, relying on the Ops team for everything also proved to be a sure recipe for disaster.
Instead of trying to follow the “you build it, you run it” paradigm, which worked at a smaller team size, but just didn’t scale along with the growing team, the tech management took a step back and decided for a different approach.
Internal Developer Platform and merrily ever after
To relieve pressure on their Ops team, while at the same time improving the end DevEx of application developers, Lano implemented an Internal Developer Platform (IDP), built using Humanitec.
By introducing the IDP’s lightweight API layer into their setup, Lano’s Ops are now able to connect their whole infrastructure into one single point of truth, where they have an overview of everything that’s going on (what is deployed where and by who, at any time) and, more importantly, can set clear golden paths for the rest of the engineering team around their CI/CD workflows.
The IDP enables true developer self-service, where every engineer can now spin up an environment, provision resources like DBs and DNS to it, and deploy their containerized images. All completely independently from a UI or CLI, without anyone from the Ops them having to jump in to support.
With the need for specific knowledge around YAML, HELM and the rest of the CI/CD toolchain bound only to Ops, every team can now deploy autonomously and can run as many QA environments in parallel as they please.
Lano’s engineers are now running 4 different QA environments concurrently (and could run many more) and deploy 5x more frequently than they used to before the IDP was introduced. MTTR has dropped by 40%. More importantly, developers can focus on coding and shipping features, instead of getting stuck (and frustrated) in dealing with tools and scripts. The Ops team can support a much larger number of application developers without hiccups. Markus (the CTO) is happy and sleeps like a baby.
