Internal Developer Platforms (IDPs), enabling developer self-service and automating DevOps overhead, are becoming a regular practice for companies that value velocity and developer experience, as the latest Puppet report highlights. With cloud-based systems and tooling growing more complex, IDPs have become essential for engineering teams to abstract away the intricacies of their underlying infrastructure from their app developers. These platforms are instrumental in reducing the cognitive load on developers, while also releasing pressure on Ops teams by automating their setup, making it easy to maintain and scale with.
As leading tech companies like Spotify and GitHub are popularizing this approach to internal tooling, IDPs are becoming mainstream and engineering organizations are naturally debating whether they should build their own in-house solution or buy a commercial offering.
We get the “build vs buy” question frequently, so we wrote the following resource for teams that are ready to adopt an Internal Developer Platform and want guidance on the best strategy.
To build or not to build
With enterprise software spending projected to grow over 500 billion dollars in 2021, it appears that many companies are finding it more convenient to buy commercial software rather than building it themselves. And while buying may not be the right answer for your specific business needs, here’s a look at four key factors that should guide your approach to answering the build vs buy question.
Cost
Companies lean towards the most cost-effective solution for their business. Internal tooling projects have the great advantage of letting you decide how much to spend on development. Except, IT projects have a very bad tendency to take much longer than forecast and end up costing a lot more.
Gallup reports that 20% of internal IT projects have on average a schedule overrun of almost 70% and a cost overrun of 200%, once you factor in the cost of the initial build, internal support, bug fixes, upgrades and keeping up with market trends.
While purchased software can have a high upfront cost or a substantial monthly price, this is a known expense for a product you can use immediately. For many CTOs and Infrastructure managers, this fixed cost vastly outweighs the unpredictability associated with building internal systems in-house.
Control
The biggest appeal of building software in-house is the notion that it can be molded to meet the company’s specific requirements as they arise. What this also means however, is that you will always be reliant on the developers building it.
It’s unsettlingly common for companies to be left with unusable code bases created by developers who no longer work at the company, adding an ongoing maintenance cost of hiring new developers to rebuild and iterate on the codebase.
As a general recommendation by the Adobe Enterprise team, if the requirement fit of commercial software is 60% or more, buy is the way to go. You may not have full control over the product roadmap, but many vendors will work closely with customers when designing or improving their products.
Connectivity
Each company has its own ecosystem of applications and services that need to be compatible with systems beyond the company, such as external vendors and technology providers. Building your own solution can help ensure that these connections are made, as long as your team is capable of undertaking the extensive process of stitching together multiple, smaller solutions.
Chances are that a more advanced and comprehensive solution already exists on the market. Some commercial software solutions can even act as an extension of your existing ecosystem, rather than entirely overhaul the architecture your Ops team has so arduously built.
Maintenance
Building software means not having to rely on an external team to develop new features or roll out upgrades. But most internal teams do not have the same availability or expertise as an external team when dealing with complex architectures, typical of enterprise companies. Even seemingly trivial things such as keeping documentation for an internal platform in order and up to date, can get pretty hard in reality. Not to mention maintaining the different integrations, API changes and everything that comes with a growing toolchain and provider ecosystem.
From multi-tenancy to data compliance, an experienced team armed with best practices and the latest market insights will typically be better equipped to handle maintenance for enterprise-scale software.
Building an Internal Developer Platform from scratch
Alright, you weighed out all key factors and decided to go on building your IDP, what does that look like? The majority of internal tooling starts off as a fork from open source to allow for faster prototyping and a quicker path to a minimal viable product. The initial goal is often to address basic concerns, such as simple deployment or CI/CD automations, and gradually add more advanced features like environment management and infrastructure orchestration.
Building your own Internal Developer Platform from scratch means you can customize it to your infrastructure’s exact needs so that it deeply and naturally integrates with the rest of your toolchain. That’s a big plus for an IDP, which is really the fundamental glue to your setup.
It also means you possibly have a shorter feedback cycle between development and production. You get full control over the product roadmap and can ship all features your app development teams request, no need to worry about being deprioritized over another larger customer.
There’s some security benefit as well. Everything can be kept on-prem, with no reliance on the outside world, if that’s what your compliance team requires. This also potentially makes it easier to monitor and control traffic.
Buying an Internal Developer Platform (or building it like a pro)
Building an Internal Developer Platform using a plug and play framework like Humanitec means you’re able to get up and running in days instead of months, with zero in-house development. Even starting with an existing open source project will require a non-trivial amount of time to roll out across your org and build robust features, not to mention adding new ones as your team and infrastructure scale. IDPs are a new frontier of software engineering, but there are tools like Humanitec so you don’t need to ask your engineers to reinvent the wheel.
Building your own solution from scratch when there are commercial tools ends up looking like undifferentiated heavy lifting, rather than focusing engineering resources on business drivers. It’s often more cost-effective for your team to purchase a tool than to build their own at the expense of customer-facing, revenue-driving features.
When leveraging an IDP framework like Humanitec, you not only relieve additional pressure from your Ops team, who would have to provide support around your in-house IDP (after building it), but you also benefit from working with experts who do nothing else but implementing IDPs for hundreds of other teams. This additional expertise can save a lot of time and resources when applied to big architectural or process design changes that affect hundreds of users in your organization.
Implementing a vendor solution also means you can have an immediate impact on your broken workflows and accumulating Ops overhead. It’s hard to quantify the effect on your bottom line, but it’s crystal clear your SRE and Platform engineers’ time is very valuable. Every month you are not implementing an IDP is costing you hundreds of thousands to millions of dollars (depending on your size) in inefficiencies, dysfunctional workflows and waiting times.
A commercial solution is constantly refined through feedback and input from a large customer base. Lots of user feedback can mean a simple, intuitive UI, and a more robust feature set that covers use cases your internal engineers may not have considered, but find very valuable.
The tradeoffs of building from scratch
There are pros to building an IDP both from scratch and using a commercial framework like Humanitec, so what are the cons?
Let’s start from the two most important things every enterprise looks at, money and time. The key thing to understand when it comes to building IDPs is that they are no piece of cake. If you want the end result to be functional and competitive with market trends, you need to invest a lot of resources and man hours into it. So the fundamental question you should ask yourself before starting to build from scratch is, is this the one thing I should have some of my most valuable assets (sr. engineers, infra team, etc.) focus on? Well, it depends.
Jason Warner, CTO at GitHub, clarifies: “If you are a small team of up to 20 developers, IDPs are mostly an overkill and you should go for a simple PaaS like Heroku. If you are a Salesforce or a GitHub, you need to build your own IDP custom to your requirements. If you are past the 20 devs mark but you are no GitHub, you should go for plug and play solution”.
Unless you are a large tech company that can throw 10s or 100s of very skilled engineers at the problem, building an IDP from scratch might not be worth prioritizing over other core business value adding activities. After all, reinventing the wheel can become quite costly, quite fast. Below we averaged the figures we gathered from over two dozen companies that built and maintain their own Internal Developer Platforms.
This is what costs on average to build an IDP for 100 application developers, with 6 Ops FTEs building it over a 6 month period. Note that this scales almost linearly, so for 500 developers you’d need about 30 FTE Ops to build it and around 10 to maintain it. Also important to notice, this estimate assumes no major hiccups in the development or reprioritization of resources. It could take you longer than 6 months to start testing your V1.
If these numbers and timelines make sense for your organization, which they very well might, the next thing you should be mindful of are second order tradeoffs, those that emerge once you have built your IDP. In particular, maintaining and enhancing an IDP requires an incredible amount of internal focus by any engineering organization. You need an internal Platform team that does little else besides iterating on your systems based on the feedback of their customers, the rest of your application development teams.
Building a proper Platform team is really the only way an homegrown IDP can keep up with external vendors, market trends and inbound feature requests from the rest of your organization. This implies 2 important things one should keep in mind. First, this is a proper team, complete of frontend, backend and QAs, not just SREs or DevOps engineers. It also needs very strong and capable product management to articulate the vision of the Internal Developer Platform and listen to all internal stakeholders and customers. Second, these will be some of the best if not the best and most experienced engineers in your organization. Centralizing all this talent into one team with a single major task vs keeping it more homogeneously present across different teams and projects is a crucial tradeoff to understand.
Building such a platform team alongside your IDP also translates into key person dependencies that inevitably develop as a result of a few of your Platform engineers holding the key to the health of your infrastructure and workflows.
The tradeoffs of building with Humanitec
There are mainly four areas that customers bring up in sales conversations: security, feature control, open source and opinionated flows.
Because a SaaS is a hosted solution, there will be traffic going out of the network. If your organization requires everything to run on-prem, Humanitec is currently out of scope (we will support on-prem starting Q4 of this year however). We also address security concerns by allowing customers to connect from on-prem infrastructures through a VPC setup, which allows for a hybrid solution that’s more compliance friendly than the regular hosted instance.
As with any vendor software, you lack control over the roadmap. You’re buying a product you can start using immediately, but you’ll be a layer removed from their product roadmap. That being said, Humanitec works closely with our customers to ship the features they need and request. Our library of integrations and drivers is an example of how our work onboarding existing customers is already benefiting the newcomers by providing an easier plug and play experience.
These drivers are open sourced for everyone to directly implement or fork and adapt to their own use cases. This is one of the few open source elements of Humanitec, for now. So if when building your IDP you’d like to stay completely in the open source universe, Humanitec is not the right place to start for you.
A final topic that comes up in client conversations is the opinionated nature of Humanitec’s approach to developer workflows. By building your Internal Developer Platform with Humanitec, you enable your Ops team to set baseline configuration templates and golden paths for the app developers to work within. This leads to opinionated workflows and a very specific view on what a developer should and should not control. We believe this is a key feature to allow for a scalable delivery setup, especially from a certain team size onwards. It’s important to mention however that depending on the permissions set through RBAC, individual developers can overrun any org-level configurations by uploading their own YAML and HELM files.
Conclusion
If you are a GitHub or a Salesforce, we can have a call to discuss best practices for architecture design, but the reality is you’ll need to build your own IDP. If you are a small team that hasn’t yet grown to the size of needing an IDP, we can keep in touch for when you do. But if you are at a stage where your Ops overhead is accumulating, your Developer Experience is deteriorating, your infrastructure is getting too complex and you need an IDP to fix it all and keep scaling, hopefully this post provided you some guidance around what you should pay attention to. If it sounded biased, well it is. But rarely we see it worth your time building something so complex as an IDP from scratch.
