Humanitec helps you create an internal developer platform for an entirely new, self-servicing developer experience. This helps teams accelerate the adoption of DevOps practices and continuous delivery to deliver cloud-native, Kubernetes-based applications.
Polar Squad is a DevOps agency that delivers cloud-native scalable and reliable development practices.
The participants were:
- Ari Flinkman (Polar Squad)
- Yair Etziony (Polar Squad)
- Firas Fafi (Polar Squad)
- Chris Stephenson (Humanitec)
Brief explanation of External Resource and Resource Drivers in Humanitec
It’s not possible (or doesn’t make sense) to run everything an application needs in a Kubernetes cluster. This includes services such as public DNS, caches, S3 buckets, and long-running databases, and teams often use managed services instead of running them inside a cluster.
Humanitec can manage and provision these external resources to the cluster but needs to know of their existence and how to communicate with them. This is why we created resource drivers. They fulfill the resource driver interface that allows Humanitec to perform specific actions on external resources with their APIs, such as creating and destroying them. They use the credentials provided by Resource Accounts to manipulate resources provided via managed services.
Resource Drivers are web services with a POST and DELETE endpoint. The POST payload includes a unique ID, the type of resource to create, and any other parameters the resource needs. The POST handler needs to provision the resource and then return data to the service that the resource needs. For example access credentials or the hostname of the resource. The DELETE endpoint tells the driver to free up the resource.
Humanitec provides several default Resource Drivers, and it is also possible to use 3rd party Resource Drivers or to write your own.
Goals for the hackathon
Similar to the goals of any hackathon, we were interested in finding out truly how extensible the platform is and watching the experience developers had in creating a resource driver for common external infrastructure.
One of the main promises of Humanitec is that development teams can use it as their internal developer platform. An important part of that is the ability to request resources their application needs and then fully provision an environment. We wanted to know how feasible this is for common use cases, how good the Humanitec documentation was, and how easy it was to develop a driver without in-depth knowledge.
Before the hackathon, the participants agreed that the aim was building a resource driver for a cloud-hosted Redis compatible memory store. This setup would allow developers to write software that used Redis but that they didn’t have to provision themselves.
Report on Hackathon
Initial planning
Based on previous experience, there was a small preference for using AWS resources, which meant using AWS ElastiCache for Redis. We decided to extend the existing S3 driver instead of creating a client from scratch.
An inconvenience we discovered is that AWS ElastiCache is only accessible within the same Subnet and Virtual private cloud (VPC) as the calling instance. This meant that we would need to spin up an Elastic Kubernetes Service (EKS) cluster to host the sample application that talks to the ElastiCache instance.
We were aiming for the following setup:
- To use Humanitec to deploy the driver service.
- To use Humanitec to provision ElasticCache for Redis using the driver it deployed alongside an application that uses the EKS cluster.
Our first surprise was that even a small ElastiCache instance could take 3 or 4 minutes to provision. This means it takes a while before a driver knows if it has an endpoint. Finding the endpoint details for the ElastiCache instance was also surprisingly difficult as it required waiting for it to become available and querying the nodes of a cluster to get the endpoint details.
Outcome
At the end of the hackathon, we declared a partial victory. We wasted most of our time dealing with small frustrations, configuration errors, or limited Go knowledge across the participants. Fortunately, Humanitec helped us here by making it easier to find logs, but that didn’t help us find errors from infrastructure hosted elsewhere.
Late into the evening, we successfully got the driver working. Humanitec used the driver to create an ElastiCache instance, but our example application deployment failed due to a time out somewhere. At this point, we felt we had learned enough for one day and wrapped up.
Learnings
Hackathons
First, we learned a few lessons that other hackathon organizers might find useful.
Do not underestimate preplanning certain aspects of a hackathon, whether that be as an organizer or as a participating team.
We thought we were all clear on the scope but had different assumptions and opinions on what our definition of done was and wasted some time discussing, experimenting, and fiddling. Instead, we should have agreed on some of the following essential building blocks before the hackathon.
- The cloud provider we would support with the driver.
- The programming language we would use.
- The infrastructure we would need and how to set it up.
- The demo app we would use to test the driver.
Platform
More crucially, we gained insights into issues with our platform, resource driver developer experience (DX), and potential pitfalls developers may experience.
While we can’t directly affect the DX of external providers, we encountered a handful of issues and inconsistencies that users of Humanitec are likely to encounter. We could highlight these in our documentation and offer helpful external resources to help developers overcome them.
We realized during the hackathon that our API doesn’t give enough feedback (especially in relation to malformed configuration), and this is something we intend to work on ASAP. As developers, we also tend to focus too much on what will happen when an application works as designed, but not when it fails. We noted this for what we created during the hackathon, but also for Humanitec itself. An application or service that gives appropriate and good feedback no matter the outcome improves DX, especially if the developer is already frustrated.
