This first blog post covers the basics that are relevant to everyone - developers, DevOps, Ops etc. It deals with "Resource Types", a core concept in Humanitec. The later articles are less relevant to developers and are focused on how Operations teams use resources. They cover resource definitions, resource matching criteria and drivers.
A possibly familiar scenario
As a cloud native developer, you'll be writing applications that make use of standard components. These might be managed services such as PostgreSQL as a service from Aiven or MongoDB ATLAS for MongoDB as a service. In your cloud based dev environment, instead of using these managed services, you might run these services on a big shared VM to save costs. Maybe the QA environment has not migrated to Aiven for PostgreSQL yet and is still using the legacy offering from AWS. That's at least 3 separate sets of connection strings you'll need. What's more, the three environments are probably each managed by different teams. Let’s face it, you are probably not top of their list when it comes to getting a new database set up for your shiny new service.
As you're spinning up a new service that needs its own PostgreSQL database, it is your job to chase down the DevOps team that looks after the shared dev PostgreSQL instance, find the person in the QA team who managed their RDS instance and manage to get a response from your ticket to the DBA team who manage the production systems in Aiven. You need to speak to each of them and make sure that they understand your requirements. You then need to get credentials (and look after them!)
Imagine if instead of chasing down things that your service needs, you could just declare what it depends on? A fresh database in every environment you deploy your code to. No chasing down the right person to provision your database. No need to fiddle around with managing a bunch of different connection strings. No missing update emails about switching instances.
What if this covered everything your service needed, from DNS names to message queues to auto-scalars. This is what Humanitec Resources give you!
What is a Resource?
At its simplest, a resource is something that is consumed or used by a Workload. A Workload in Humanitec is the same as a Kubernetes workload. It's what holds the containers that contain the code you actually want to run. Often the code in these containers need other services or bits of infrastructure to operate correctly.
For example, a container might:
- store data in a database such as PostgreSQL,
- read events from a message queue such as AWS SQS or
- expose an API that needs to be accessible via a DNS name on the internet.
These things: databases, message queues or DNS names are examples of "Resources" in Humanitec.
Every Resource has a Type
There are often many ways in which a resource can be provisioned. For example, a PostgreSQL database can be provided by:
- an instance running on a physical machine in a data centre,
- by a cloud service such as Amazon RDS or Google CloudSQL or
- as another workload running in the Kubernetes cluster.
In most cases, the developer does not actually care exactly how the resource their application is using is provisioned. What matters is what sort of resource it is and that there is a standard way of making use of it.
A Resource Type provides a way of working with resources without worrying about how the actual resource is provisioned. Resource Types define a set of inputs and outputs that are specific to the technology they represent. The inputs are hints that a developer can give to how the resource should be provisioned. The outputs are parameters that the workload will need in order to make use of the resource.
For example, let's consider a service that makes use of a PostgreSQL database. The service uses UUIDs, so it needs the PostgreSQL uuid-ossp extension installed in the database to help generate new UUIDs for keys. In order to connect to the database the service will need a connection string. This should contain information about the host, port, user, password and database name needed to connect to the instance that the database is in.
In Humanitec, this would be represented as a Resource of type postgres. The input would list uuid-ossp as a required extension. The outputs would be 5 values:
- the host which the PostgreSQL instance is running on
- the port that the instance is listening on
- the name of the database in the instance and
- the username and password of the user with permissions on the database.
Private and Shared resource dependencies
We know how to define a resource using a Resource Type. But how do we describe that our workload depends on a resource of a particular type?
In Humanitec you can define either a private or shared resource. The only difference between them is whether they can be used by every workload or just a specific one. You can think of it as a way of organizing your resources. In a big microservice app containing 20 or 30 services, it's easier to see what resources are used by which service if most of them are private.
Fortunately, other than the scope, both shared and private resources work the same way. You:
- choose an ID to help you reference the resource,
- select the type of the resource and
- provide any inputs if necessary.
Placeholders to inject resources into configs
Once you have defined a dependency on a resource of a particular type, you somehow need to get your code to make use of it. This means getting data about the resources into the configuration of your service. Normally this is done by Environment Variables of configuration files. For resources, the resource outputs are the data we need.
Humanitec provides a form of templating using Placeholders. The format is similar to template strings in JavaScript or variable substitution in shell scripts. They start with ${ and end with }. Inside is the path to the value separated by dots. The placeholders are resolved at deployment time to the value they represent.
For example, if a shared PostgreSQL database resource with ID users is defined, a DATABASE_HOST environment variable can be populated with the following placeholder:
${shared.users.host}
In this case:
- shared indicates that the resource is a shared resource
- users is the ID of the resource
- host is one of the resource outputs for the resource type postgres.
As Humanitec supports full template strings, we can do more than just assign one environment variable at a time. We can construct an entire connection string! For example:
postgresql://${shared.users.username}:${shared.users.password}@${shared.users.host}:${shared.users.port}/${shared.users.name}
Private resources are referenced with the unituative identifier of externals.
Putting it all together
Let's take a look at how you would go about adding a postgres database as a resource dependency on a workload.
We start off by adding a postgres resource as a private dependency on our workload:
Next we get to choose an ID for this resource dependency so we can reference it later:
Finally, we can build a connection string out of placeholders that reference the resource dependencies outputs:
Summary
- A Resource is something that is consumed by a Workload.
- Every Resource has a Type which takes a set of Inputs and produces a set of Outputs.
- Resources can either be private to a workload or shared between workloads.
- Resource outputs can be injected into environment variables and files via Placeholders.
In the next blog post, we’ll show how Ops/DevOps/Backend engineers (anyone in charge of managing your setup essentially) can configure exactly how Resources get provisioned by Humanitec, as developers request them through the UI, CLI or API.
