How I ported CoreOS Tectonic Installer to DigitalOcean for a cheaper hosting solution for my prototyping Kubernetes clusters.
Kubernetes can be quite a beast to set up on your own, as evidenced by Kelsey Hightower’s tutorial for doing it yourself, Kubernetes the Hard Way. Consequently, it’s popular to use services providing managed Kubernetes clusters, such as Google Kubernetes Engine or Red Hat OpenShift, or letting the by now more or less standard tool Kops create your clusters for you.
Tectonic offers an approach similar to that of the aforementioned Kops project, giving you the ability to easily stand up a Kubernetes cluster on your platform of choice (even bare metal), via the powerful infrastructure provisioning tool Terraform. Kops can also work on top of Terraform, even though by default it handles provisioning itself. What makes Tectonic stand out from Kops is that it doesn’t just stand up bare-bones Kubernetes clusters, but augments them with a holistic stack of essential services that one would otherwise have to install oneself:
- Prometheus — the de facto standard Kubernetes open source monitoring system
- Grafana — a visual dashboard for Prometheus
- Alertmanager — a companion to Prometheus for sending monitoring alerts to various destinations
- Tectonic Console — a comprehensive graphical console for observing and administrating your cluster
- Container Linux Update Operator — an agent for rebooting individual Container Linux members of the cluster in a cooperative way
Not least, Tectonic also sets up an authentication layer within the cluster, so that one has to log in in order to access the various public services such as the Tectonic Console and the Grafana monitoring dashboard. This is non-trivial to set up if you’re bootstrapping your own clusters (for example with kops) and installing Prometheus/Grafana. I’ve gone down this path myself, and it was unclear to us even which Kubernetes ready authentication solutions were available to us. In comparison it’s a real relief to bring up a Tectonic cluster, and straight away being able to launch a secured (through username and password) monitoring dashboard.
As I mentioned earlier, the act of creating Tectonic clusters is handled through the Terraform infrastructure provisioning tool. Terraform is a declarative tool, in that it reads a collection of configuration files in order to compile a graph of actions it is to carry out. Such actions can include e.g. creating virtual machines on Amazon Web Services or copying files to S3 buckets.
Tectonic has a companion project, Tectonic Installer, that contains a collection of Terraform configurations for creating Tectonic clusters on a number of supported platforms. At the time of writing, these platforms include:
- Amazon Web Services/GovCloud
- Microsoft Azure
- Google Cloud Platform
- Bare Metal
Basically, one picks the platform on which to stand up a cluster, configures Terraform correspondingly according to the Tectonic Installer documentation and invokes Terraform to create the cluster according to one’s configuration and the configuration tree of Tectonic Installer. Terraform will automatically provision the required infrastructure (such as AWS EC2 instances to host Kubernetes processes) and deploy Kubernetes software to it. Tectonic Installer has also a graphical installer, but it will not be described in this article as I have no experience with it — I have only used Terraform directly on the command line (and frankly prefer this).
Take note of the last item in the list of supported platforms, DigitalOcean, as it (and Tectonic Installer) is the real topic of this article. This is a very interesting cloud hosting platform, due to its extreme affordability, ease of use and straightforwardness. In comparison to the larger cloud hosting platforms such as AWS and Google Cloud, DigitalOcean offers great transparency wrt. hosting costs. One pays a fixed sum per resource one uses, e.g. per virtual machine (or droplet in the DigitalOcean vernacular).
Especially after DigitalOcean slashed its prices in half not so long ago, it has become a very interesting platform due to sheer affordability. For my prototyping Tectonic clusters, I pay about $80 per month, which is far less than I would pay for a Google Kubernetes Engine cluster for example!
Porting Tectonic Installer to DigitalOcean
When I realized I needed a cheaper hosting solution for my prototyping Kubernetes clusters though, and I was also interested in starting to use Tectonic, Tectonic Installer lacked support for this platform. I decided to make it my next project to port Tectonic Installer to DigitalOcean, both as I thought it would be very interesting and be a great learning opportunity and because I foresaw that I would be happier with this solution and save money.
I then forked the Tectonic Installer repository on GitHub back in May 2017 and started creating a DigitalOcean implementation, fashioned after the one for AWS. This was in no way an easy job, as Tectonic Installer is implemented directly as Terraform configuration files, which don’t lend themselves easily to code reuse and variability from user input. It was also very tricky to figure out the inner workings of the Tectonic Installer implementation and what was required to make a working implementation — thankfully, CoreOS employee Aleks Saul came to the rescue. He had implemented Tectonic Installer for VMware and knowing the ins and outs of the system, he could advise me on hurdles I faced such as generating correct certificates, hostnames and routes.
Even after getting the DigitalOcean port working, with the gracious help of Aleks and others (such as Ed Rooth of CoreOS), it would be a long and winding path before finally seeing it integrated in the official codebase. As it was a lengthy process for my patch to be reviewed and accepted, I would need to continuously synchronize my branch and keep it up to date with the official codebase. Finally, though, in February this year, my port was accepted and released as part of Tectonic Installer 1.8.7-tectonic.2!
Thanks to CoreOS kindly including my port of Tectonic Installer to DigitalOcean in their official codebase, albeit in a pre-alpha state at the time being, we are now equipped with a highly affordable way to produce state of the art container clusters (i.e. with Kubernetes) in the cloud. This makes me very happy as not only am I able to deploy my prototype apps to Kubernetes without breaking the bank, but I am also able to share this method with the community at large!
In order to try Tectonic Installer for DigitalOcean yourself, download the latest release and follow the official documentation on how to install with Terraform on Amazon Web Services. There is no documentation yet for installing on DigitalOcean, so AWS is the closest we get in this regard — just make sure to replace references to AWS with DigitalOcean.
There are some limitations to the Tectonic Installer implementation for DigitalOcean compared to other platforms such as AWS. Perhaps most importantly, the control plane does not have high availability, i.e. it consists of a single node and its uptime will depend on said node. We can only have a single master node for the banal reason that the DigitalOcean load balancer doesn’t support HTTPS health checks, while the Kubernetes API doesn’t provide HTTP endpoints for security reasons. Additionally, Kubernetes doesn’t yet have in-built support for creating load balancers on this platform or for creating persistent volumes. We are also experiencing a snag when tearing down clusters, because Terraform isn’t able to delete a floating IP address, so it has to be done manually before repeating the cluster teardown.