Skip to main content

Glasskube v0.0.1 β€” Technical Preview

Β· 5 min read
Philip Miglinci
Co-Founder
Louis Weston
Co-Founder

The aim of this post is to share our technical preview of how a cloud native package manager could work and what challenges need to be solved.

Glasskube is fully open-source. Support us by leaving a star: ⭐ glasskube/glasskube ⭐

Introducing Glasskube β€” The next generation Package Manager For Kubernetes​

Glasskube GUI Mockup

Package Management on Kubernetes is one of the most pressing issues in the Cloud Native community. A concept which is widely known from other ecosystems like desktop and mobile computing has yet not been realized for cloud computing. For example on Android and iOS, millions of developers publish their packages in the Play Store or App Store to reach their users. The package manager also makes sure all users receive the latest version published by the developer and the developer receives crash reports and user feedback as a return to improve their applications, but as a cloud native developer there is no package manager you can rely on β€” yet.

Our first release (v0.0.1) already features a working prototype that can install basic packages, but a lot of challenges still need to be solved.

A cloud native architecture​

Glasskube itself is designed as a cloud native application. Featuring an easy to install client that comes with a graphical user interface and autocompletion for your favorite shell.

At the heart of the Glasskube package ecosystem lies our central package registry which holds the package manifests. In a future version we also plan to support 3rd party registries and the possibility to use multiple registries in a cluster.

The Glasskube package operator syncs the latest manifest into the cluster and makes sure it will be updated as soon as a new manifest is available.

Challenges that need to be solved​

We already covered some our upcoming features in our public roadmap, but I would also like to take this opportunity to shortly speak about broader challenges.

Kubernetes version compatibility​

Kubernetes releases minor versions every 4 months, which often come with new API versions. Package authors need to adapt their packages to these changes. In Kubernetes, a particular release might include more than one API version of a resource, so that packages can be compatible with a broader range of Kubernetes versions. These compatible versions are often only documented in the package distributors' changelog. Glasskube aims to incorporate this kind of metadata in combination with automatic checks from tools like kube-no-trouble or Pluto.

The user should not be required to drudgingly check all packages for compatibility and package developers should get feedback if their package is not compatible with the latest API versions.

Package dependencies​

Cloud native applications often interoperate and there are some packages that can be found in almost every Kubernetes cluster. For example: cert-manager, Ingress controllers or database operators. Due to the lack of a package manager and ecosystem these dependencies are still often only documented in the Getting started section of an application.

In an ideal world a package author could simply specify a dependency of their package and the package manager ensures that all these prerequisites are fulfilled.

Testing​

In order to support multiple Kubernetes versions, dependencies and packages Glasskube needs to build massive automated testing infrastructure for all packages in its central package registry.

Feedback and package quality​

As seen in other package managers like the arch user repository or the Play Store and App Store users' feedback and reviews help other users to decide between different packages. Also, application developer will incorporate users' feedback to gain popularity and better ratings in the package manager.

Glasskube and Helm​

Glasskube is no replacement for Helm. Helm has its strengths in configuring releases through templating and having the ability to perform upgrades and rollbacks.

Glasskube is laser focused on the administrator who needs to only install and kustomize (pun intended πŸ˜‰) a single application, but who also needs to make sure multiple packages are kept up-to-date and secure throughout multiple Kubernetes version upgrades and adapting to inevitable breaking changes.

Getting started​

Follow our Getting Started guide if you want to try Glasskube for yourself and install your first package.

On macOS, you can use Homebrew to install and update Glasskube.

brew install glasskube/tap/glasskube

After installing Glasskube on your local machine, make sure to install the necessary components in your Kubernetes cluster by running glasskube bootstrap. For more information, check out our bootstrap guide.

Release Notes​

All release notes can be found on GitHub: https://github.com/glasskube/glasskube/releases/tag/v0.0.1

Release Video​

Get involved​

The easiest way to get involved is to tackle one of our open issues. You are also welcome to join our Discord.

If you are a cloud native developer, please submit your package.

As Glasskube is still in its very early days, your feedback is highly appreciated.

Let us know what you think, we would love to hear from you or support us by leaving a star: ⭐ glasskube/glasskube ⭐

5 shortcomings of Helm

Β· 10 min read
Philip Miglinci
Co-Founder
Jakob Steiner
Software Engineer

helm-thumbnail

5 reasons we are trying to build the next generation of deployment automation for Kubernetes.

Glasskube is fully open-source. Support us by leaving a star: ⭐ glasskube/glasskube ⭐

Introduction​

As a seasoned DevOps engineer, I found that Helm, the popular Kubernetes deployment tool, has some shocking shortcomings. In this post I want to discuss some of those which, in my opinion, require a new vision of a more modern deployment solution. If you never heard about Helm before, in a nutshell it is:

A framework for packaging Kubernetes resources (apps) to charts, publish them and let them easily be installed via a command line interface.

The goal of this post is not to hate on the smart and talented people who built helm, but maybe we can kindle a productive and healthy discussion about where we need to go as the DevOps industry to stay relevant in the coming years. But to fully understand the following, I think it is important to understand what developments lead us to where we are today. So, before we start, let's quickly dive into the history of Helm.

In 2015, a company called Deis created Helm, a package manager for Kubernetes. Deis is now part of Azure Kubernetes Service but the original project still exists as Helm Classic. At the same time, Google had a project called Kubernetes Deployment Manager, which was similar to Google Deployment Manager but for Kubernetes resources, rather than GCS resources.

At the beginning of 2016, the two projects decided to merge, which resulted in the release of Helm v2 later that year. Helm v2 consists of a client and server component (Helm and tiller, respectively), where the latter was the continuation of the original Kubernetes Deployment Manager project. Tiller was designed to handle deployment states to make it easier for multiple users to use Helm without interfering with each other.

In 2018, Helm launched the Helm Hub as a central place to discover charts which are otherwise found in distributed repositories. Helm Hub was rebranded to Artifact Hub in 2020.

With the release of Kubernetes 1.6, which had Role Based Access Control (RBAC) enabled by default, production deployments of helm became more difficult, because of the many security policies that were required by tiller. So, people started to experiment with a new approach that could do the same thing without requiring a server component, which resulted in the release of Helm v3 in 2019.

As you can see, helm has a very rich history. It became the gold-standard of packaging apps for Kubernetes and is used by DevOps engineers all over the world. But just because helm is the biggest player on the field, it doesn't mean that it is without flaws. So then, why say goodbye to helm?

5 shortcomings:​

1. Helm doesn't provide a mechanism for upgrading Custom Resource Definitions​

helm does provide a method of packaging Custom Resource Definitions (CRDs) by placing them in a dedicated crds directory, but these are ignored during upgrades! This is intentional and designed to prevent accidental data loss. Therefore, upgrading a chart does not automatically upgrade it's associated CRDs, which is unexpected for many engineers and leads to more manually involved and error-prone upgrade procedures and other anti-patterns.

To combat this major design flaw, chart developers have come up with several strategies, the most popular of which are:

  • Putting the CRDs into the chart's template directory
  • Creating separate sub-charts just for CRDs

An alternative way to overcome this shortcoming is to not invoke helm commands directly, but rather use a CI/CD solution, like Flux. Flux provides a setting to automatically update CRDs during a helm upgrade, but it is off by default.

2. Helm dependency management​

The way to specify a dependency in a helm chart is to reference it as a sub-chart. This method can work great for tightly coupled dependencies that you might want to install separately or as part of another chart, but it has some weaknesses that are important to understand:

  • Sub-charts are always installed in the same namespace as the primary release and there is no way to change this.
  • There exists no mechanism to share a dependency between two releases.

For example, our Glasskube Operator Helm Chart depends on kube-prometheus-stack, velero and a bunch of other dependencies, some of which are already installed in many Kubernetes clusters. To provide an installation experience that is as simple as possible, the chart references all those dependencies as sub-charts, but using this approach, all those dependencies are bundled in the Glasskube Operator release and can not be changed or updated separately. Additionally, there is no way to check whether a dependency is already installed, so a user might end up with two separate installations of the same helm chart!

Ideal tooling would allow chart developers to specify external dependencies and simply ensure that those are present in a cluster before a chart can be installed. This way, dependencies could be shared among consumers. This is how package managers for operating systems work since forever. Why does Kubernetes need to be different?

3. Helm chart creation is not user-friendly​

So far, we discussed problems that affect you as a chart user. But what does the situation look like for chart developers?

Well, let's start by creating a new chart. This can be achieved by calling helm create your-chart. I invite you to quickly open a terminal, run this command and go through all the files it creates. As I'm sure you will agree, it's… a lot. I still remember the moment when I wanted to create my first helm chart and saw the results of this command thinking, β€œthis can't be right.”

.:
total 8,0K
drwxr-xr-x. 2 kosmoz kosmoz 40 7. Dez 13:23 charts/
-rw-r--r--. 1 kosmoz kosmoz 1,2K 7. Dez 13:23 Chart.yaml
drwxr-xr-x. 3 kosmoz kosmoz 200 7. Dez 13:23 templates/
-rw-r--r--. 1 kosmoz kosmoz 1,9K 7. Dez 13:23 values.yaml

./charts:
total 0

./templates:
total 28K
-rw-r--r--. 1 kosmoz kosmoz 1,9K 7. Dez 13:23 deployment.yaml
-rw-r--r--. 1 kosmoz kosmoz 1,8K 7. Dez 13:23 _helpers.tpl
-rw-r--r--. 1 kosmoz kosmoz 925 7. Dez 13:23 hpa.yaml
-rw-r--r--. 1 kosmoz kosmoz 2,1K 7. Dez 13:23 ingress.yaml
-rw-r--r--. 1 kosmoz kosmoz 1,8K 7. Dez 13:23 NOTES.txt
-rw-r--r--. 1 kosmoz kosmoz 326 7. Dez 13:23 serviceaccount.yaml
-rw-r--r--. 1 kosmoz kosmoz 370 7. Dez 13:23 service.yaml
drwxr-xr-x. 2 kosmoz kosmoz 60 7. Dez 13:23 tests/

./templates/tests:
total 4,0K
-rw-r--r--. 1 kosmoz kosmoz 388 7. Dez 13:23 test-connection.yaml

In total, helm create generates 10 files in different sub-directories and it is not immediately apparent which ones of those are actually essential for a chart and which ones are just example code. I once complained about this to a DevOps engineer who had already created dozens of charts and they laughed and said:

β€œThe first step in creating a chart is running helm create. The second is deleting 90% of the results.”

Really? That's the best we can do? Okay, let's accept that and say you figured out the structure of your new chart. Now, you probably want to add some resources. Of course, you can just drop your existing YAML files into the chart's templates directory, but you're probably interested in using some parameters from your values.yaml in your resources. After all, that would kind of be the point of creating a helm chart in the first place. To look at an example, go back to your terminal (where, previously, you created your helm chart) and check out the file templates/serviceaccount.yaml:

{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "your-chart.serviceAccountName" . }}
labels:
{{- include "your-chart.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}

Now, I know what you're thinking:

That doesn't look like the YAML I know! What is ' include, toYaml and nindent and what's up with all the - and {{ and |?

It's true, although helm template files use the file name extension for YAML, they are actually just templates. Helm templates are based on the Go template language which is very flexible and powerful but doesn't really know anything about YAML or Kubernetes. That's why it's necessary to call lots of these conversion functions inside the template files.

As a result many popular charts end up with template files that contain more template language stuff than actual YAML. This makes them hard to read and maintain, especially as someone who wasn't involved in its creation.

4. The values.yaml file is an anti-pattern​

Now, let's go back to something that's a little more tangible for you as a helm user. As a Kubernetes application developer who writes resources as YAML files, you are probably used to having rich support in your development environment, including strict schema validation and super comprehensive autocomplete. Creating a values.yaml file for a chart release is a little different. See, there is no general schema for what goes and doesn't go inside a values.yaml file. Thus, your development environment cannot help you beyond basic YAML syntax highlighting. The only way to verify if your values.yaml file is valid is to run it through helm and see what happens. Using helm template allows you to render these helm templates which detects possible errors in the configuration file.

A lot of chart developers want to give users the possibility to fine tune most aspects of final deployment. As a result, the number of possibilities for configuration is often unreasonably large and complicated, mimicking the actual resources they want to create, but without any schema validation!

5. Inability to interact with the Kubernetes API​

We already discussed 4 shortcomings of helm, but in my opinion the biggest downside of helm is this: A helm release is strictly a one-shot operation. Once a helm release is successfully installed, helm's job is done. But here's the thing: Installing an application is usually not the hard part, maintaining an installation and keeping it running is. Unfortunately, helm doesn't really help you with that.

After finishing installation of a release, helm can not perform any additional changes due to its design as a strictly client-side application. This inability to interact with the release during later stages of a release's life-cycle means that helm as a deployment method inherently static, but modern software deployments are often required to be very dynamic.

A simple example that an Operator can do, but helm can't would be setting the Ingress class and annotations dynamically based on the detected Kubernetes environment:

Detecting the cloud environment:

private val dynamicCloudProvider
get() = when {
kubernetesClient.configMaps().inNamespace("kube-system").withName("shoot-info")
.get()?.data?.get("extensions")?.contains("shoot-dns-service") == true ->
CloudProvider.gardener

kubernetesClient.nodes().withLabel("eks.amazonaws.com/nodegroup").list().items.isNotEmpty() ->
CloudProvider.aws

kubernetesClient.nodes().withLabel("csi.hetzner.cloud/location").list().items.isNotEmpty() ->
CloudProvider.hcloud

else ->
CloudProvider.generic
}

Applying configurations based on the environment:

protected val defaultIngressClassName: String?
get() = when (configService.cloudProvider) {
CloudProvider.aws -> "alb"
else -> configService.ingressClassName
}

protected fun getDefaultAnnotations(primary: P, context: Context<P>): Map<String, String> =
configService.getCommonIngressAnnotations(primary) +
when (configService.cloudProvider) {
CloudProvider.aws -> awsDefaultAnnotations
CloudProvider.gardener -> gardenerDefaultAnnotations
else -> getCertManagerDefaultAnnotations(context) + ingressNginxDefaultAnnotations

Conclusion​

Although many developers are a bit scared of helm at first, its simple design gave helm the lead it currently holds in the space of Kubernetes deployment methods. Helm is currently the de-facto standard for managing complex application deployments, but that doesn't mean we shouldn't challenge its design and point out shortcomings. New requirements for applications will require more dynamic deployment methods and we DevOps engineers and application developers must be prepared.

This is why we started Glasskube: An easier way to deploy applications and infrastructure on Kubernetes with Glasskube glasskube/glasskube.

If you want to follow our progress make sure to star glasskube/glasskube and join our Discord.