At Banzai Cloud we secure our Kubernetes services using Vault and OAuth2 tokens. This has not always been the case, though we’ve had authentication in our project (even though it was
basic) from a very early PoC stage - and we suggest that you do the same.
Usually, inbound connections to Kubernetes cluster services are accessed via
Ingress. Just to recap, public services are typically accessed through a
loadbalancer service. However, that can be expensive. Ingress gives us a way to route requests to services based on the request host or path, centralizing a number of services into a single entry point. Turning on ingress authentication on Kubernetes is pretty simple and this post is about how to highlight these steps and introduce a small utility that automatically generates ingress passwords. Hopefully, this will make it easier to ramp up a project with a security-centric approach from day one.
Authentication and authorization of Pipeline users with OAuth2 and Vault Dynamic credentials with Vault using Kubernetes Service Accounts Dynamic SSH with Vault and Pipeline Secure Kubernetes Deployments with Vault and Pipeline Policy enforcement on K8s with Pipeline The Vault swiss-army knife The Banzai Cloud Vault Operator Vault unseal flow with KMS Kubernetes secret management with Pipeline Container vulnerability scans with Pipeline Kubernetes API proxy with Pipeline
Set up and configure the Ingress Controller with authentication 🔗︎
Kubernetes easily exposes services through an ingress resource. The simplest way to secure an application is to set up authentication in the
Ingress Controller. This feature is accessible with a simple annotation configuration.
The first step is to create authentication credentials via the
$htpasswd -nb user1 SecretPassword42 user1:$apr1$F/4euI1M$v7i/LYuouxfuPyNG.7g.g1
To use it as a Kubernetes secret we have to encode it in base64.
$ echo -n "user1:$apr1$F/4euI1M$v7i/LYuouxfuPyNG.7g.g1" |base64 dXNlcjE6LzRldUkxTS9MWXVvdXhmdVB5TkcuN2cuZzE=
Now we can create a Kubernetes secret based on this hash.
$ cat <<EOF | kubectl create -f - apiVersion: v1 kind: Secret type: Opaque metadata: name: myingressauthsecret data: auth: dXNlcjE6LzRldUkxTS9MWXVvdXhmdVB5TkcuN2cuZzE= EOF
Finally, we create the Ingress based on the annotations.
$ cat <<EOF | kubectl create -f - apiVersion: extensions/v1beta1 kind: Ingress metadata: name: <ingress-name> annotations: ingress.kubernetes.io/auth-type: basic # name of the secret that contains the user/password definitions ingress.kubernetes.io/auth-secret: myingressauthsecret ingress.kubernetes.io/auth-realm: "Authentication Required - foo" spec: rules: - host: <yourchoice>.<cluster-id>.k8s.gigantic.io http: paths: - path: / backend: serviceName: <service-name> servicePort: <service-port> EOF
The problem with this solution 🔗︎
Great, we now have a very simple, though not extremely secure, functioning authentication. Unfortunately it’s not very practical, making it difficult to manage and to change access. To overcome this problem we created a small application called Ingress Auth Generator that helps manage passwords. It works according to a simple principle. It specifically scans the Ingresses for the
auth-secret annotation. After it looks up the secret and finds a
username/password field, it generates the correct values instead of the auth attribute. This tool comes in handy when performing a manual release or testing. For example, manually setting the password when deploying a Helm chart
$ helm install --set global.password=example
Example Helm chart deployment 🔗︎
Checkout Ingress Auth Generator repository
Add Banzai Cloud Helm repository
Add Banzai Cloud Helm repo and build the dependencies
Install the demo chart with params
Test the login
First, keep in mind that when using
https is required. The simplest way of dealing with this in Kubernetes is the Cert-manager. We definitely need a stronger solution for production - check out how we are accomplishing this in Pipeline using OAuth2 and Vault. If you’d like your clusters and deployments to be provisioned on multiple cloud providers and to have their security automatically set up for you, you should consider Pipeline.