Cloud Infrastructure Community's Space

How to use Azure Key Vault in Azure Kubernetes Services

Azure Key Vault is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys. The Azure Key Vault Provider for Secrets Store CSI Driver allows for the integration of an Azure key vault as a secret store with an Azure Kubernetes Service (AKS) cluster via a CSI Volume.

In this article, we’re going to use Azure Key Vault to mount a set of secrets to the Azure Kubernetes Servicse cluster.

Prerequisites

How to use Azure Key Vault in AKS

Step 1. Create an AKS cluster with Azure Key Vault Provider

We need to create an Azure resource group which contains our being deployed resources. Then an AKS cluster with Azure Key Vault Provider add-ons

# make sure you've logged in to Azure CLI

# create Azure resource group
az group create --name cli-rg --location northeurope

# then create an AKS cluster with add-ons
az aks create -n cli-aks -g cli-rg --enable-addons azure-keyvault-secrets-provider --enable-managed-identity

# or you can update your existing AKS cluster to use Azure Key Vault secrets provider
az aks enable-addons --addons azure-keyvault-secrets-provider --name cli-aks --resource-group cli-rg

Step 2. Verify Azure Key Vault Provider in your AKS

Use KubeCTL get your AKS cluster context and verify our Azure Key Vault Provider for Secrets Store CSI Driver instalation status:

# get AKS kubernetes context
az aks get-credentials --name cli-aks --resource-group cli-rg

# switch to the context
kubectl config use-context cli-aks

# now get the status
kubectl get pods -n kube-system -l 'app in (secrets-store-csi-driver, secrets-store-provider-azure)'
#Output
NAME                                     READY   STATUS    RESTARTS   AGE
aks-secrets-store-csi-driver-4vpkj       3/3     Running   2          4m25s
aks-secrets-store-csi-driver-ctjq6       3/3     Running   2          4m21s
aks-secrets-store-csi-driver-tlvlq       3/3     Running   2          4m24s
aks-secrets-store-provider-azure-5p4nb   1/1     Running   0          4m21s
aks-secrets-store-provider-azure-6pqmv   1/1     Running   0          4m24s
aks-secrets-store-provider-azure-f5qlm   1/1     Running   0          4m25s

Step 3. Create an Azure Key Vault and its secrets

We’re going to create an Azure Key Vault in the same resource group where cli-aks located by:

# create Azure Key Vault
az keyvault create --name cli-kv --resource-group cli-rg -l northeurope

# and create a Key Vault secret
az keyvault secret set --vault-name cli-kv -n EnvironmentName --value "production"

Step 4. Configure AKS Key Vault access policy

At the step 1, we’ve created an AKS cluster with --enable-managed-identity enabled, an Managed Identity was created as well. We need to configure to allow AKS cluster access/manage our Key Vault in Key Vault access policy. See bellow:

# get AKS Managed Identity which will be used to authenticate against Azure services
aksManagedIdentityId=$(az aks show -g cli-rg -n cli-aks --query addonProfiles.azureKeyvaultSecretsProvider.identity.clientId -o tsv)

# set policy to access keys in your key vault
az keyvault set-policy -n cli-kv --key-permissions get --spn $aksManagedIdentityId
# set policy to access secrets in your key vault
az keyvault set-policy -n cli-kv --secret-permissions get --spn $aksManagedIdentityId
# set policy to access certs in your key vault
az keyvault set-policy -n cli-kv --certificate-permissions get --spn $aksManagedIdentityId

Step 4. Using Key Vault in Kubernetes Deployment

Now create a Secret Provider class by kubectl apply -f provider_class.yaml with following content:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: azure-sync
  namespace: default
spec:
  provider: azure
  secretObjects:
  - secretName: appsecrets
    type: Opaque
    data:
    - objectName: "EnvironmentName"
      key: EnvironmentName
  parameters:
    usePodIdentity: "false"
    useVMManagedIdentity: "true"
    userAssignedIdentityID: "your aksManagedIdentityId get above"
    keyvaultName: "cli-kv"
    objects: |
      array:
        - |
          objectName: "EnvironmentName"
          objectType: secret
    tenantId: "<your tenant Id>"

After you’ve created the Kubernetes secret and driver class, you can reference it by setting an environment variable in your pod, as shown in the following example code (save to file pod.yaml):

kind: Pod
apiVersion: v1
metadata:
  namespace: default
  name: simple-express
spec:
  containers:
    - name: simple-express
      image: hidetran/simple-express:latest
      volumeMounts:
        - name: secrets-store01-inline
          mountPath: "/mnt/secrets-store"
          readOnly: true
      env:
      - name: ENV_NAME
        valueFrom:
          secretKeyRef:
            name: appsecrets
            key: EnvironmentName
  volumes:
    - name: secrets-store01-inline
      csi:
        driver: secrets-store.csi.k8s.io
        readOnly: true
        volumeAttributes:
          secretProviderClass: "azure-sync"

Now run kubectl apply -f pod.yaml and verify pod status:

# get pods status by kubectl get pods
kubectl get pods
NAME             READY   STATUS    RESTARTS   AGE
simple-express   1/1     Running   0          34s

# get the logs of pods
k logs pods/simple-express
{"level":"info","message":"NodeJS express running on port 8000"}
{"level":"info","message":"dummy logs from simple-express at Mon Oct 03 2022 06:39:04 GMT+0000 (Coordinated Universal Time)"}

You can check the pods information to verify that your ENV_NAME variable environment was populated correctly with

# get pods 
kubectl get pods
# Output
NAME             READY   STATUS    RESTARTS   AGE
simple-express   1/1     Running   0          3s


# now access to pods
kubectl exec -it simple-express /bin/sh

# try getting content of root app
curl localhost:8000

# Output
{
    "sevice_name": "Simple Express on production",
    "port": 8000,
    "description": "Simple nodejs app running with Express.",
    "instance": "simple-express"
}

Conclusions

The above commands should make it relatively straightforward to try the secret store CSI provider and understand what it does. It works especially well in GitOps scenarios where you cannot store secrets in Git and you do not have pipelines that can retrieve Azure Key Vault secrets at deploy time.

Leave a Reply

Your email address will not be published. Required fields are marked *

Press ESC to close