Restoring Kubernetes’ ability to sign certificates

Note

Requires at least version 6.0

Since YAOOK/k8s migrated to Hashicorp Vault for managing PKI and therefore clears the Kubernetes cluster root CA key from the control plane, YAOOK/k8s clusters are not able to respond to certificate signing requests (CSRs) anymore since access to PKI keys ceased. [1]

While in the long term we want to integrate Vault via a Kubernetes custom signer [2] which would redirect CSRs in Kubernetes directly to Vault, for now we provide a workaround fix to restore the CSR functionality. The fix must be explicitly turned on by setting [kubernetes.controller_manager].enable_signing_requests=true in the config.

Enabling the fix

Attention

This weakens the security of your deployment!

  1. Enable signing requests in the config

    Set [kubernetes.controller_manager].enable_signing_requests=true in config/config.toml

  2. Configure k8s_control_plane Vault policy

    Executing ‘tools/vault/init.sh’ will take care of that. This assumes you already have a Vault instance running and reachable from your YAOOK/k8s shell environment.

    Note

    Requires a Vault root token

    ./managed-k8s/tools/vault/init.sh
    

    The Kubernetes cluster root CA key is made available through the k8s-pki/cluster-root-ca secret in the Vault kv2 store of the cluster. The script adds a Vault policy rule (among other things) that grants read-only access to it for the control plane nodes role.

  1. Create the Kubernetes cluster root CA and backup its key

    Executing ‘tools/vault/mkcluster-root.sh’ will take care of that.

    Note

    Requires a Vault root token

    ./managed-k8s/tools/vault/mkcluster-root.sh
    

    The script creates the Kubernetes cluster root CA and backs up its private key to k8s-pki/cluster-root-ca in the Vault kv2 store of the cluster [3].

  2. Build the cluster using the apply-all action

    This will copy and configure the Kubernetes cluster root CA key from Vault’s kv2 store on all control plane nodes.

  1. Optional: Check that certificate signing is functional again now

    See the Kubernetes documentation on how to do that.

Disabling the fix

  1. Disable signing requests in the config

    tomlq --in-place --toml-output '.kubernetes.controller_manager.enable_signing_requests=false' config/config.toml
    
  1. Nothing to care about, go ahead as normal