Environment Variable Reference
The cluster management action scripts rely extensively on environment variables to interact with the cluster. A full overview of the variables is provided below. It is strongly recommended to read the whole document before starting to initialize a cluster repository for the first time.
Tip
It is recommended to use direnv to automatically
set the required variables. The cluster repository contains an .envrc
which should be committed and contain all cluster specific settings.
It sources ~/.config/yaook-k8s/env
which should contain all user specific
settings which apply to all clusters.
Additionally, .envrc.local
is sourced which should not be committed and
contain settings which are specific to cluster and user.
Hint
This repository contains a template file which you can use. However, you must adjust some of its values.
Minimal Required Changes
When initializing your env vars from the template, you´ll need to minimally (sic!) adjust the following ones:
If you’re deploying on top of OpenStack:
SSH Configuration
TF_VAR_keypair
(user specific)
VPN Configuration
wg_private_key_file
(user specific)wg_user
(user specific)
For potentially productive setups, setting
YAOOK_K8S_CA_*_OVERRIDE
as described in the template is strongly encouraged.
Details about these can be found below.
General
Environment Variable |
Default |
Description |
---|---|---|
|
Boolean value which either force enables or force disables coloured output of the scripts. By default, the scripts check whether they are running inside a tty. If they are, they will use coloured output. This environment variable can be set to override the auto-detection. |
OpenStack credentials
We support v3password
(user name / password) and
v3applicationcredential
(application credentials) as authentication
schemes. They differ in the set of environment variables you have to
provide.
Both schemes need:
OS_AUTH_URL
,OS_REGION_NAME
,OS_INTERFACE
andOS_IDENTITY_VERSION
.User name/password based authentication requires additionally:
OS_PASSWORD
,OS_PROJECT_DOMAIN_ID
,OS_PROJECT_NAME
,OS_USERNAME
,OS_USER_DOMAIN_NAME
.Application credential based authentication requires additionally:
OS_AUTH_TYPE=v3applicationcredential
,OS_APPLICATION_CREDENTIAL_ID
,OS_APPLICATION_CREDENTIAL_SECRET
.These MUST be set if you want to deploy on OpenStack.
These variables are used by Terraform to create, maintain and destroy the underlying harbour infrastructure layer. They are also needed by the Cloud Controller Manager when applying the k8s-base layer.
Warning
These credentials are copied into the cluster. You SHOULD use a dedicated OpenStack project for your cluster.
Warning
Only use this exact set of variables. Using other,
semantically similar variables (such as OS_PROJECT_DOMAIN_NAME
instead of OS_PROJECT_DOMAIN_ID
) is not supported and will lead
to a broken cluster; the configuration files inside the cluster are
generated solely based on the variables listed above.
Warning
Currently the combination of thanos and application credentials is not supported.
Sample openrc for user name/password based authentication
export OS_AUTH_TYPE=v3password # optional
export OS_AUTH_URL=https://identity.xyz:5000/v3
export OS_PROJECT_ID=0xdeadbeef
export OS_PROJECT_NAME="janedoes-project"
export OS_USER_DOMAIN_NAME="Default"
export OS_PROJECT_DOMAIN_ID="default"
export OS_USERNAME="jane.doe@xyz"
export OS_PASSWORD="super_secure"
export OS_REGION_NAME="abcd"
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3
Sample openrc for application credentials based authentication
export OS_AUTH_TYPE=v3applicationcredential
export OS_AUTH_URL=https://identity.xyz:5000/v3
export OS_APPLICATION_CREDENTIAL_ID="0xdeadbeef"
export OS_APPLICATION_CREDENTIAL_SECRET="alsoSuperSecure"
export OS_REGION_NAME="abcd"
export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3
External resources
Environment Variable |
Default |
Description |
---|---|---|
|
|
This git URL is used by |
|
|
If set to If set to |
|
|
Git URL to a repository with wireguard
keys to provision. Can be enabled by
setting |
|
|
Path to the Terraform root module to change the working directory for the execution of the Terraform commands. |
Secret Management
Environment Variable |
Default |
Description |
---|---|---|
|
(unset) |
Address of the HashicorpVault server expressed as a URL and port. |
|
(unset) |
Vault authentication token. |
VPN Configuration
Environment Variable |
Default |
Description |
---|---|---|
|
|
This variable defines the name
of the WireGuard interface to
create. Interface name length is
restricted to 15 bytes and should
start with |
|
|
The ID of the wireguard endpoint to use when connecting
to the VPN, as defined in |
|
|
Path to your WireGuard private key
file. This is not copied to any
remote machine, but needed to
generate the local configuration
locally and to bring the VPN tunnel
up. You MUST adjust this
variable or |
|
Alternatively you can directly
export your WireGuard private key
instead of a path to it. The key
is injected via |
|
|
|
Your WireGuard user name as
defined in the wireguard configuration
(or, if enabled, |
|
|
If set to true, |
SSH Configuration
Environment Variable |
Default |
Description |
---|---|---|
|
|
Defines the keypair name (in OpenStack) which will be used during the creation of new instances. Does not affect instances which have already been created. You MUST adjust this variable if you want to deploy on top of OpenStack. This variable is used by the |
|
The SSH user to use to log into the machines. This variable SHOULD be set. By default, the Ansible automation is written such that it’ll auto-detect one of the default SSH users ( |
Behavior-altering variables
The variables in this section should not be set during normal operation. They disable safety checks or give consent to potentially dangerous operations.
Environment Variable |
Default |
Description |
---|---|---|
|
|
Boolean value which defaults to false. If set to |
|
|
Boolean value which defaults to false. If set to |
|
|
Boolean value which defaults to false. If set to |
|
By default, |
|
|
This allows to pass additional flags to Ansible. The variable is interpolated into the ansible call without further quoting, so it can be used to do all kinds of fun stuff. A primary use is to force diff output or only execute some tags: |
Note
The destruction of the cluster will fail if Thanos monitoring data is still present in the object store. The reason for that is that terraform is not configured to delete the data by default. The reason for that, in turn, is that we want the operator to be aware that possibly contract-relevant monitoring data needs to be explicitly saved before destroying the cluster.
Note
You should not use the AFLAGS
-mechanism to pass
sustained variables to Ansible. These variables should be set in your
Ansible configuration file or hosts file(s).
Note
If you have already initialized you cluster repository,
you’ll need to rerun the
init.sh
-script
after enabling the Customization layer.
Vault tooling variables
YAOOK_K8S_CA_ORGANIZATION_OVERRIDE
: Overrides the “organization” name in X.509 identities for CAs (root and intermediate) created by the Vault tooling.YAOOK_K8S_CA_COUNTRY_OVERRIDE
: Overrides the “country” identifier in X.509 identities for CAs (root and intermediate) created by the Vault tooling.VAULT_TOKEN
: Standard environment variable where the Vault CLI, all scripts and the LCM look for a ready-to-use token. Note that the LCM (and only the LCM, i.e. the ansible roles) ignores this variable ifVAULT_AUTH_METHOD
is set to a value different thantoken
.VAULT_AUTH_METHOD
(LCM only, default:token
): The authentication method to use for all orchestrator-controlled Vault operations. The only other supported value isapprole
, which requiresVAULT_AUTH_PATH
,VAULT_ROLE_ID
andVAULT_SECRET_ID
to be set.VAULT_AUTH_PATH
(LCM only, no default): Path to the authentication engine to use. Only used for non-token
VAULT_AUTH_METHOD
.VAULT_ROLE_ID
(LCM only, no default): IfVAULT_AUTH_METHOD
is set toapprole
, this must be set to the role ID to authenticate with.VAULT_SECRET_ID
(LCM only, no default): IfVAULT_AUTH_METHOD
is set toapprole
, this must be the secret ID to authenticate with.YAOOK_K8S_VAULT_PATH_PREFIX
(default:yaook
): Vault URI path prefix to be used for all secrets engines used by yaook/k8s. Changing this is not fully supported and at your own risk.YAOOK_K8S_VAULT_POLICY_PREFIX
(default:yaook
): Vault policy name prefix to be used for all policies created by yaook/k8s. Changing this is not fully supported and at your own risk.YAOOK_K8S_VAULT_NODES_APPROLE_NAME
(default:$YAOOK_K8S_VAULT_PATH_PREFIX/nodes
): Vault auth engine mount point to be used for the approle engine used to authenticate nodes. Changing this is not fully supported and at your own risk.
Miscellaneous
Variables which do not really fit into another category.
Environment Variable |
Default |
Description |
---|---|---|
MINIMAL_ACCESS_VENV |
|
Boolean value which defaults to |
Template
The template file is located at templates/envrc.template.sh
.
# Example .envrc file.
# shellcheck shell=bash
# This file is meant to be checked into the VCS and to contain
# environment variables specific to the cluster and shared
# between users.
# User specific variables are read from 3 locations.
#
# 1. ~/.config/yaook-k8s/env
# 2. the next .envrc in parent directories
# (via source_up)
# 3. .envrc.local
#
# The first can contain user specific variables
# that apply to all clusters.
# The second can be used to target subsets of
# clusters.
# The third is for user and cluster specific
# variables and is sourced in the end of this .envrc
# so it can override variables.
source_env ~/.config/yaook-k8s/env || true
source_up || true
# For up-to-date direnv versions one can also use:
# https://direnv.net/man/direnv-stdlib.1.html#codesourceenvifexists-ltfilenamegtcode
#source_env_if_exists ~/.config/yaook-k8s/env
# https://direnv.net/man/direnv-stdlib.1.html#codesourceupifexists-ltfilenamegtcode
#source_up_if_exists
source_env "$PWD/.envrc.local" || true
# For up-to-date direnv versions one can also use:
# https://direnv.net/man/direnv-stdlib.1.html#codesourceenvifexists-ltfilenamegtcode
#source_env_if_exists "$PWD/.envrc.local"
source_env ./managed-k8s/.envrc.lib.sh || true
use flake_if_nix ./managed-k8s
layout poetry ./managed-k8s
# For more details on existing environment variables and their effects,
# please see docs/admin/cluster-repo.md in the managed-k8s lcm
# repository.
# Wireguard: Role out C&H company members
export WG_COMPANY_USERS=false
# Wireguard: Interface and config file name
wg_conf_name='wg0'
export wg_conf_name
# Specifies which wireguard endpoint to use as client
export wg_endpoint=0
# Optional: Vault: Activate Hashicorp Vault Docker container
export USE_VAULT_IN_DOCKER=false
# These should be set according to your org and country. This will be used to
# provision root and/or intermediate CAs depending on which workflow you chose.
# It is irrelevant for development/testing setups, but you should probably get
# this right for productive setups in order to avoid any confusion.
#export YAOOK_K8S_CA_ORGANIZATION_OVERRIDE='Your Company Ltd.'
#export YAOOK_K8S_CA_COUNTRY_OVERRIDE='XX'
# Vault: Define Vault data storage path
#VAULT_DIR="$(pwd)/vault"
#export VAULT_DIR
# Vault: Env var script for the Vault docker instance
# Resource your envrc.sh file again after you've started Vault,
# to source Vault related information, like VAULT_ADDR, VAULT_ROOT_TOKEN
#source "$(pwd)/managed-k8s/actions/vault_env.sh"
# Optional: Useful to be able to interact with the cluster via kubectl.
KUBECONFIG="$(pwd)/etc/admin.conf"
export KUBECONFIG
# Optional: Set this variable to false to init new clusters with the newest commit
# on the default (devel) branch instead of the latest release.
# export MANAGED_K8S_LATEST_RELEASE=false
if [ -f "$KUBECONFIG" ] && ! yq -r '.users[0].user."client-certificate-data"' "$KUBECONFIG" | base64 -d | openssl x509 -checkend 186400 -noout >/dev/null; then
echo "======="
echo "WARNING: Your kubeconfig is expired or will expire within the next 24h. Please run ./managed-k8s/actions/k8s-login.sh to renew it"
echo "======="
fi