Skip to main content

CNDI Config Reference

This is a guide that explains the different properties of the YAML object CNDI takes as config. The following tables refer to the properties of the corresponding top-level keys. To use CNDI, you need to author a config file written in the syntax described below. For an interactive configuration experience check out our config tool!

global

Under the top-level key global you can set global configuration properties

ParameterTypeDefault valueDescription
debugBooleanfalseActivate debug mode during the deployment. This will display additional diagnostic information which may be useful to troubleshoot potential issues that may arise during different stages of deployment. Note that various authentication information may be displayed in plain text during the operation.

nodes

The top-level key nodes is a YAML array of Node objects, the table below shows the parameters a single Node object can contain

ParameterTypeUsed for kindDefault ValueDescription
ignoreBoolean*falseTells CNDI whether or not to ignore the node definition for which it is specified. This is useful for disabling parts of your config.
name*String*undefinedNode name. The supplied value should correspond to the hostname portion of the FQDN of the host only and must conform to the RFC 1123. In case of local/remote deployments (see kind* below), the user must ensure that the name can be resolved by the system DNS resolver prior to node deployment.
kind*local | remote| aws| azure| gcp|multipass|vmware*undefinedThe type of node where the CNDI environment is to be deployed.
role*controller|worker*undefinedNode cluster role. Each cluster has to contain a minimum of a single controller node. For non-HA clusters, additional nodes have to be specified as worker nodes. In HA cluster configurations this parameter governs only the node on which Apache Airflow and components are deployed, as effectively all nodes in such clusters serve the purpose of a controller.
hostStringremoteundefinedIP address or FQDN of the remote host on which CNDI is/has been deployed. Required if the kind is remote
labelsString[]*undefinedThe array contains the labels and their values that will be applied to the specific node after CNDI deployment. The practical use of this option is to differentiate between nodes and their capabilities using a simple key-value pair. This would allow the user to assign tasks to specific nodes, e.g. a node with compute/graphics acceleration for AI/ML tasks, node with SSDs for data processing, etc.
ssh portIntremote22SSH remote port
usernameStringremote"cndi"Remote user which CNDI can use to connect to the target deployment node. The user must have ability to use sudo.
passwordStringremote"ask"Remote credentials for CNDI access. If ssh key is not specified, this password is used to both connect and obtain root access. If ssh key is specified, this password is only used to obtain root access.
ssh keyStringremote""Absolute path to the SSH private key file used to connect to the remote node, associated with the user specified in the username key. If the key is protected with a password, this needs to be specified using the key ssh key password
ssh key passwordStringremote""SSH key password associated with the private key used to connect to the remote node. If set to “ask”, CNDI will prompt the user for the password during deployment (the password will not be saved).
vcenter urlStringvmwarevcenter.urlURL for connecting to the vCenter server (API)
vcenter insecure connectionBooleanvmwarevcenter.insecure connectionCheck validity of SSL certificates on the target vCenter server. If API server certificates are invalid, deployment will fail, unless this value is set to “true”
vcenter usernameStringvmwarevcenter.usernameUsername with VM creation privileges in the target ESXi server/cluster.
vcenter passwordStringvmwarevcenter.passwordPassword associated with the account.
vcenter tls cert pathStringvmwarevcenter.tls cert pathOverride system root certificate authorities by specifying path(s) to the certificate files (colon separated list of absolute paths)
vcenter tls known hostsStringvmwarevcenter.tls known hostsFiles for thumbprint based authentication, can be used exclusively or in combination with tls cert path.
vcenter datacenterStringvmwarevcenter.datacenterDatacenter in which the virtual machine and the resources will be allocated.
vm memoryIntvmwarevcenter.vm memoryAmount of memory (RAM) assigned to the virtual machine in megabytes (MB).
vm cpuIntvmwarevcenter.vm cpuNumber of vCPU cores assigned to the virtual machine.
vm disk sizeStringvmwarevcenter.vm disk sizeBoot disk size, with GB denoted as G ie: 30G.
vm networkStringvmwarevcenter.vm networkThe name of the virtual switch to which the virtual machine should be connected to.
vm datastoreStringvmwarevcenter.vm datastoreDatastore where the VM’s disk file will be stored.
vm respoolStringvmwarevcenter.vm respoolVM Resource Pool - i.e. the set of vCenter managed resources where the VM is going to be deployed in.
ova deployment imageStringvmwarevcenter.ova deployment imageLinux image deployed to the newly created VM. The default image (Ubuntu 20.04 LTS) will be downloaded if not found at the default location. User specified images have to be downloaded manually by the user prior to deployment.
gcp subnetStringgcpgcp.subnetNetwork subnet to which the new instances should be connected to. If not specified, the default subnet will be used.
gcp compute zoneStringgcpgcp.compute zoneCompute zone to which the instances must be provisioned.
gcp machine typeStringgcpgcp.machine typeInstance machine type - see link for reference.
gcp image projectStringgcpgcp.image projectGCP project containing the custom or pre-built image that should be used for deployment, used to locate the deployment image.
gcp image familyStringgcpgcp.image familyGCP image family, used to locate the deployment image - see link for reference.
gcp boot disk sizeStringgcpgcp.boot disk sizeInstance boot disk size in valid units.
gcp node imageStringgcpgcp.node imageName of the image that should be used for deployment in place of default.
gcp node public ipStringgcpThe static, public IP address which will be assigned to the deployed instance during the deployment. If not specified, an ephemeral public IP address is automatically provisioned during the deployment by GCP. This option is used primarily for enabling the correct certbot behaviour, as automatic SSL certificate generation requires a valid DNS record tied to the public IP address of the CNDI controller. This requires the user to manually reserve a static, publicly routable IP address and create the relevant DNS records prior to deployment.
aws regionStringawsaws.regionValid EC2 region name. The region has to be accessible to the user whose credentials are provided in the CNDI configuration, or used with the AWS CLI. If not specified, the default region specified for the AWS CLI will be used, and if that is not defined, us-east-1 (N. Virginia) will be used.
aws availability zoneStringawsaws.availability zoneEC2 Availability zone name. The specified availability zone needs to exist in the specified region. If this is not the case, the first availability zone in the selected region will be used.
aws instance typeStringawsaws.instance typeEC2 instance type. The selected instance type must be available in the selected region and availability zone. Minimum recommended deployment is t2.large, but this will depend on the type of orchestrated tasks.
aws boot disk sizeStringawsaws.boot disk sizeSize of the AMI boot disk in GB. Smallest size is 8 GB.
aws subnet idStringawsaws.subnet idAWS Subnet ID. If empty, CNDI will use the default subnet for the selected availability zone.
aws create extra flagsStringawsaws.create extra flagsAdditional flags to be supplied to AWS CLI during creation of the instance. See link for reference.
aws eip allocidStringawsThe AWS Elastic IP AllocationId for the specific external, public IP address which should be associated with the instance during the deployment. If not specified, a random public IP address is automatically provisioned during the deployment by the VPC. This option is used primarily for enabling the correct certbot behaviour, as automatic SSL certificate generation requires a valid DNS record tied to the public IP address of the CNDI controller. This requires the user to manually reserve a static, publicly routable IP address and create the relevant DNS records prior to deployment.
az machine typeStringazureazure.machine typeAzure machine size. For details see link.
az subnetStringazureazure.subnetVirtual network subnet name (existing or new)
az node imageStringazureazure.node imageNode image used for deployment in place of default.
az public ipStringazureThe name of an existing external/public IP address object in the target resource group. This parameter needs to be unique to a specific node and cannot be shared. The default action is to automatically create a public IP Address during deployment. This option is used primarily for enabling the correct certbot behaviour during the deployment, as SSL certificate generation requires a valid DNS record tied to the public IP address of the CNDI controller.

Deployment Options

The following is a list of all the environments CNDI can deploy to, and the related configuration properties for each deployment environment

aws

Under the top-level key aws you can set properties that relate to your CNDI deployment on Amazon Web Services

ParameterTypeDefault ValueDescription
iam access keyString | "skip"""AWS Access Key, or skip if the pre-configured AWS CLI profile will be used (skips authentication check and relies on AWS CLI).
iam access secretString""AWS Access Secret as a plain-text value.
profileString"default"AWS CLI profile, if required. If iam access secret is not provided, it needs to be pre-configured prior to CNDI deployment.
regionString""Valid EC2 region name. The region has to be accessible to the user whose credentials are provided in the CNDI configuration, or used with the AWS CLI. If not specified, the default region specified for the AWS CLI will be used, and if that is not defined, us-east-1 (N. Virginia) will be used.
availability zoneString""EC2 Availability zone name. The specified availability zone needs to exist in the specified region. If this is not the case, the first availability zone in the selected region will be used.
instance typeString"t2.large"EC2 instance type. The selected instance type must be available in the selected region and availability zone. Minimum recommended deployment is t2.large, but this will depend on the type of orchestrated tasks.
boot disk sizeString"30"Size of the AMI boot disk in GB. Smallest size is 8 GB.
subnet idString""AWS Subnet ID. If empty, CNDI will use the default subnet for the selected availability zone.
security group idString""AWS Security Group ID. If none is specified, default security group associated with the VPC will be used.
update security groupBooleanfalseUpdate the selected (or default) security group’s ingress rules to match configuration in the ingress key, otherwise allow only SSH access to AWS nodes.
create extra flagsString""Additional flags to be supplied to AWS CLI during creation of the instance. See link for reference.
cloud-init dataString"templates/cloud-config.yml"Cloud-init configuration for the new instance. If a custom one is required it should be based off the template provided.

gcp

Under the top-level key gcp you can set properties that relate to your CNDI deployment on Google Cloud Platform

ParameterTypeDefault ValueDescription
accountString""Email address associated with the GCP account that should be used to provision resources.
projectString""Valid GCP project id. This needs to be pre-created in the GCP console prior to deployment.
subnetString"default"Network subnet to which the new instances should be connected to. If not specified, the default subnet will be used.
compute zoneString“us-central1-a”Compute zone to which the instances must be provisioned.
machine typeString"e2-medium"Instance machine type - see link for reference.
image projectStringgcp.projectGCP project containing the custom or pre-built image that should be used for deployment, used to locate the deployment image.
image familyString""GCP image family, used to locate the deployment image - see link for reference.
boot disk sizeString"30GB"Instance boot disk size in valid units.
node imageString“cndi-ubuntu-minimal-2004-lts”Name of the image that should be used for deployment in place of default.
update firewall tagsBooleanfalseUpdate node firewall tags to match the configuration set via the ingress key. If set to False, only SSH access will be allowed to the GCP nodes.
cloud-init dataString“templates/cloud-config.yml”Cloud-init configuration for the new instance. If a custom one is required it should be based off the template provided.

azure

Under the top-level key azure you can set properties that relate to your CNDI deployment on Azure

ParameterTypeDefault ValueDescription
accountString""Valid Azure username with appropriate rights to create the required compute resources in the selected region. Mutually exclusive with service principal.
passwordString""Account password. If Azure CLI has been preconfigured by the user, there is no need to specify the password in the CNDI configuration file.
service principalString""Azure Service Principal. If supplied, then service credentials need to be provided as well as the Azure tenant ID for the tenant key. Mutually exclusive with user.
service credentialsString""Azure service credentials - a valid password or a X509 certificate. Required only if service principal has been supplied.
tenantString""Azure Tenant. Required only if service principal has been supplied.
subscriptionStringautomatically determinedAzure subscription ID associated with the tenant. Used to select a particular tenant in a multi-tenant environment.
resource groupString"CNDI"Azure Resource Group to be associated with the instance.
locationString"eastus"Azure datacenter region. For details see link.
machine typeString"Standard_D2s_v3"Azure machine size. For details see link.
subnetString"deployment"Virtual network subnet name (existing or new)
vnetString"cndi-vnet"Virtual network name (existing or new)
node imageString“canonical:0001-com-ubuntu-server-focal:20_04-lts-gen2:latest”Node image used for deployment in place of default.
update net security groupBooleanfalseUpdate the default security group for the new deployment to allow access to ports 80 and 443. This option will apply the same ruleset to the subnet and the cluster controller node’s network interface. When false, only SSH traffic is permitted through to all the nodes (Azure default configuration).
cloud-init dataString“templates/cloud-config.yml”Cloud-init configuration for the new instance. If a custom one is required it should be based off the template provided.

vcenter

Under the top-level key vcenter you can set properties that relate to your CNDI deployment on vCenter

ParameterTypeDefault ValueDescription
urlString""URL for connecting to the vCenter server (API)
insecure connectionBooleanfalseCheck validity of SSL certificates on the target vCenter server. If API server certificates are invalid, deployment will fail, unless this key is set to “true”
usernameString"ask"Username with VM creation privileges in the target ESXi server/cluster.
passwordString"ask"Password associated with the account.
tls cert pathString""Override system root certificate authorities by specifying path(s) to the certificate files (colon separated list of absolute paths)
tls known hostsString""Files for thumbprint based authentication, can be used exclusively or in combination with tls cert path.
datacenterString""Datacenter in which the virtual machine and the resources will be allocated.
vm memoryInt4096Amount of memory (RAM) assigned to the virtual machine in megabytes (MB).
vm cpuInt2Number of vCPU cores assigned to the virtual machine.
vm disk sizeString"20G"Boot disk size, with GB denoted as G.
vm networkString""The name of the virtual switch to which the virtual machine should be connected to.
vm datastoreString""Datastore where the VM’s disk file will be stored.
vm respoolString""VM Resource Pool - i.e. the set of vCenter managed resources where the VM is going to be deployed in.
ova deployment imageString“files/focal-server-cloudimg-amd64.ova”Linux image deployed to the newly created VM. The default image (Ubuntu 20.04 LTS) will be downloaded if not found at the default location. User specified images have to be downloaded manually by the user prior to deployment.
cloud-init dataString“templates/cloud-config.yml”Cloud-init configuration for the new instance. If a custom one is required it should be based off the template provided.

multipass

ParameterTypeDefault ValueDescription
vm cpuString"2"Number of vCPU cores assigned to the virtual machine.
vm memoryString4GAmount of memory allocated to the virtual machine, with gigabytes denoted as G.
vm disk sizeString"30G"Boot disk size, with GB denoted as G.
cloud-init dataString“templates/cloud-config.yml”Cloud-init configuration for the new instance. If a custom one is required it should be based off the template provided.

airflow

Under the top-level key airflow you can set properties that define the behaviour of the Airflow cluster that CNDI will deploy

ParameterTypeDefault ValueDescription
expose configBooleanfalseControls whether or not Airflow configuration will be exposed to the users in the Airflow Web UI.
dag dir list intervalInt30Default local DAG folder refresh interval in seconds, regardless of the DAG repository type.
delete worker pods on failureBooleanfalseDelete worker pods on failure. Failure here implies Kubernetes-related issues (such as lack of resources to execute pods, unavailable nodes, etc.)
delete worker podsBooleantrueDelete worker pods after job completion. May be required for debugging/troubleshooting purposes.
worker pods creation batch sizeInt10Number of worker pods generated per scheduler execution cycle. Value should be adjusted down for less powerful VMs, and depends on the type of scheduled workloads (long-lived vs. short-lived)
api auth backendString"airflow.api.auth.backend.deny_all"The default authentication backend for Airflow API access. The default value disables Airflow API. A list of valid values for this key can be found here.
log retention daysString"15"Number of days to retain the internal Airflow scheduler logs for. Note that this is different from the Airflow task logs. If this value is too high, free disk space may be depleted on the nodes running the Scheduler.
imageString"apache/airflow:2.1.4-python3.8"Docker Hub repository for the Apache Airflow docker image.
templateString“templates/airflow2-setup-template.yml”Path to the alternative Airflow deployment template file, allowing the user to manually configure the Airflow deployment manifest.
pod templateString“templates/airflow2-pod-template.yml”Path to the alternative Kubernetes pod template file used to generate Airflow task pods, allowing the user to manually configure an alternative deployment manifest, if required.
schedule with airflow podsBooleantrueControls whether or not Airflow tasks should be scheduled on the nodes running Airflow components (scheduler, webserver). By default (true), this is a preferred affinity, i.e. the tasks will only be scheduled if all other nodes are resource constrained or become unavailable. Setting this to false will prevent Airflow tasks from being scheduled on the nodes running Airflow components entirely. If ha scheduling.type is set to auto, this option will be forced to true to allow Airflow tasks to execute, as all nodes in that configuration (aside from those running hop-web) are used to run Airflow components.
schedule hopweb nodesBooleanfalseSchedule Airflow tasks on nodes executing hop-web pods. This is generally undesirable due to high hop-web resource requirements.
dag storageAirflow.DagStorage
webui-authAirflow.WebUIAuth
databaseAirflow.Database

Airflow.WebUIAuth

the Airflow.WebUIAuth entry defines the type of Authentication in front of the Airflow Web UI

ParameterTypeDefault ValueDescription
typeString"rbaconly"Airflow Web UI authentication type
admin userString“admin”Airflow admin user login. Must conform to standard login specifications (allow alphanumeric characters plus dash - and underscore _
admin passwordStringAutomatically generatedAirflow admin user’s password. If specified, should be encapsulated in double quotes. Default if not specified is to autogenerate a password (displayed at the end of deployment)
admin emailString"admin@airflow.local"Airflow admin user’s email address. Must conform to email specification RFC5322

Airflow.DagStorage

the Airflow.DagStorage entry defines how user defined DAGs will be stored and accessed by Airflow, some config is only required for specific types

ParameterTypeUsed With TypeDefault ValueDescription
typelocal|nfs|custom|git*localDetermines the type of storage used for the Airflow DAG repository. Note that local deployments are only sensible for single node deployments, as the volume has to be accessible to every Airflow worker pod on task initialization.
templateString*"templates/pv-pvc-template.yml"Path to the storage template used for creating the DAG persistent volume and volume claim, should the user wish to customize the default template.
pathStringlocal""Absolute path to the local DAG storage directory. This will be created should it not exist on the target node. The user can then manually copy/upload DAGs to this location. To prevent data loss, make sure that this folder is regularly backed up.
capacityStringlocal"1Gi"Resource allocation for the DAG persistent volume. Units as per document.
pathStringnfs""Absolute path to the local DAG storage directory. This will be created should it not exist on the target node. The user can then manually copy/upload DAGs to this location. To prevent data loss, make sure that this folder is regularly backed up.
claimnameStringcustom""A valid Persistent Volume Claim name for the DAG storage Persistent Volume. The Persistent Volume has to be created either through the config objects key in the CNDI configuration, or manually, post-deployment.
git repoStringgit""Git repository URL as quoted string.
git branchStringgit"main"Git repository branch. Typically either “master” or “main”, but user may specify any valid branch reference.
ssh privkeyStringgit""Absolute path reference to the SSH private key file in unix std. notation. Must be accessible to CNDI.
ssh known hostsBooleangittrueUse known_hosts to determine if the target SSH server signature is authentic/unchanged. CNDI will automatically look for the appropriate value for the SSH key signature.
git userStringgit""Git SSH authentication user - mutually exclusive with ssh privkey
git passwordStringgit""Git SSH authentication password - mutually exclusive with ssh privkey. Must be specified if git user is specified.

Airflow.Database

ParameterTypeDefault ValueDescriptionRequired
userString“airflowdb”Database connection user.if AirflowDatabase.host key is specified.
passwordStringAutomatically GeneratedDatabase connection password. If left out of the configuration, CNDI will generate a password for you.if AirflowDatabase.host key is specified.
hostString""Database server FQDN or IP address. If left empty, CNDI will deploy an in-cluster Postgres instance.if AirflowDatabase.host is not specified, it is assumed that CNDI will deploy a Postgres Database inside the cluster.
portInt5432Database server port numberfalse
nameString"airflow"Database namefalse
templateString"templates/pgsql-template.yml"Kubernetes manifest for postgres database deployment. If a custom deployment is desired, the user is responsible for configuring the database storage resources within the manifest as this will not be done automatically by CNDI.false
storageAirflow.DatabaseStorage

Airflow.DatabaseStorage

ParameterTypeUsed With TypeDefault ValueDescription
typelocal | nfs| custom*localType of persistent volume to be used for storing Airflow database, configuration and logs.
pathStringlocal"/cluster/airflow-postgres-data"Absolute path for where to store the postgres data on the controller node
capacityString*"5Gi"Resource allocation for the database storage persistent volume. Units as per document.
claimnameStringcustom""Kubernetes Persistent Volume Claim name
pathStringnfs"/cluster/airflow-postgres-data"A valid nfs path for where to store the postgres data

Hop Options

hop-web

ParameterTypeDefault ValueDescription
installBooleanfalseProvision hop-web to the deployed cluster.
templateString"templates/hopweb-setup-template.yml"Path to the alternative hop-web deployment template file, allowing the user to manually configure the hop-web deployment manifest.
imageString“apache/incubator-hop-web:latest”Docker image and tag for the hop-web container used for the deployment.
users[HopUser][]A single user definition is sufficient for CNDI to provision a hop-web instance in the cluster. The list of possible keys and their default values is provided below.

HopUser

ParameterTypeDefault ValueDescription
nameString"user"Name of the unique instance, which will be used to access the user’s environment via the url “https://<host>/hop-web/<name>”. Has to conform to RFC3986 specification for URIs (allowing only alphanumeric characters, underscore _ and dash - symbols.
nodeString""Name of the node, with a matching entry under nodes, that will be used to deploy hop-web. If allocating multiple hop-web instances (users) to the same cluster node, please take note of hop-web hardware requirements. This key needs to be present in every user definition.
storage type"none" | "local" | "nfs" | "custom"""Provision an instance-specific volume for the specified user. This folder is not visible to other hop-web users. This key specifies the type of persistent storage to use for storage volume.
storage pathString“/cluster/hop-web/<name>”Path to the folder containing instance-specific data.
storage capacityString"5Gi"Resource allocation for the storage persistent volume. Units as per document.
storage claimString""Kubernetes Persistent Volume Claim name, specified only if storage type=custom
shared storageBooleantrueProvision shared storage between hop-web deployments and Airflow. Defaults to Airflow scratch settings (see airflow.scratch above)

Auth Providers

basicauth

ParameterTypeDefault ValueDescription
realmString"CNDI hop-web"Message displayed to user on authentication
userString"admin"Username used for basic authentication mechanism
passwordString""Admin password for basic auth authentication mechanism. If left blank CNDI will automatically generate a password and display on deployment completion.
htpasswd pathString""Path to the standard Apache HTTPD password flat file containing allowed login/password combinations for site ingress. It is typically generated using the htpasswd utility. If specified, it will take precedence over the values specified for the admin user and admin password keys.

oauth-google

ParameterTypeDefault ValueDescription
client idString""Google OAuth Client ID
client secretString""Google OAuth Client Secret
email domainString""Restrict OAuth authentication to a specific domain (e.g. example.com)
templateString"templates/oauth2-proxy-install-template.yml"Path to the custom template for Google OAuth configuration of the oauth2-proxy and nginx-ingress service.

oauth-azure

ParameterTypeDefault ValueDescription
client idString""Azure AD Application ID
client secretString""Azure AD Application Key
tenantString""Azure Tenant ID
email domainString""Restrict OAuth authentication to a specific domain (e.g. example.com)
templateString"templates/oauth2-proxy-install-template.yml"Path to the custom template for Google OAuth configuration of the oauth2-proxy and nginx-ingress service.
proxyargsString“--session-cookie-minimal=true”Space-delimited list of additional parameters to be passed on to oauth2-proxy. If additional settings are specified, the default argument should be kept.

Options

logs

ParameterTypeDefault ValueDescription
type"local" | "nfs" | "custom""local"Type of persistent volume to be used for storing Airflow logs.
pathString"/cluster/logs"Absolute path (on the controller node, type=local) or a valid NFS mount path (type=nfs) for the postgres data. Note that local log storage is only usable with a single node cluster and is not supported for multi-node clusters.
capacityString"5Gi"Resource allocation for the log storage persistent volume. Units as per document.
templateString“templates/pv-pvc-template.yml”Path to the storage template used for creating the Airflow log persistent volume and volume claim, should the user wish to customize the default template. Used only for type=local and type=nfs.
claimnameString""Kubernetes Persistent Volume Claim name, specified only if type=custom

scratch

ParameterTypeDefault ValueDescription
type"local" | "nfs" | "custom""local"Type of persistent volume to be used for scratch storage (temporary shared storage between Airflow worker pods). Type “none” disables scratch storage.
pathString"/cluster/scratchvol"Absolute path (on the controller node, type=local) or a valid NFS mount path (type=nfs) for the postgres data. Note that local scratch storage is only usable with a single node cluster and is not supported for multi-node clusters.
capacityString"5Gi"Resource allocation for the scratch storage persistent volume. Units as per document.
templateString“templates/pv-pvc-template.yml”Path to the storage template used for creating the scratch persistent volume and volume claim, should the user wish to customize the default template. Used only for type=local and type=nfs.
claimnameString""Kubernetes Persistent Volume Claim name, specified only if type=custom

smtp

ParameterTypeDefault ValueDescription
hostString""IP address or FQDN of the SMTP server used as the relay for system generated e-mails from the Airflow service.
starttlsBooleanfalseUse STARTTLS encryption for SMTP server authentication.
sslBooleanfalseUse SSL encryption for SMTP server authentication.
userString""Valid username used for SMTP server authentication.
passwordString""Password for SMTP server authentication.
portInt25SMTP server port. Will most likely need to be changed to a provider specific setting if using encrypted authentication methods.
mailfromString"admin@airflow.local"Email address that will appear on messages sent from the Airflow service.
retrylimitInt1Number of retries, should sending fail.

ingress

ParameterTypeDefault ValueDescription
type"none" | "noauth" | "basicauth" | "oauth-google" | "oauth-azure""none"
hostString""Hostname used for ingress. If using SSL (ssl key is True), it is mandatory to supply a FQDN corresponding to the supplied certificate, otherwise certificate warnings will be issued by the browser on accessing the Airflow Web UI.
sslBooleantrueUse Secure Sockets Layer (SSL) encryption for the ingress controller. Also controls the default firewall/security group permissions for deployments for port 443. If specified without specifying the SSL certificate configuration, automatically generated certificates will be used, which will result in certificate warning being raised by the browser on access.
ssl-redirectBooleantrueRedirect user accessing Airflow via HTTP/port 80 to HTTPS/port 443. Implies ingress.ssl=True
ssl-tlscrtString""Path to the file containing the certificate (public key and signature) for the host, in PEM encoding.
ssl-tlskeyString""Path to the file containing the private key for the host certificate, in PEM encoding.
cert-managercert-managerundefinedEntry contains info required for enabling cert-manager, which enables SSL automatically
basic authbasicauthundefinedEntry contains info required for enabling ingress.type basicauth
oauth-googleoauth-googleundefinedEntry contains info required for enabling ingress.type oauth-google
oauth-azureoauth-azureundefinedEntry contains info required for enabling ingress.type oauth-azure

cluster

Under the key cluster you can set cluster configuration properties

ParameterTypeDefault valueDescription
ha-enabledBooleantrueHigh Availability cluster configuration.
dashboardBooleantrueControls the automatic deployment of the kubernetes dashboard service, available at https://<host>/dashboard . Automatically enabled for all deployments except when ingress.type is set to “noauth”.
registry sizeString20GiSize of the cluster container registry
registryString[][]Array of valid container images that CNDI will push into the local registry. [ “hop”, “git”, “hop-web”, ...any valid image reference... ] The array can contain predefined images such as hop, hop-web and git, or it can contain full image references (e.g. “docker.io/alpine/git:latest”). Note that all image references must have a valid container registry URL (docker.io in case of Docker Hub, ghcr.io in case of GitHub Container registry, etc.)
image secretsString[][]Array of name referencesfor imagePullSecrets. The individual secrets must be added as appropriate Secret manifests using the config objects key.

tailscale

ParameterTypeDefault ValueDescription
use tailscaleBooleanfalseAutomatically install and join nodes to the private VPN network using tailscale. This is required for correct functioning of multi-cloud and hybrid-cloud deployments, as well as deployments where the network configuration prevents direct communication via a private network interface.
host addition keyString""Tailscale host addition key used to join an existing private VPN. Should be a Reusable key unless a single node is being added to the network.

cert-manager

ParameterTypeDefault ValueDescription
enabledBooleanfalseEnables configuration and installation of cert-manager with the default Let’s Encrypt free SSL certificate issuing service. This will allow CNDI to automatically provision an SSL certificate for the cluster on deployment. The principlaIf enabled, ssl-tlscrt and ssl-tlscrt keys must not be specified, and a valid ingress host FQDN must be specified (via the key host).
emailString""A valid email address which will be passed on to Let’s Encrypt service during registration. This email address will be used to notify the user about certificate expiration and renewal, as well as other Let’s Encrypt system messages.
issuerString"letsencrypt-staging"cert-manager issuer for the new certificate. CNDI has two preset issuers, but the user is able to replace these or add new ones using a custom issuer template (see the key cert-manager issuer template). The default issuer will generate an invalid temporary certificate signed by the staging CA, useful only for testing. To create a valid certificate using the Let’s Encrypt CA, specify the issuer as ‘letsencrypt-production’.
templateString“https://github.com/jetstack/cert-manager/releases/download/v1.4.0/cert-manager.yaml”cert-manager deployment template.
issuer templateString“templates/cert-manager-issuers.yml”Kubernetes manifest containing definitions of ClusterIssuer objects representing different providers used for obtaining an SSL certificate for cluster ingress. All issuers supported by cert-manager are also supported by CNDI for deployment.

config objects

The top-level key config objects is a YAML array of Kubernetes manifests

TypeDefault ValueDescription
String[][]Array of valid Kubernetes manifests, which can be represented by JSON notation as:
[
manifest name: { key: value, … },
another manifest: { key: value, … },
...
]