Terraform provider

The provider kcorehypervisor/kcore talks to the controller over gRPC, the same API kctl uses. It creates a cluster, installs and joins nodes, and then manages everything on it: VMs, networks, volumes, Ceph, Kubernetes clusters, ingress, host updates, and the Vault policies that protect the VMs. Use the release that matches the controller. For kcore 0.7.0 that is provider 0.8.0.

terraform {
  required_providers {
    kcore = {
      source  = "kcorehypervisor/kcore"
      version = "0.7.0"
    }
  }
}

provider "kcore" {
  controller_address = "controller.example.com:9090"
  tls_cert_path      = "/path/to/client.crt"
  tls_key_path       = "/path/to/client.key"
  tls_ca_path        = "/path/to/ca.crt"
  vault_address      = "vault.example.com:9191"   # only for kcore_vault_* resources
}

Each setting has an environment variable: KCORE_CONTROLLER_ADDRESS, KCORE_TLS_CERT, KCORE_TLS_KEY, KCORE_TLS_CA, KCORE_VAULT_ENDPOINT. insecure (KCORE_INSECURE) turns TLS verification off. Use it on a bench, not in production.

Resources

AreaResources
Cluster and nodeskcore_cluster (CA, sub-CA, controller and kctl certificates, written where Terraform runs), kcore_node (install, bootstrap controller or join), kcore_operator, kcore_ssh_key
Computekcore_vm (including gpus, vgpus, pci_devices), kcore_workload (containers), kcore_kubernetes_cluster, kcore_slurm_cluster, kcore_postgresql
Networkkcore_network, kcore_security_group, kcore_security_group_attachment, kcore_ingress_gateway, kcore_ingress_route, kcore_waf_policy
Storagekcore_volume, kcore_volume_snapshot, kcore_snapshot_policy, kcore_disk_layout, kcore_ceph_cluster, kcore_shared_filesystem, kcore_object_store, kcore_object_user
Updateskcore_cluster_version (move the cluster to a release tag), kcore_cluster_update (hand-written rollout)
Hardwarekcore_vgpu_host (NVIDIA, Intel, AMD vGPU hosts)
Backupkcore_vault_target, kcore_vault_policy, kcore_vault_replication (need vault_address)

Data sources read what exists: kcore_vm, kcore_node, kcore_nodes, kcore_gpus, kcore_kubernetes_cluster, kcore_kubernetes_clusters, kcore_slurm_cluster, kcore_postgresql, and list sources for volumes, security groups, SSH keys, operators, ingress, WAF policies, and cluster updates.

Roll out a release

This is the whole update. Change the version and apply. The controller resolves the tag, updates a canary node, compares it with the others for the soak time, then updates the rest one at a time. A failed node rolls back.

resource "kcore_cluster_version" "this" {
  version             = "v0.7.0"
  drain_vms           = true
  canary_soak_seconds = 14400   # 4 hours. 0 means 1 hour, or 24 hours across a NixOS release change
  canary_node         = "node-b"
}

no_canary = true skips the canary and the soak. Another version is refused while a rollout runs. Destroying the resource only forgets it. The nodes keep their version. promote and rollback stay kctl commands. See Cluster & node upgrades.

A VM, a protected disk, and a backup policy

resource "kcore_vm" "web" {
  name               = "web-1"
  cpu                = 4
  memory_bytes       = 8589934592
  storage_backend    = "zfs"
  storage_size_bytes = 53687091200
  image_url          = "https://images.example.com/debian-12.qcow2"
  image_sha256       = "<64-char hex digest>"
  nic { network = "production" }
}

resource "kcore_vault_target" "offsite" {
  name     = "offsite"
  backend  = "s3"
  location = "s3://vault-backups/prod"
}

resource "kcore_vault_policy" "nightly" {
  name         = "nightly"
  schedule     = "@daily"
  target       = kcore_vault_target.offsite.name
  vms          = [kcore_vm.web.name]
  keep_daily   = 14
  keep_weekly  = 8
  keep_monthly = 12
}

The Vault target here is S3 with Object Lock, so the bucket must be created with Object Lock on. See Ransomware resistance.

What stays in kctl

Terraform describes state. One-shot operations are commands: console, drain, migrate, node approval, certificate rotation and revocation, backup and restore runs, promote and rollback of an update. Destroying kcore_node drops it from state and leaves the installed disk alone.

A few fields are CLI and YAML only. kcore_vm.storage_backend takes filesystem, lvm, or zfs. Put a Ceph VM on the SAN with kctl create vm --storage-backend ceph or a VM manifest. A Ceph data disk is kcore_volume with storage_class = "ceph". antiAffinity, restartPriority, and nodeLabels are also manifest and CLI fields. YAML manifests and kctl cover them.