← Back to blog
NVIDIA Inception Program logo

kcore has been accepted into the NVIDIA Inception program for startups. For a small team building a hypervisor, it comes at the point where GPU work stops being a side feature. Two things shipped in 0.7.0 make that the right time: one NVIDIA card can now be shared by several VMs, and a Slurm cluster runs on the hosts themselves.

One card, several VMs

Until now a GPU went to one VM. That is fine for a workstation guest and wasteful for inference, where most guests need a few gigabytes of video memory, not a whole card. kcore now slices an NVIDIA card into vGPUs. You turn a node into a vGPU host once, and each VM asks for a profile:

kctl node vgpu enable node-193 --host-card 0000:61:00.0 \
  --url http://mirror.lan/NVIDIA-Linux-x86_64-595.91.04-vgpu-kvm-aie.run \
  --sha256 2bc7f9be... --file NVIDIA-Linux-x86_64-595.91.04-vgpu-kvm-aie.run

kctl create vm infer-01 --vgpu A2-4C ...

The node downloads the NVIDIA Virtual GPU Manager, checks its SHA256, and loads it from its Nix configuration. The .run file never goes into git. If the card is still on nouveau, the command schedules the reboot, and a reboot is refused while a VM is meant to be running unless you force it. Leave the target node empty and the controller places the VM on a ready node that can supply the profile.

The same record is kind: NodeVgpu for kctl apply, kcore_vgpu_host and kcore_vm.vgpus in the Terraform provider, and a node-vgpu kind in the MCP server, so an agent can see which slices are free. The vGPU manager needs an NVIDIA AI Enterprise entitlement.

Intel and AMD cards use plain SR-IOV with no licence. intel-2vf is half an Intel card and radeon-4vf a quarter of an AMD one, with a function count per card. The manifest is the same as for NVIDIA.

The patches go to Cloud Hypervisor

kcore runs guests on Cloud Hypervisor. NVIDIA's vGPU manager was written against QEMU, so making an A2 slice boot took eight small patches to Cloud Hypervisor and its vfio-ioctls crate. Each one fixes a place where Cloud Hypervisor and QEMU behave differently:

Carrying patches means rebuilding Cloud Hypervisor on every release, and it means kcore users run code nobody upstream has reviewed. So the plan is to submit them. The two vfio-ioctls changes go to rust-vmm first, because one Cloud Hypervisor patch calls the new function. The rest follow, each opened as an issue first, as Cloud Hypervisor's contribution guide asks. Each one will say plainly that it was tested on one NVIDIA A2. The eighth patch keeps a QEMU-style -uuid on the process command line because the vGPU manager reads it from there. That is a workaround for the vendor manager, not something for Cloud Hypervisor's main, so kcore keeps carrying it while we look for a cleaner fix.

Slurm on the bare metal

HPC and training jobs do not want a VM between them and the hardware. A SlurmCluster runs Slurm as kcoreOS host services: a controller, an optional backup controller, login nodes, accounting with slurmdbd and MariaDB, and compute nodes.

kind: SlurmCluster
metadata:
  name: hpc-gpu
spec:
  clusterName: hpc-gpu
  controller: { nodeId: node-101, address: 10.0.0.11, iface: enp6s0 }
  accounting: { enabled: true, nodeId: node-104 }
  gpuDriver: true
  compute:
    - nodeId: node-103
      address: 10.0.0.13
      iface: enp6s0
      partitions: [batch, gpu]
      gres: auto
kctl slurm plan -f hpc-gpu.yaml
kctl apply -f hpc-gpu.yaml

Compute nodes are cordoned, so the VM scheduler leaves them to slurmd. Plan refuses a compute node that still hosts guests. The controller, login, and accounting roles can share a node with VMs. gres: auto writes gres.conf from the GPU inventory the controller already has, and plan fails rather than writing a config that cannot start if the NVIDIA driver is off.

Cluster updates know Slurm's version rules: accounting first, then the controller, then compute, then login, and each compute node is drained before its rebuild. The munge key and the database password are generated on the controller and never returned by the API. The dashboard, the MCP server, and the Terraform provider cover the same resource, so one kcore estate can hold VMs, vGPU guests, and a batch cluster.

If you run inference or HPC workloads on VMware or bare Slurm today and want to try this on your own hardware, write to team@tacconiconsulting.com. The GPU guide covers passthrough and vGPU, and the Slurm guide has the full manifest reference.