vSphere Provider¶

The vSphere provider manages virtual machines on VMware vSphere (vCenter Server and standalone ESXi) via the govmomi SDK. It is the most mature provider in the VirtRigaud tree and powers production deployments.

This page is aligned to VirtRigaud v0.3.8. Capability claims trace back to the provider's GetCapabilities response in internal/providers/vsphere/server.go.

Capabilities at a glance¶

The vSphere provider advertises the following via GetCapabilities (internal/providers/vsphere/server.go:372-383):

For the full cross-provider matrix and resilience / observability story, see:

• Provider capabilities matrix — every capability across all providers.
• Operations — Resilience — the v0.3.6 CircuitBreaker wraps every outbound RPC to this provider.
• Operations — Observability — metric families that surface vSphere-side state.

RPC support¶

The provider implements the full gRPC contract (proto/provider/v1/provider.proto):

• Validate — connectivity + credential check against vCenter, with a live probe + session reconnect (see Session resilience).
• Create — clone from a template/VM or import disk via OVF/OVA.
• Delete — VM teardown with disk cleanup.
• Power — On / Off / Reboot / Shutdown-Graceful.
• Describe — current state, IPs (via VMware Tools), ConsoleUrl, raw vSphere properties.
• Reconfigure — CPU / memory / disk changes (hot when possible).
• SnapshotCreate / SnapshotDelete / SnapshotRevert — disk-only.
• CloneCreate — full or linked.
• TaskStatus — polls the underlying govmomi Task until terminal. Counts toward the virtrigaud_provider_tasks_inflight gauge.
• ConsoleUrl — vSphere web client URL with VM instance UUID.
• ImagePrepare — OVF/OVA import and content-library deploy.
• ExportDisk / ImportDisk / GetDiskInfo — disk export and import advertising vmdk / qcow2 / raw plus export compression (#178). These back the cross-provider migration pipeline.

Prerequisites¶

• vCenter Server 7.0+ or ESXi 7.0+, reachable over HTTPS (port 443).
• A service account or API session token with rights for VM lifecycle, datastore, network, and resource management. See Service account permissions below.
• TLS: a valid CA-issued cert for vCenter, or insecureSkipVerify: true during development.

Authentication¶

The vSphere provider reads credentials from a Kubernetes Secret referenced via Provider.spec.credentialSecretRef. The provider controller mounts that Secret read-only at /etc/virtrigaud/credentials inside the provider pod. The Secret keys are read as files (internal/providers/vsphere/server.go:129-140), so the key names matter:

TLS verification is controlled by the Provider.spec.insecureSkipVerify field, which the controller exports to the pod as the TLS_INSECURE_SKIP_VERIFY env var. Leave it false in production.

Provider CR + Secret¶

apiVersion: v1
kind: Secret
metadata:
  name: vsphere-credentials
  namespace: virtrigaud-system
type: Opaque
stringData:
  username: "virtrigaud@vsphere.local"
  password: "REPLACE_ME"
---
apiVersion: infra.virtrigaud.io/v1beta1
kind: Provider
metadata:
  name: vsphere-prod
  namespace: virtrigaud-system
spec:
  type: vsphere
  endpoint: https://vcenter.example.com/sdk
  credentialSecretRef:
    name: vsphere-credentials
  insecureSkipVerify: false
  runtime:
    mode: Remote
    image: "ghcr.io/projectbeskar/virtrigaud/provider-vsphere:v0.3.8"
    service:
      port: 9443
      tls:
        enabled: true
        secretRef:
          name: provider-vsphere-tls
        insecureSkipVerify: false

Service account permissions¶

Create a dedicated vSphere user with these privileges:

• Datastore: AllocateSpace, Browse, FileManagement
• Network: Assign
• Resource: AssignVMToPool
• Virtual machine: full set (or a tailored subset covering Inventory, Interaction, Configuration, Provisioning, Snapshot management)
• Global: EnableMethods, DisableMethods, Licenses

TLS / mTLS (v0.3.7+)¶

Starting in v0.3.7, the manager enforces that every Provider CR has a spec.runtime.service.tls block. A Provider without this block fails to reconcile and its status will show TLSConfigured=False, Reason=TLSBlockMissing — no Deployment is created.

For full mTLS details see Security — mTLS.

spec.runtime.service.tls fields¶

TLS material mounts at /etc/virtrigaud/tls inside the provider pod. Both manager and provider pin TLS 1.3. A missing Secret with tls.enabled: true sets TLSConfigured=False, Reason=SecretRefMissing.

The TLSConfigured status condition reasons are:

Example Secret¶

apiVersion: v1
kind: Secret
metadata:
  name: provider-vsphere-tls
  namespace: virtrigaud-system
type: kubernetes.io/tls
data:
  tls.crt: LS0tLS1CRUdJTi...  # base64-encoded cert
  tls.key: LS0tLS1CRUdJTi...  # base64-encoded key
stringData:
  ca.crt: |
    -----BEGIN CERTIFICATE-----
    # Your CA certificate
    -----END CERTIFICATE-----

Endpoint formats¶

The Provider.spec.endpoint is exposed to the provider pod as PROVIDER_ENDPOINT.

Cloud-init on vSphere¶

vSphere does not use a NoCloud ISO. The provider injects cloud-init via VMware's guestinfo OVF properties (internal/providers/vsphere/server.go:1321-1380). The keys it sets are:

The guest must have a recent cloud-init that supports the VMware datasource (Ubuntu cloud images and most distro cloud images do).

Datastore selection (precedence)¶

VirtRigaud supports three ways to choose a datastore. Precedence is strict — higher entries win:

StoragePod (Datastore Cluster) auto-selection¶

When a StoragePod is configured, the provider enumerates the cluster's member datastores via the vSphere API, retrieves the FreeSpace summary for each, and picks the member with the most free space (internal/providers/vsphere/server.go:205-262, resolveDatastoreFromStoragePod). This is a lightweight alternative to Storage DRS and does not require SDRS to be enabled.

Configuring a default StoragePod¶

Via the Provider CR:

apiVersion: infra.virtrigaud.io/v1beta1
kind: Provider
metadata:
  name: vsphere-prod
spec:
  type: vsphere
  endpoint: https://vcenter.example.com/sdk
  credentialSecretRef:
    name: vsphere-credentials
  defaults:
    storagePod: "DatastoreCluster-SSD"
    cluster: "Compute-Cluster"
    folder: "/vm/applications"

Per-VM placement¶

apiVersion: infra.virtrigaud.io/v1beta1
kind: VirtualMachine
metadata:
  name: web-application
spec:
  providerRef:
    name: vsphere-prod
  # ... classRef, imageRef, networks, etc.
  placement:
    cluster: "Compute-Cluster"
    resourcePool: "Production"
    folder: "/vm/applications"
    storagePod: "DatastoreCluster-SSD"   # ignored if datastore: is set
    # datastore: "datastore-ssd"          # explicit override wins

SCSI controller configuration (vSphere only)¶

v0.3.6 ships an explicit SCSIControllerSpec on each disk entry. It is vSphere-only — libvirt and proxmox ignore it. Reference: api/infra.virtrigaud.io/v1beta1/virtualmachine_types.go:422-442.

disks:
  - name: data
    sizeGiB: 500
    type: thin
    scsi:
      controller: 1                # bus 0-3 (default: first available)
      controllerType: pvscsi       # pvscsi | lsilogic | lsilogic-sas | buslogic
      sharedBus: virtualSharing    # noSharing | virtualSharing | physicalSharing

When to use the scsi block:

• RDM (raw device mapping) or shared cluster disks: pvscsi with sharedBus: virtualSharing (for in-cluster shared) or physicalSharing (for cross-host shared).
• More than 15 disks on a single VM: each SCSI controller holds up to 15 devices; spread disks across multiple controllers (controller: 0, controller: 1, ...) for >15.
• Legacy guests: drop to lsilogic or buslogic for guests that lack the pvscsi driver.

If you do not set scsi, the provider creates a single pvscsi controller with noSharing and attaches every disk to it.

VMClass¶

VMClass defines CPU/memory/firmware defaults and vSphere-specific tuning via extraConfig:

apiVersion: infra.virtrigaud.io/v1beta1
kind: VMClass
metadata:
  name: standard-vm
spec:
  cpu: 4
  memory: "8Gi"
  firmware: UEFI
  diskDefaults:
    type: thin
    size: "40Gi"
  guestToolsPolicy: install
  performanceProfile:
    cpuHotAddEnabled: true
    memoryHotAddEnabled: true
    latencySensitivity: normal
  securityProfile:
    secureBoot: true
    tpmEnabled: true
  resourceLimits:
    cpuReservation: 1000             # MHz
    memoryReservation: "2Gi"
  extraConfig:
    "numvcpus.coresPerSocket": "2"   # CPU topology hint

See the full CRD reference for every field.

VMImage¶

Reference vSphere templates, content library items, or OVF/OVA:

apiVersion: infra.virtrigaud.io/v1beta1
kind: VMImage
metadata:
  name: ubuntu-22-04-template
spec:
  source:
    template: "ubuntu-22.04-template"
    folder: "/vm/templates"
  guestOS: "ubuntu64Guest"

Complete VM example¶

apiVersion: infra.virtrigaud.io/v1beta1
kind: VMNetworkAttachment
metadata:
  name: app-network
spec:
  network:
    vsphere:
      portgroup: "VM Network"
---
apiVersion: infra.virtrigaud.io/v1beta1
kind: VMNetworkAttachment
metadata:
  name: mgmt-network
spec:
  network:
    vsphere:
      portgroup: "Management"

---
apiVersion: infra.virtrigaud.io/v1beta1
kind: VirtualMachine
metadata:
  name: web-application
spec:
  providerRef:
    name: vsphere-prod
  classRef:
    name: standard-vm
  imageRef:
    name: ubuntu-22-04-template
  powerState: On
  disks:
    - name: data
      sizeGiB: 500
      type: thin
      scsi:
        controllerType: pvscsi

  # Network configuration (references VMNetworkAttachment resources).
  # Static addressing is requested inline here (pushed to the guest via
  # guestinfo.network.* — see "Static IP Assignment" below). The mgmt NIC omits
  # these fields and therefore uses DHCP / the template default.
  networks:
    - name: app
      networkRef:
        name: app-network
      ipAddress: "192.168.100.50"
      prefix: 24
      gateway: "192.168.100.1"
      dns: "192.168.1.10,8.8.8.8"
    - name: mgmt-network
      networkRef:
        name: mgmt-network

  # vSphere placement (no 'datacenter' field — datacenter is derived from the provider endpoint)
  placement:
    cluster: "Compute-Cluster"
    resourcePool: "Production"
    folder: "/vm/applications"
    storagePod: "DatastoreCluster-SSD"
  userData:
    cloudInit:
      inline: |
        #cloud-config
        hostname: web-application
        users:
          - name: ubuntu
            sudo: ALL=(ALL) NOPASSWD:ALL
            ssh_authorized_keys:
              - "ssh-ed25519 AAAA..."
        packages:
          - nginx
          - open-vm-tools
        runcmd:
          - systemctl enable --now nginx open-vm-tools

Async task tracking¶

vSphere operations that are long-running (clone, snapshot, reconfigure-with-large-disk-add) return a govmomi.Task reference. VirtRigaud wraps that as a proto TaskRef in the gRPC response and the manager polls TaskStatus until terminal.

Since v0.3.6, every in-flight task is counted by the virtrigaud_provider_tasks_inflight{provider_type="vsphere", provider="<name>"} gauge (G7.3 / PR #130). The gauge is seeded to 0 at boot so it appears on /metrics from the first scrape; useful for catching stuck tasks. See Observability.

Session resilience (v0.3.8)¶

As of v0.3.8 (#190), the vSphere provider keeps its vCenter session alive and recovers from long idle periods automatically:

• A keepalive runs against the govmomi session so it does not lapse during quiet periods.
• Validate performs a live probe of the session and reconnects if the session has gone stale.

The practical effect: a provider pod that has been idle for hours (overnight, between batch runs) no longer fails its next RPC with a NotAuthenticated / session is not authenticated error and then has to be restarted. The breaker stays closed across idle windows.

If you previously worked around stale sessions with a periodic provider-pod restart (CronJob, etc.), you can drop that workaround on v0.3.8.

Console access¶

Describe populates status.consoleURL with a vSphere web client deep link that includes the VM's instance UUID. Open it in a browser; vCenter prompts for authentication and lands you on the VM's summary tab.

kubectl get vm web-application -n my-app -o jsonpath='{.status.consoleURL}'
# https://vcenter.example.com/ui/app/vm;nav=h/urn:vmomi:VirtualMachine:vm-123:xxxxx/summary

Multi-vCenter¶

Deploy one Provider CR per vCenter. Each gets its own provider pod, its own gRPC port, and its own CircuitBreaker series in metrics:

apiVersion: infra.virtrigaud.io/v1beta1
kind: Provider
metadata:
  name: vsphere-dc-a
spec:
  type: vsphere
  endpoint: https://vcenter-a.example.com/sdk
  credentialSecretRef:
    name: vsphere-credentials-a
---
apiVersion: infra.virtrigaud.io/v1beta1
kind: Provider
metadata:
  name: vsphere-dc-b
spec:
  network:
    vsphere:
      portgroup: "LS-Backend-App"
    type: external

---
# Reference them from the VM
apiVersion: infra.virtrigaud.io/v1beta1
kind: VirtualMachine
metadata:
  name: multi-nic-vm
spec:
  # ...
  networks:
    - name: frontend
      networkRef:
        name: frontend-network
    - name: backend
      networkRef:
        name: backend-network

Static IP Assignment (guestinfo)¶

When a template boots without DHCP, VirtRigaud can hand the guest a static IP by writing it into the VM's guestinfo properties at create time. This is the mechanism used by the vSphere provider for static addressing.

How to request a static IP¶

Set the address fields inline on the VirtualMachine's spec.networks entry — not on the VMNetworkAttachment:

apiVersion: infra.virtrigaud.io/v1beta1
kind: VirtualMachine
metadata:
  name: static-ip-vm
spec:
  providerRef:
    name: vsphere-prod
  classRef:
    name: standard-vm
  imageRef:
    name: ubuntu-22-04-template
  powerState: On
  networks:
    - name: app
      # networkRef is optional. When omitted, the template's existing NIC is
      # reused and only the guestinfo IP settings below are applied.
      networkRef:
        name: app-network
      ipAddress: "192.168.100.50"   # static IPv4 address
      prefix: 24                     # CIDR prefix length (1–32)
      gateway: "192.168.100.1"       # default gateway
      dns: "192.168.1.10,8.8.8.8"    # comma-separated resolvers

Guest-side helper (template bake-in)¶

A minimal oneshot unit that reads the keys with vmware-rpctool and renders a netplan config on first boot:

#!/usr/bin/env bash
# /usr/local/sbin/apply-guestinfo-network.sh — run before networking comes up.
set -euo pipefail

get() { vmware-rpctool "info-get guestinfo.network.$1" 2>/dev/null || true; }

IP=$(get ip); PREFIX=$(get prefix); GW=$(get gateway); DNS=$(get dns)

# No static IP published → leave DHCP/template defaults in place.
[ -n "$IP" ] || exit 0

IFACE=$(ls /sys/class/net | grep -E '^(en|eth)' | head -n1)
DNS_YAML=$(echo "$DNS" | tr ',' '\n' | sed 's/^/        - /')

cat > /etc/netplan/90-guestinfo.yaml <<EOF
network:
  version: 2
  ethernets:
    ${IFACE}:
      dhcp4: false
      addresses: [${IP}/${PREFIX:-24}]
      routes:
        - to: default
          via: ${GW}
      nameservers:
        addresses:
${DNS_YAML}
EOF

netplan apply

# /etc/systemd/system/apply-guestinfo-network.service
[Unit]
Description=Apply static network from vSphere guestinfo
Before=network-pre.target
Wants=network-pre.target

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/apply-guestinfo-network.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Enable it in the template (systemctl enable apply-guestinfo-network.service) before converting the VM to a template. open-vm-tools must be installed so that vmware-rpctool is available.

Verifying¶

From a host with govc, confirm the keys landed on the created VM:

govc vm.info -e static-ip-vm | grep guestinfo.network
# guestinfo.network.ip:      192.168.100.50
# guestinfo.network.prefix:  24
# guestinfo.network.gateway: 192.168.100.1
# guestinfo.network.dns:     192.168.1.10,8.8.8.8

If the keys are present but the guest is still on DHCP, the guest-side helper is missing or failed — check journalctl -u apply-guestinfo-network inside the VM.

High Availability¶

# VM with HA/DRS settings
apiVersion: infra.virtrigaud.io/v1beta1
kind: VirtualMachine
metadata:
  name: critical-application
spec:
  providerRef:
    name: vsphere-prod
  # ... other config ...

  # High availability configuration
  availability:
    # HA restart priority
    restartPriority: "high"          # disabled, low, medium, high
    isolationResponse: "powerOff"    # none, powerOff, shutdown
    vmMonitoring: "vmMonitoringOnly" # vmMonitoringDisabled, vmMonitoringOnly, vmAndAppMonitoring

    # DRS configuration
    drsAutomationLevel: "fullyAutomated"  # manual, partiallyAutomated, fullyAutomated
    drsVmBehavior: "fullyAutomated"       # manual, partiallyAutomated, fullyAutomated

    # Anti-affinity rules
    antiAffinityGroups: ["web-tier", "database-tier"]

    # Host affinity (pin to specific hosts)
    hostAffinityGroups: ["production-hosts"]

Snapshot Management¶

# Advanced snapshot configuration
apiVersion: infra.virtrigaud.io/v1beta1
kind: VMSnapshot
metadata:
  name: pre-upgrade-snapshot
spec:
  vmRef:
    name: web-application

  # Snapshot settings
  name: "Pre-upgrade snapshot"
  description: "Snapshot before application upgrade"
  memory: true                    # Include memory state
  quiesce: true                   # Quiesce guest filesystem

  # Retention policy
  retention:
    maxSnapshots: 3               # Keep max 3 snapshots
    maxAge: "7d"                  # Delete after 7 days

  # Schedule (optional)
  schedule: "0 2 * * 0"          # Weekly at 2 AM Sunday

Troubleshooting¶

CircuitBreaker open for the vSphere provider¶

Check /metrics:

virtrigaud_circuit_breaker_state{provider_type="vsphere", provider="vsphere-prod"} == 2

When this fires, the manager has fast-failed enough RPCs to the vCenter that it has opened the breaker (default 5 consecutive failures with 30s reset; see Resilience). Investigate the underlying vCenter — credentials, certificate, network — rather than restarting the manager.

"Invalid configuration for device '0'" on disk attach¶

Older releases hit this when a StoragePod-resolved datastore was reattached to additional disks; the returned object.Datastore was missing its inventory path. Fixed in v0.3.3 by routing through p.finder.Datastore(ctx, best.Name). Upgrade.

"Login failed: incorrect user name or password"¶

The username in Provider.spec.credentialSecretRef Secret must match the principal name expected by vCenter — usually user@vsphere.local, not bare user. Also check whether the user is locked out via Single Sign-On.

NotAuthenticated after a long idle period¶

On releases before v0.3.8 the govmomi session could lapse during long idle windows, and the next RPC would fail with NotAuthenticated / session is not authenticated until the provider pod was restarted. v0.3.8 (#190) adds a session keepalive and a live-probe reconnect in Validate, so the session self-heals. If you still see this, upgrade to v0.3.8 and confirm the provider image tag is :v0.3.8. See Session resilience.

Template not found¶

The VMImage.spec.source.template value is resolved via the govmomi finder. List templates with:

govc ls /Datacenter/vm/templates/

If the template lives in a sub-folder, set VMImage.spec.source.folder to the inventory path of the parent folder.

Datastore issues¶

# Datastore capacity
govc datastore.info datastore-name

# All datastores
govc ls /Datacenter/datastore/

# StoragePod (Datastore Cluster) members
govc object.collect -s /Datacenter/datastore/DatastoreCluster-SSD childEntity

If a StoragePod-selected datastore exists but the create still fails, check whether the storage policy assigned to the VMClass conflicts with the member datastores in the cluster.

Validation walkthrough with govc¶

export GOVC_URL='https://vcenter.example.com'
export GOVC_USERNAME='administrator@vsphere.local'
export GOVC_PASSWORD='password'
export GOVC_INSECURE=1        # only for dev

govc about                    # connectivity + credentials
govc ls                       # datacenters
govc ls /Datacenter/host/     # clusters & hosts
govc ls /Datacenter/datastore/
govc ls /Datacenter/network/
govc ls /Datacenter/vm/templates/

Debug logging¶

# in Provider.spec.runtime
runtime:
  logLevel: debug
  env:
    - name: GOVMOMI_DEBUG
      value: "true"

GOVMOMI_DEBUG=true dumps every SOAP/REST exchange against vCenter — useful for diagnosing permission errors and unexpected task failures, but verbose; gate it behind a maintenance window.

Performance tips¶

• Hot-add disabled for latency-sensitive VMs: hot-add CPU/memory adds a thin layer of indirection. For real-time / VoIP / HFT workloads, set cpuHotAddEnabled: false and memoryHotAddEnabled: false.
• Eager-zeroed thick for IOPS-heavy disks: trade space for predictable write latency.
• pvscsi everywhere: unless your guest is ancient, pvscsi outperforms the LSI variants for any sustained I/O.
• Multiple SCSI controllers: spread disks across controllers (scsi.controller: 0, scsi.controller: 1, ...) to parallelise queue depth.

API reference¶

• Full CRD field reference: Generated CRD docs.
• Provider gRPC contract: Generated gRPC docs.
• Capability matrix (all providers): Capabilities.

Support¶

• Documentation: VirtRigaud Docs
• Issues: GitHub Issues — label provider/vsphere
• govmomi reference: vmware/govmomi