Failed to get VolumeID from volumeMigrationService for volumePath

Solution Verified - Updated

Environment

  • Red Hat OpenShift Container Platform 4.13
  • Red Hat OpenShift Container Platform 4.14
  • VMWare vSphere 7/8

Issue

After an OCP cluster is upgraded to a version where CSI migration of in-tree vSphere volumes is automatically enabled, volumes fail to attach with error:

rpc error: code = Internal desc = failed to get VolumeID from volumeMigrationService for volumePath: "[<datastore>] kubevols/<volume>.vmdk

Upon further inspection, following logs can be observed in the CSI driver pods:

30T18:42:43.774562897Z","caller":"migration/migration.go:525","msg":"failed to register volume \"
.... error: ServerFaultCode: A specified parameter was not correct: dsDiskUrl","TraceId":"375e6e5f-da57-4d30-b20e-f0cdc3b8dfff"}

Resolution

In order to avoid attach/detach issues on vSphere platform, once CSI migration is enabled - customers are recommended to specify full path to Datacenter and Datastore in cloud-config.

Full path to Datacenter and Datastore can be obtained by logging into vCenter UI.

Full path to Datacenter and Datastore can be specified by editing - cloud-provider-config in openshift-config namespace:

data:
  config: |
    [Global]
    secret-name = "vsphere-creds"
    secret-namespace = "kube-system"
    insecure-flag = "1"

    [Workspace]
    server = "vcenter.foobar.com"
    datacenter = "/folder/dc-name"
    default-datastore = "/folder/dc-name/datastore/ds-name"
    folder = "/folder/dc-name/vm/cluster-name"

    [VirtualCenter "vcenter.foobar.com"]
    datacenters = "/folder/dc-name"

Root Cause

The root cause of this problem is, when an in-tree vSphere volume is migrated to CSI, on vSphere platform the volume must be registered as an FCD.

The in-tree vSphere driver was a bit flexible in how the Datacenter path was specified, but when the volume gets converted to FCD, then the full path to the Datacenter must be used.

SBR
Components
Category
Tags

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.