Set Up Pods to Use SRIOV to Access HW Accelerators¶

You can configure pods with SR-IOV access to a HW accelerator device by adding the appropriate ‘resources’ request in the pod specification.

About this task

The following procedure shows an example of launching a container image with ‘resources’ request for a VF to the ACC100 device.

Procedure

Request the resource name of the desired accelerator.

~(keystone_admin)$ kubectl get node controller-0 -o json | json_pp | grep -A 15 '"allocatable"'
"allocatable" : {
  "cpu" : "126",
  "ephemeral-storage" : "9417620260",
  "hugepages-1Gi" : "10Gi",
  "hugepages-2Mi" : "0",
  "intel.com/intel_fec_acc100" : "16",
  "memory" : "503822580Ki",
  "pods" : "110"
},
"capacity" : {
  "cpu" : "128",
  "ephemeral-storage" : "10218772Ki",
  "hugepages-1Gi" : "10Gi",
  "hugepages-2Mi" : "0",
  "intel.com/intel_fec_acc100" : "16",
  "memory" : "525674740Ki",

Create a pod.yml file that requests 16 ACC100 VFs (i.e. intel.com/intel_fec_acc100: ‘16’)

~(keystone_admin)$ cat >> pod0.yml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pod0
  annotations:
  labels:
    app: pod0
spec:
  replicas: 1
  selector:
    matchLabels:
      app: pod0
  template:
    metadata:
      name: pod0
      labels:
        app: pod0
    spec:
      restartPolicy: Always
      containers:
      - name: pod0
        image: "windse/pktgen-testpmd-bbdev:d1911r4p1912"
        volumeMounts:
        - mountPath: /mnt/huge-1048576kB
          name: hugepage
        - mountPath: /sys/devices
          name: uio
        command: ["/bin/bash", "-ec", "sleep infinity"]
        securityContext:
          privileged: false
          capabilities:
            add:
              ["IPC_LOCK", "SYS_ADMIN"]
        resources:
          requests:
            memory: 4Gi
            intel.com/intel_fec_acc100: '16'
            windriver.com/isolcpus: 24
          limits:
            hugepages-1Gi: 2Gi
            memory: 4Gi
            intel.com/intel_fec_acc100: '16'
            windriver.com/isolcpus: 24
      volumes:
      - name: hugepage
        emptyDir:
          medium: HugePages
      - name: uio
        hostPath:
          path: /sys/devices
EOF

Start the pod.

~(keystone_admin)$ kubectl create -f pod0.yml

Wait for the pod to start and execute the following command:

~(keystone_admin)$ kubectl exec -it pod0 -- env | grep PCIDEVICE_INTEL_COM_INTEL_FEC_ACC100

The following PCI addresses and “VFIO_TOKEN” corresponding to the SR-IOVs are displayed:

PCIDEVICE_INTEL_COM_INTEL_FEC_ACC100=0000:32:00.5,0000:32:00.1,0000:32:00.2,0000:32:00.4,0000:32:01.0,0000:32:01.1,0000:32:01.2,0000:32:01.3,0000:32:00.0,0000:32:00.6,0000:32:00.7,0000:32:01.4,0000:32:01.7,0000:32:00.3,0000:32:01.6,0000:32:01.5
PCIDEVICE_INTEL_COM_INTEL_FEC_ACC100_INFO={"0000:32:00.0":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.0"},"vfio":{"dev-mount":"/dev/vfio/315","mount":"/dev/vfio/vfio"}},"0000:32:00.1":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.1"},"vfio":{"dev-mount":"/dev/vfio/316","mount":"/dev/vfio/vfio"}},"0000:32:00.2":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.2"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.3":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.3"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.4":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.4"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.5":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.5"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.6":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.6"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:00.7":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:00.7"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.0":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.0"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.1":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.1"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.2":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.2"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.3":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.3"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.4":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.4"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.5":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.5"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.6":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.6"},"vfio":{"mount":"/dev/vfio/vfio"}},"0000:32:01.7":{"extra":{"VFIO_TOKEN":"02bddbbf-bbb0-4d79-886b-91bad3fbb510"},"generic":{"deviceID":"0000:32:01.7"},"vfio":{"mount":"/dev/vfio/vfio"}}}

Set Up Pods to Use SRIOV to Access HW Accelerators

Set Up Pods to Use SRIOV to Access HW Accelerators¶

StarlingX