[2020 updated] ISCSI multipath support in Kubernetes (v1.6) or Openshift.

by

Multipathing is an important feature in most of the storage setups, so for ISCSI based storage systems.
In most of the ISCSI based storages, multiple paths can be defined as the same IQN shared with more than one portal IPs.
Even if one of the network interface/portal is down, the share/target can be accessed via other active interfaces/portals.
This is indeed a good feature considering I/O from ISCSI initiator to Target and high availability of data path.
However the ISCSI plugin in Kubernetes was not capable of making use of multipath feature and it
was always just one path configured by default in Kubernetes. If that path goes down, the target can not be accessed.

Recently I added multipath support to ISCSI kubernetes plugin with this Pull Request.

With this functionality, a Kubernetes user/admin can specify the other Target Portal IPs in a new field called portals in ISCSI volume. That’s the only change required from admin side!!. If there are multiple portals, admin can mention these additional target portals in the portals field as shown below.

The new structure will look like this.

iscsi: targetPortal: 10.0.2.15:3260 + portals: [‘10.0.2.16:3260’, ‘10.0.2.17:3260’] iqn: iqn.2001-04.com.example:storage.kube.sys1.xyz lun: 0 fsType: ext4 readOnly: true

If you are directly using the above volume definition in POD spec, your pod spec may look like this.

apiVersion: v1 kind: Pod metadata: name: iscsipd spec: containers: – name: iscsipd-rw image: kubernetes/pause volumeMounts: – mountPath: “/mnt/iscsipd” name: iscsipd-rw volumes: – name: iscsipd-rw iscsi: targetPortal: 10.0.2.15:3260 portals: [‘10.0.2.16:3260’, ‘10.0.2.17:3260’, ‘10.0.2.18:3260’] iqn: iqn.2016-04.test.com:storage.target00 lun: 0 fsType: ext4 readOnly: true

Once the pod is up and running, you could check and verify below outputs from the Kubernetes Host where the pod is running:

# multipath -ll mpatha (360014059bafbe58ba644b2889c34903f) dm-2 LIO-ORG ,disk01 size=20G features=’0′ hwhandler=’0′ wp=rw |-+- policy=’service-time 0′ prio=1 status=active | – 44:0:0:0 sdb 8:16 active ready running -+- policy=’service-time 0′ prio=1 status=enabled `- 42:0:0:0 sdc 8:32 active ready running -+- policy=’service-time 0′ prio=1 status=enabled `- 46:0:0:0 sdd 8:48 active ready running -+- policy=’service-time 0′ prio=1 status=enabled `- 48:0:0:0 sde 8:64 active ready running

ISCSI session looks like below:

# iscsiadm -m session tcp: [10] 10.0.2.15:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash) tcp: [12] 10.0.2.16:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash) tcp: [14] 10.0.2.17:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash) tcp: [16] 10.0.2.18:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash)

The device paths:

# ll /dev/disk/by-path/ lrwxrwxrwx. 1 root root 9 Feb 16 15:58 ip-10.0.2.15:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sdb lrwxrwxrwx. 1 root root 9 Feb 16 15:41 ip-10.0.2.16:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sdc lrwxrwxrwx. 1 root root 9 Feb 16 15:58 ip-10.0.2.17:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sdd lrwxrwxrwx. 1 root root 9 Feb 16 15:41 ip-10.0.2.18:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sde

I believe, this is a nice feature added to Kubernetes ISCSI storage. Please let me know your comments/feedback/suggestions on this.

Share this post:

Share on LinkedIn

3 thoughts on “[2020 updated] ISCSI multipath support in Kubernetes (v1.6) or Openshift.”

  1. We are trying to implement ISCSI persistent volumes on OpenShift, but the Dell/EMC Unity storage we have is exposing two different IQN’s for the different paths:

    root@vsrv3240 ~]# iscsiadm -m node
    10.0.0.21:3260,2 iqn.1992-04.com.emc:cx.ckm00174201943.a5
    10.0.0.22:3260,1 iqn.1992-04.com.emc:cx.ckm00174201943.b5

    According to the documentation, it does this to easily identify the “Storage Processor”. I don’t believe this is something we can just turn off…

    Do you have any idea how we could configure a Kubernetes PersistentVolume to use both portals and both IQN’s?

  2. We’re trying to implement ISCSI persistent volumes on our Dell/EMC Unity storage, but it looks like it is exposing ISCSI Targets on two different IQN’s, depending on the Storage Processor:

    root@vsrv3240 ~]# iscsiadm -m node
    10.0.0.21:3260,2 iqn.1992-04.com.emc:cx.ckm00174201943.a5
    10.0.0.22:3260,1 iqn.1992-04.com.emc:cx.ckm00174201943.b5

    To get multipath working, we’d need to log in on both portals with the specified IQN. Do you have any idea how this could be achieved using the Kubernetes ISCSI persistent volume driver?

Comments are closed.