Multipathing is an important feature in most of the storage setups, so for ISCSI based storage systems.
In most of the ISCSI based storages, multiple paths can be defined as the same IQN shared with more than one portal IPs.
Even if one of the network interface/portal is down, the share/target can be accessed via other active interfaces/portals.
This is indeed a good feature considering I/O from ISCSI initiator to Target and high availability of data path.
However the ISCSI plugin in Kubernetes was not capable of making use of multipath feature and it
was always just one path configured by default in Kubernetes. If that path goes down, the target can not be accessed.
Recently I added multipath support to ISCSI kubernetes plugin with this Pull Request.
With this functionality, a Kubernetes user/admin can specify the other Target Portal IPs in a new field called portals in ISCSI volume. That’s the only change required from admin side!!. If there are multiple portals, admin can mention these additional target portals in the portals field as shown below.
The new structure will look like this.
iscsi:
targetPortal: 10.0.2.15:3260
+
portals: [‘10.0.2.16:3260’, ‘10.0.2.17:3260’]
iqn: iqn.2001-04.com.example:storage.kube.sys1.xyz
lun: 0
fsType: ext4
readOnly: true
If you are directly using the above volume definition in POD spec, your pod spec may look like this.
apiVersion: v1
kind: Pod
metadata:
name: iscsipd
spec:
containers:
– name: iscsipd-rw
image: kubernetes/pause
volumeMounts:
– mountPath: “/mnt/iscsipd”
name: iscsipd-rw
volumes:
– name: iscsipd-rw
iscsi:
targetPortal: 10.0.2.15:3260
portals: [‘10.0.2.16:3260’, ‘10.0.2.17:3260’, ‘10.0.2.18:3260’]
iqn: iqn.2016-04.test.com:storage.target00
lun: 0
fsType: ext4
readOnly: true
Once the pod is up and running, you could check and verify below outputs from the Kubernetes Host where the pod is running:
# multipath -ll
mpatha (360014059bafbe58ba644b2889c34903f) dm-2 LIO-ORG ,disk01
size=20G features=’0′ hwhandler=’0′ wp=rw
|-+- policy=’service-time 0′ prio=1 status=active
| – 44:0:0:0 sdb 8:16 active ready running
-+- policy=’service-time 0′ prio=1 status=enabled
`- 42:0:0:0 sdc 8:32 active ready running
-+- policy=’service-time 0′ prio=1 status=enabled
`- 46:0:0:0 sdd 8:48 active ready running
-+- policy=’service-time 0′ prio=1 status=enabled
`- 48:0:0:0 sde 8:64 active ready running
ISCSI session looks like below:
# iscsiadm -m session
tcp: [10] 10.0.2.15:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash)
tcp: [12] 10.0.2.16:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash)
tcp: [14] 10.0.2.17:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash)
tcp: [16] 10.0.2.18:3260,1 iqn.2016-04.test.com:storage.target00 (non-flash)
The device paths:
# ll /dev/disk/by-path/
lrwxrwxrwx. 1 root root 9 Feb 16 15:58 ip-10.0.2.15:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sdb
lrwxrwxrwx. 1 root root 9 Feb 16 15:41 ip-10.0.2.16:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sdc
lrwxrwxrwx. 1 root root 9 Feb 16 15:58 ip-10.0.2.17:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sdd
lrwxrwxrwx. 1 root root 9 Feb 16 15:41 ip-10.0.2.18:3260-iscsi-iqn.2016-04.test.com:storage.target00-lun-0 -> ../../sde
I believe, this is a nice feature added to Kubernetes ISCSI storage. Please let me know your comments/feedback/suggestions on this.