[How to] GlusterFS Dynamic Volume Provisioner in Kubernetes (>= v1.4 ) / Openshift.

I am happy to share that Gluster Dynamic Volume Provisioner is available in kubernetes tree since 1.4 release !!

This is one of the feature which enables the User/Developer in Kubernetes/Openshift to have Persistent Volume Claim ( PVC ) request to be satisfied dynamically without an admin intervention. IMHO, it gives a nice user experience and its a good feature to have in container orchestrators. Till this feature came in, the persistent volumes in the store was created statically.

The static provisioning workflow looks like this:

Eventhough it was easy to perform the static provisioning of volumes, it has few limitations in my view.

*) The admin has to create the persistent volumes upfront and keep it in persistent store.

*) When a claim request comes to the controller, it check for the size of the request against the available PVs in the pool and if the ( available PV size >= size of the request ) it bind the claim.

The latter can lead to wastage of storage in most of the case.

These kind of limitations have been lifted with Dynamic provisioning. Now the admin define the storage classes and the user/developer request the persistent volumes using the storage class reference in the PVC request. The storage classes can pass the parameters of the plugin using the storage class key value pairs.

I have created the following diagram to simply the workflow of dynamic (Ex: GlusterFS) provisioning.

As you can see in the above diagram, the GlusterFS plugin in Kubernetes/Openshift make use of “Heketi” to provision GlusterFS volumes. If you want to know more about Heketi and how it can be used to manage gluster clusters, please refer this Wiki. In short, Heketi is a volume manager for GlusterFS clusters. It manage Gluster trusted pool and create volumes based on demand.

Lets start from Storage Class which allows us to do dynamic provisioning in kubernetes.

Here is an example of the storage class parameters:


resturl : Gluster REST service/Heketi service url which provision gluster volumes on demand. The general format should be IPaddress:Port and this is a mandatory parameter for GlusterFS dynamic provisioner. If Heketi service is exposed as a routable service in openshift/kubernetes setup, this can have a format similar to heketi-storage-project.cloudapps.mystorage.com where the fqdn is a resolvable heketi service url.

restuser : Gluster REST service/Heketi user who has access to create volumes in the Gluster Trusted Pool.

secretNamespace + secretName : Identification of Secret instance that containes user password to use when talking to Gluster REST service. These parameters are optional, empty password will be used when both secretNamespace and secretName are omitted.

for more details on these parameters please refer github.com/kubernetes/kubernetes/tree/master/examples/experimental/persistent-volume-provisioning

To summarize, user/developer request the persistent storage using the claim and mention the storage class which need to be used/mapped with the claim. As soon as the claim request comes in, the GlusterFS plugin in kubernetes create a volume with the requested size and BIND the persistent volume to Claim. When there is a request to delete the claim, the subjected volume is deleted from the backend gluster trusted pool. The glusterfs plugin in kubernetes make use of ‘Heketi’ to provision a volume dynamically.

Here is the demo video of Dynamic GlusterFS provisioner in Kubernetes.

As always comments/suggestions/questions are welcome.. 🙂

Digiprove sealCopyright secured by Digiprove © 2017 Humble Chirammal

Leave a Reply

Your email address will not be published. Required fields are marked *