In previous blog posts, we discussed, how to use GlusterFS as a persistent storage in Kubernetes and Openshift. In nutshell, the GlusterFS can be deployed/used in a Kubernetes/openshift environment as :
[terminal]
*) Contenarized GlusterFS ( Pod )
*) GlusterFS as Openshift service and Endpoint (Service and Endpoint).
*) GlusterFS volume as Persistent Volume (PV) and using GlusterFS volume plugin to bind this PV to a Persistent Volume Claim ( PVC)
*) GlusterFS template to deploy GlusterFS pods in an Openshift Environment.
[/terminal]
All the configuration files that can be used to deploy GlusterFS can be found @ github.com/humblec/glusterfs-kubernetes-openshift/ or github.com/gluster/glusterfs-kubernetes-openshift. Let’s see how to use these files to deploy GlusterFS in Kubernetes and Openshift. We will start with Deploying GlusterFS pods in an Openshift/Kubernetes Environment. Deploying GlusterFS Pod:
[Update] The pod file is renamed to gluster-pod.yaml in the mentioned repo. More details about Gluster Containers can be found @http://www.slideshare.net/HumbleChirammal/gluster-containers
GlusterFS pods can be deployed in Kubernetes/Openshift, so that Gluster Nodes are deployed in containers and it can provide persistent storage for Openshift/Kubernetes setup. The examples files in this repo are used for this demo.
Step 1: Create GlusterFS pod
[terminal]
[root@atomic-node2 gluster_pod]# oc create -f gluster-1.yaml
[/terminal]
Step 2: Get details about the GlusterFS pod.
[terminal]
[root@atomic-node2 gluster_pod]# oc describe pod gluster-1
Name: gluster-1
Namespace: default
Image(s): gluster/gluster-centos
Node: atomic-node1/10.70.43.174
Start Time: Tue, 17 May 2016 10:19:17 +0530
Labels: name=gluster-1 Status:
Running Reason: Message: IP: 10.70.43.174
Replication Controllers:
Containers: glusterfs:
Container ID: docker://ff8f4af700d725dfe0e08939ec011c34ddf9dedc7204e0ced1cc355a56150742
Image: gluster/gluster-centos
Image ID: docker://033de9c44a8aabde55ce8a2b751ccf5bc345fdb534ea30e79a8fa70b82dc7761
QoS Tier:
cpu: BestEffort
memory: BestEffort
State: Running
Started: Tue, 17 May 2016 10:20:35 +0530
Ready: True Restart
Count: 0
Environment Variables:
Conditions: Type Status Ready True
Volumes:
brickpath:
Type: HostPath (bare host directory volume)
Path: /mnt/brick1 default-token-72d89:
Type: Secret (a secret that should populate this volume)
SecretName: default-token-72d89
Events: FirstSeen LastSeen Count From SubobjectPath Reason Message ───────── ──────── ───── ──── ───────────── ────── ─────── 1m 1m 1
{scheduler }
Scheduled Successfully assigned gluster-1 to atomic-node1 1m 1m 1
{kubelet atomic-node1} implicitly required container POD
Pulled Container image “openshift3/ose-pod:v3.1.1.6” already present on machine 1m 1m 1
{kubelet atomic-node1} implicitly required container POD Created Created with docker id f55ce55e6ea3 1m 1m 1
{kubelet atomic-node1} implicitly required container
POD Started Started with docker id f55ce55e6ea3 1m 1m 1 {kubelet atomic-node1} spec.containers{glusterfs}
Pulling pulling image “gluster/gluster-centos” 8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs}
Pulled Successfully pulled image “gluster/gluster-centos” 8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs}
Created Created with docker id ff8f4af700d7 8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs}
Started Started with docker id ff8f4af700d7
[/terminal]
From above logs, you can see it pulled `gluster/gluster-centos` container image and deployed containers from it.
[terminal]
[root@atomic-node2 gluster_pod]# oc get pods
NAME READY STATUS RESTARTS AGE
gluster-1 1/1 Running 0 1m
[/terminal]
Examine the container and make sure it has a running GlusterFS daemon.
[terminal]
[root@atomic-node2 gluster_pod]# oc exec -ti gluster-1 /bin/bash Examine the processes running in this container and the `glusterd` service information.
[root@atomic-node1 /]# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.4 0.0 40780 2920 ? Ss 04:50 0:00 /usr/sbin/init root 20 0.3 0.0 36816 4272 ? Ss 04:50 0:00 /usr/lib/syste root 21 0.0 0.0 118476 1332 ? Ss 04:50 0:00 /usr/sbin/lvme root 37 0.0 0.0 101344 1228 ? Ssl 04:50 0:00 /usr/sbin/gssp rpc 44 0.1 0.0 64904 1052 ? Ss 04:50 0:00 /sbin/rpcbind root 209 0.1 0.1 364716 13444 ? Ssl 04:50 0:00 /usr/sbin/glus root 341 1.1 0.0 13368 1964 ? Ss 04:51 0:00 /bin/bash root 354 0.0 0.0 49020 1820 ? R+ 04:51 0:00 ps aux
[root@atomic-node1 /]# service glusterd status
Redirecting to /bin/systemctl status glusterd.service ● glusterd.service – GlusterFS, a clustered file-system server Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2016-05-17 04:50:41 UTC; 35s ago Process: 208 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid –log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 209 (glusterd) CGroup: /system.slice/docker-ff8f4af700d725dfe0e08939ec011c34ddf9dedc7204e0ced1cc355a56150742.scope/system.slice/glusterd.service └─209 /usr/sbin/glusterd -p /var/run/glusterd.pid –log-level INFO… ‣ 209 /usr/sbin/glusterd -p /var/run/glusterd.pid –log-level INFO… May 17 04:50:36 atomic-node1 systemd[1]: Starting Gluste… May 17 04:50:41 atomic-node1 systemd[1]: Started Gluster… Hint: Some lines were ellipsized, use -l to show in full. Let’s fetch some more details about GlusterFS in this container.
[root@atomic-node1 /]# gluster –version
glusterfs 3.7.9 built on Mar 20 2016 03:19:49 Repository revision: git://git.gluster.com/glusterfs.git Copyright (c) 2006-2011 Gluster Inc.
[root@atomic-node1 /]#
[root@atomic-node1 /]# mount |grep mnt /dev/mapper/atomic-node1-root on /mnt/brick1 type xfs (rw,relatime,seclabel,attr2,inode64,noquota) This container is built on top of CentOS base image as shown below.
[root@atomic-node1 /]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core)
[root@atomic-node1 /]#
[/terminal]
In this article, we discussed, how to run GlusterFS as a pod in Kubernetes or Openshift setup.
[Part 2] covers `how to use GlusterFS as a service, Persistent Volume for a Persistent Volume Claim`.
[Part 3] covers `how to use GlusterFS template to deploy GlusterFS pods in an Openshift/kubernetes setup`.