The Gluster Blog

Gluster blog stories provide high-level spotlights on our users all over the world

Possible configurations of GlusterFS in Kubernetes/OpenShift setup

Gluster
2016-08-26

In previous blog posts we discussed, how to use GlusterFS as a persistent storage in Kubernetes and Openshift. In nutshell, the GlusterFS can be deployed/used in a kubernetes/openshift environment as :

*) Contenarized GlusterFS ( Pod )
*) GlusterFS as Openshift service and Endpoint (Service and Endpoint).
*) GlusterFS volume as Persistent Volume (PV) and using GlusterFS volume plugin
to bind this PV to a Persistent Volume Claim ( PVC)
*) GlusterFS template to deploy GlusterFS pods in an Openshift Environment.

All the configuration files that can be used to deploy GlusterFS can be found @ github.com/humblec/glusterfs-kubernetes-openshift/ or github.com/gluster/glusterfs-kubernetes-openshift. Lets see how to use these files to deploy GlusterFS in kubernetes and Openshift.

We will start with Deploying GlusterFS pods in an Openshift/Kubernetes Environment.

Deploying GlusterFS Pod:

[Update] The pod file is renamed to gluster-pod.yaml in the mentioned repo. More details about Gluster Containers can be found @http://www.slideshare.net/HumbleChirammal/gluster-containers

GlusterFS pods can be deployed in Kubernetes/Openshift, so that Gluster Nodes are deployed in containers and it can provide persistent storage for Openshift/Kubernetes setup.

The examples files in this repo are used for this demo.

Step 1: Create GlusterFS pod

[root@atomic-node2 gluster_pod]# oc create -f gluster-1.yaml

Step 2: Get details about the GlusterFS pod.


[root@atomic-node2 gluster_pod]# oc describe pod gluster-1
Name: gluster-1
Namespace: default
Image(s): gluster/gluster-centos
Node: atomic-node1/10.70.43.174
Start Time: Tue, 17 May 2016 10:19:17 +0530
Labels: name=gluster-1
Status: Running
Reason:
Message:
IP: 10.70.43.174
Replication Controllers:
Containers:
glusterfs:
Container ID: docker://ff8f4af700d725dfe0e08939ec011c34ddf9dedc7204e0ced1cc355a56150742
Image: gluster/gluster-centos
Image ID: docker://033de9c44a8aabde55ce8a2b751ccf5bc345fdb534ea30e79a8fa70b82dc7761
QoS Tier:
cpu: BestEffort
memory: BestEffort
State: Running
Started: Tue, 17 May 2016 10:20:35 +0530
Ready: True
Restart Count: 0
Environment Variables:
Conditions:
Type Status
Ready True
Volumes:
brickpath:
Type: HostPath (bare host directory volume)
Path: /mnt/brick1
default-token-72d89:
Type: Secret (a secret that should populate this volume)
SecretName: default-token-72d89
Events:
FirstSeen LastSeen Count From SubobjectPath Reason Message
───────── ──────── ───── ──── ───────────── ────── ───────
1m 1m 1 {scheduler } Scheduled Successfully assigned gluster-1 to atomic-node1
1m 1m 1 {kubelet atomic-node1} implicitly required container POD Pulled Container image "openshift3/ose-pod:v3.1.1.6" already present on machine
1m 1m 1 {kubelet atomic-node1} implicitly required container POD Created Created with docker id f55ce55e6ea3
1m 1m 1 {kubelet atomic-node1} implicitly required container POD Started Started with docker id f55ce55e6ea3
1m 1m 1 {kubelet atomic-node1} spec.containers{glusterfs} Pulling pulling image "gluster/gluster-centos"
8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs} Pulled Successfully pulled image "gluster/gluster-centos"
8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs} Created Created with docker id ff8f4af700d7
8s 8s 1 {kubelet atomic-node1} spec.containers{glusterfs} Started Started with docker id ff8f4af700d7

From above logs, you can see it pulled `gluster/gluster-centos` container image and deployed containers from it.


[root@atomic-node2 gluster_pod]# oc get pods
NAME READY STATUS RESTARTS AGE
gluster-1 1/1 Running 0 1m

Examine the container and make sure it has a running GlusterFS daemon.


[root@atomic-node2 gluster_pod]# oc exec -ti gluster-1 /bin/bash

Examine the processes running in this container and the `glusterd` service information.


[root@atomic-node1 /]# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.4 0.0 40780 2920 ? Ss 04:50 0:00 /usr/sbin/init
root 20 0.3 0.0 36816 4272 ? Ss 04:50 0:00 /usr/lib/syste
root 21 0.0 0.0 118476 1332 ? Ss 04:50 0:00 /usr/sbin/lvme
root 37 0.0 0.0 101344 1228 ? Ssl 04:50 0:00 /usr/sbin/gssp
rpc 44 0.1 0.0 64904 1052 ? Ss 04:50 0:00 /sbin/rpcbind
root 209 0.1 0.1 364716 13444 ? Ssl 04:50 0:00 /usr/sbin/glus
root 341 1.1 0.0 13368 1964 ? Ss 04:51 0:00 /bin/bash
root 354 0.0 0.0 49020 1820 ? R+ 04:51 0:00 ps aux
[root@atomic-node1 /]# service glusterd status
Redirecting to /bin/systemctl status glusterd.service
● glusterd.service - GlusterFS, a clustered file-system server
Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2016-05-17 04:50:41 UTC; 35s ago
Process: 208 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 209 (glusterd)
CGroup: /system.slice/docker-ff8f4af700d725dfe0e08939ec011c34ddf9dedc7204e0ced1cc355a56150742.scope/system.slice/glusterd.service
└─209 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO...
‣ 209 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO...
May 17 04:50:36 atomic-node1 systemd[1]: Starting Gluste...
May 17 04:50:41 atomic-node1 systemd[1]: Started Gluster...
Hint: Some lines were ellipsized, use -l to show in full.

Let’s fetch some more details about GlusterFS in this container.


[root@atomic-node1 /]# gluster --version
glusterfs 3.7.9 built on Mar 20 2016 03:19:49
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@atomic-node1 /]#
[root@atomic-node1 /]# mount |grep mnt
/dev/mapper/atomic-node1-root on /mnt/brick1 type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

This container is built on top of CentOS base image as shown below.


[root@atomic-node1 /]# cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
[root@atomic-node1 /]#

In this article we discussed, how to run GlusterFS as a pod in Kubernetes or Openshift setup.

[Part 2] covers `how to use GlusterFS as a service, Persistent Volume for a Persistent Volume Claim`.

[Part 3] covers `how to use GlusterFS template to deploy GlusterFS pods in an Openshift/kubernetes setup`.

BLOG

  • 13 Nov 2019
    Announcing Gluster 7.0

    The Gluster community is pleased to announce the release of 7.0, our latest release. This is a major release that includes a range of code improvements and stability fixes along with a few features as noted below. A selection of the key features and bugs addressed are documented in this...

    Read more
  • 15 Oct 2019
    Gluster and CentOS Stream

    Progress cannot be made without change. As technologists, we recognize this every day. Most of the time, these changes are iterative: progresssive additions of features to projects like Gluster. Sometimes those changes are small, and sometimes not. And that’s, of course, just talking about our project. But one of the...

    Read more
  • 26 Apr 2019
    Gluster Monthly Newsletter, April 2...

    Upcoming Community Happy Hour at Red Hat Summit! Tue, May 7, 2019, 6:30 PM – 7:30 PM EDT https://cephandglusterhappyhour_rhsummit.eventbrite.com has all the details. Gluster 7 Roadmap Discussion kicked off for our 7 roadmap on the mailing lists, see [Gluster-users] GlusterFS v7.0 (and v8.0) roadmap discussion https://lists.gluster.org/pipermail/gluster-users/2019-March/036139.html for more details. Community...

    Read more