<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
comments inline.<br>
<br>
<div class="moz-cite-prefix">On 03/12/15 01:08, Surya K Ghatty
wrote:<br>
</div>
<blockquote
cite="mid:201512021939.tB2JdMXV008569@d03av01.boulder.ibm.com"
type="cite">
<p>Hi Soumya, Kaleb, all:<br>
<br>
Thanks for the response!<br>
<br>
<br>
Quick follow-up to this question - We tried running ganesha and
gluster on two separate machines and the configuration seems to
be working without issues. <br>
<br>
Follow-up question I have is this: what changes do I need to
make to put the Ganesha in active active HA mode - where backend
gluster and ganesha will be on a different node. I am using the
instructions here for putting Ganesha in HA mode. <a
moz-do-not-send="true"
href="http://www.slideshare.net/SoumyaKoduri/high-49117846"><a class="moz-txt-link-freetext" href="http://www.slideshare.net/SoumyaKoduri/high-49117846">http://www.slideshare.net/SoumyaKoduri/high-49117846</a></a>.
This presentation refers to commands like gluster
cluster.enable-shared-storage to enable HA.<br>
<br>
1. Here is the config I am hoping to achieve: <br>
glusterA and glusterB on individual bare metals - both in
Trusted pool, with volume gvol0 up and running.<br>
<br>
<br>
<br>
Ganesha 1 and 2 on machines ganesha1, and ganesha1. And my
gluster storage will be on a third machine gluster1. (with a
peer on another machine gluster2).<br>
<br>
Ganesha node1: on a VM ganeshaA.<br>
Ganesha node2: on another vm GaneshaB.<br>
<br>
I would like to know what it takes to put ganeshaA and GaneshaB
in Active Active HA mode. Is it technically possible?<br>
<br>
</p>
</blockquote>
<br>
Technically possible, but difficult to do that, u must manually
follow the steps which are internally by "gluster nfs-ganesha
enable"<br>
(Kaleb will have clear idea about it) <br>
<blockquote
cite="mid:201512021939.tB2JdMXV008569@d03av01.boulder.ibm.com"
type="cite">
<p>a. How do commands like cluster.enable-shared-storage work in
this case?<br>
</p>
</blockquote>
you should manually configure a shared storage(an export which both
GaneshaA and GaneshaB can access)<br>
<blockquote
cite="mid:201512021939.tB2JdMXV008569@d03av01.boulder.ibm.com"
type="cite">
<p>b. where does this command need to be run? on the ganesha node,
or on the gluster nodes?<br>
<br>
</p>
</blockquote>
As a I mentioned before, u cannot do this with help of gluster cli
if ganesha cluster outside trusted pool.<br>
<br>
I don't understand your requirement correctly, if it falls to any
of the following, I had answered according to my best knowledge<br>
<br>
1.) "ganesha should run on nodes in which gluster volume(/bricks) is
created"<br>
i. created trust pool using glusterA, glusterB, GaneshaA, GaneshaB<br>
ii. create volume using glusterA and glusterB<br>
iii. add GaneshaA and GaneshaB on server list in ganesha-ha.conf
file<br>
iv then follow remaining the steps for exporting volume via
nfs-ganesha<br>
<br>
2.) "ganesha cluster(vms) should not be part of gluster trusted
pool"<br>
(hacky way)<br>
i.) created trusted pool using glusterA and glusterB.<br>
ii.) create and start volume gvol0 using it<br>
iii.) created trusted pool using GaneshaA and GaneshaB<br>
iv.) before enabling nfs-ganesha option, add EXPORT{} for gvol0 in
/etc/ganesha/ganesha.conf<br>
in both GaneshaA and GaneshaB<br>
<br>
Note : The value for hostname in EXPORT{ FSAL {} } should be
glusterA or glusterB.<br>
<br>
<blockquote
cite="mid:201512021939.tB2JdMXV008569@d03av01.boulder.ibm.com"
type="cite">
<p><br>
2. Also, is it possible to have multiple ganesha servers point
to the same gluster volume in the back end? say, in the
configuration #1, I have another ganesha server GaneshaC that is
not clustered with ganeshaA or ganeshaB. Can it export the
volume gvol0 that ganeshaA and ganeshaB are also exporting?<br>
<br>
</p>
</blockquote>
<br>
Yes it is possible, but u may need to start GaneshaC manually
(running two different ganesha clusters in trusted pool via cli is
not supported)<br>
<blockquote
cite="mid:201512021939.tB2JdMXV008569@d03av01.boulder.ibm.com"
type="cite">
<p>thank you!<br>
<br>
<br>
</p>
</blockquote>
<br>
Regards,<br>
Jiffin<br>
<br>
<blockquote
cite="mid:201512021939.tB2JdMXV008569@d03av01.boulder.ibm.com"
type="cite">
<p>Surya.<br>
<br>
Regards,<br>
<br>
Surya Ghatty<br>
<br>
"This too shall pass"<br>
________________________________________________________________________________________________________<br>
Surya Ghatty | Software Engineer | IBM Cloud Infrastructure
Services Development | tel: (507) 316-0559 | <a class="moz-txt-link-abbreviated" href="mailto:ghatty@us.ibm.com">ghatty@us.ibm.com</a><br>
<br>
<br>
<img src="cid:part2.08020807.06080509@redhat.com" alt="Inactive
hide details for Soumya Koduri ---11/18/2015 05:08:02 AM---On
11/17/2015 10:21 PM, Surya K Ghatty wrote: > Hi:"
height="16" width="16" border="0"><font color="#424282">Soumya
Koduri ---11/18/2015 05:08:02 AM---On 11/17/2015 10:21 PM,
Surya K Ghatty wrote: > Hi:</font><br>
<br>
<font color="#5F5F5F" size="2">From: </font><font size="2">Soumya
Koduri <a class="moz-txt-link-rfc2396E" href="mailto:skoduri@redhat.com"><skoduri@redhat.com></a></font><br>
<font color="#5F5F5F" size="2">To: </font><font size="2">Surya
K Ghatty/Rochester/IBM@IBMUS, <a class="moz-txt-link-abbreviated" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a></font><br>
<font color="#5F5F5F" size="2">Date: </font><font size="2">11/18/2015
05:08 AM</font><br>
<font color="#5F5F5F" size="2">Subject: </font><font size="2">Re:
[Gluster-users] Configuring Ganesha and gluster on separate
nodes?</font><br>
</p>
<hr style="color:#8091A5; " size="2" noshade="noshade"
width="100%" align="left"><br>
<br>
<br>
<tt><br>
<br>
On 11/17/2015 10:21 PM, Surya K Ghatty wrote:<br>
> Hi:<br>
><br>
> I am trying to understand if it is technically feasible to
have gluster<br>
> nodes on one machine, and export a volume from one of these
nodes using<br>
> a nfs-ganesha server installed on a totally different
machine? I tried<br>
> the below and showmount -e does not show my volume
exported. Any<br>
> suggestions will be appreciated.<br>
><br>
> 1. Here is my configuration:<br>
><br>
> Gluster nodes: glusterA and glusterB on individual bare
metals - both in<br>
> Trusted pool, with volume gvol0 up and running.<br>
> Ganesha node: on bare metal ganeshaA.<br>
><br>
> 2. my ganesha.conf looks like this with IP address of
glusterA in the FSAL.<br>
><br>
> FSAL {<br>
> Name = GLUSTER;<br>
><br>
> # IP of one of the nodes in the trusted pool<br>
> *hostname = "WW.ZZ.XX.YY" --> IP address of GlusterA.*<br>
><br>
> # Volume name. Eg: "test_volume"<br>
> volume = "gvol0";<br>
> }<br>
><br>
> 3. I disabled nfs on gvol0. As you can see, *nfs.disable is
set to on.*<br>
><br>
> [root@glusterA ~]# gluster vol info<br>
><br>
> Volume Name: gvol0<br>
> Type: Distribute<br>
> Volume ID: 16015bcc-1d17-4ef1-bb8b-01b7fdf6efa0<br>
> Status: Started<br>
> Number of Bricks: 1<br>
> Transport-type: tcp<br>
> Bricks:<br>
> Brick1: glusterA:/data/brick0/gvol0<br>
> Options Reconfigured:<br>
> *nfs.disable: on*<br>
> nfs.export-volumes: off<br>
> features.quota-deem-statfs: on<br>
> features.inode-quota: on<br>
> features.quota: on<br>
> performance.readdir-ahead: on<br>
><br>
> 4. I then ran ganesha.nfsd -f /etc/ganesha/ganesha.conf -L<br>
> /var/log/ganesha.log -N NIV_FULL_DEBUG<br>
> Ganesha server was put in grace, no errors.<br>
><br>
> 17/11/2015 10:44:40 : epoch 564b5964 : ganeshaA:<br>
> nfs-ganesha-26426[reaper] fridgethr_freeze :RW LOCK :F_DBG
:Released<br>
> mutex 0x7f21a92818d0 (&fr->mtx) at<br>
>
/builddir/build/BUILD/nfs-ganesha-2.2.0/src/support/fridgethr.c:484<br>
> 17/11/2015 10:44:40 : epoch 564b5964 : ganeshaA:<br>
> nfs-ganesha-26426[reaper] nfs_in_grace :RW LOCK :F_DBG
:Acquired mutex<br>
> 0x7f21ad1f18e0 (&grace.g_mutex) at<br>
>
/builddir/build/BUILD/nfs-ganesha-2.2.0/src/SAL/nfs4_recovery.c:129<br>
> *17/11/2015 10:44:40 : epoch 564b5964 : ganeshaA :<br>
> nfs-ganesha-26426[reaper] nfs_in_grace :STATE :DEBUG :NFS
Server IN GRACE*<br>
> 17/11/2015 10:44:40 : epoch 564b5964 : ganeshaA :<br>
> nfs-ganesha-26426[reaper] nfs_in_grace :RW LOCK :F_DBG
:Released mutex<br>
> 0x7f21ad1f18e0 (&grace.g_mutex) at<br>
>
/builddir/build/BUILD/nfs-ganesha-2.2.0/src/SAL/nfs4_recovery.c:141<br>
><br>
<br>
You shall still need gluster-client bits on the machine where <br>
nfs-ganesha server is installed to export a gluster volume.
Check if you <br>
have got libgfapi.so installed on that machine.<br>
<br>
Also, ganesha server does log the warnings if its unable to
process the <br>
EXPORT/FSAL block. Please recheck the logs if you have got any.<br>
<br>
Thanks,<br>
Soumya<br>
<br>
> 5. [root@ganeshaA glusterfs]# showmount -e<br>
> Export list for ganeshaA:<br>
> <empty><br>
><br>
> Any suggestions on what I am missing?<br>
><br>
> Regards,<br>
><br>
> Surya Ghatty<br>
><br>
> "This too shall pass"<br>
>
________________________________________________________________________________________________________<br>
> Surya Ghatty | Software Engineer | IBM Cloud Infrastructure
Services<br>
> Development | tel: (507) 316-0559 | <a class="moz-txt-link-abbreviated" href="mailto:ghatty@us.ibm.com">ghatty@us.ibm.com</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> </tt><tt><a moz-do-not-send="true"
href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></tt><tt><br>
><br>
<br>
</tt><br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>