<div dir="ltr">That's correct and my original question was actually if a two node + arbiter setup is possible. The documentation provided by Soumya only mentions two servers in the example ganesha-ha.sh script. Perhaps that could be updated as well then, to not give the wrong impression. <div><br></div><div>I could try to change the script to disable quorum, but wouldn't that defeat the purpose? What will happen in case one node goes down unexpectedly?</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 22 September 2015 at 12:47, Kaleb Keithley <span dir="ltr"><<a href="mailto:kkeithle@redhat.com" target="_blank">kkeithle@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Hi,<br>
<br>
IIRC, the setup is two nodes gluster+ganesha nodes plus the arbiter node for gluster quorum.<br>
<br>
Have I remembered that correctly?<br>
<br>
The Ganesha HA in 3.7 requires a minimum of three servers running ganesha and pacemaker. Two might work if you change the ganesha-ha.sh to not enable pacemaker quorum, but I haven't tried that myself. I'll try and find time in the next couple of days to update the documentation or write a blog post.<br>
<div><div class="h5"><br>
<br>
<br>
----- Original Message ----<br>
><br>
><br>
><br>
> On 21/09/15 21:21, Tiemen Ruiten wrote:<br>
> > Whoops, replied off-list.<br>
> ><br>
> > Additionally I noticed that the generated corosync config is not<br>
> > valid, as there is no interface section:<br>
> ><br>
> > /etc/corosync/corosync.conf<br>
> ><br>
> > totem {<br>
> > version: 2<br>
> > secauth: off<br>
> > cluster_name: rd-ganesha-ha<br>
> > transport: udpu<br>
> > }<br>
> ><br>
> > nodelist {<br>
> > node {<br>
> > ring0_addr: cobalt<br>
> > nodeid: 1<br>
> > }<br>
> > node {<br>
> > ring0_addr: iron<br>
> > nodeid: 2<br>
> > }<br>
> > }<br>
> ><br>
> > quorum {<br>
> > provider: corosync_votequorum<br>
> > two_node: 1<br>
> > }<br>
> ><br>
> > logging {<br>
> > to_syslog: yes<br>
> > }<br>
> ><br>
> ><br>
> ><br>
><br>
> May be Kaleb can help you out.<br>
> ><br>
> > ---------- Forwarded message ----------<br>
</div></div><span class="">> > From: *Tiemen Ruiten* <<a href="mailto:t.ruiten@rdmedia.com">t.ruiten@rdmedia.com</a> <mailto:<a href="mailto:t.ruiten@rdmedia.com">t.ruiten@rdmedia.com</a>>><br>
> > Date: 21 September 2015 at 17:16<br>
> > Subject: Re: [Gluster-users] nfs-ganesha HA with arbiter volume<br>
</span><div><div class="h5">> > To: Jiffin Tony Thottan <<a href="mailto:jthottan@redhat.com">jthottan@redhat.com</a> <mailto:<a href="mailto:jthottan@redhat.com">jthottan@redhat.com</a>>><br>
> ><br>
> ><br>
> > Could you point me to the latest documentation? I've been struggling<br>
> > to find something up-to-date. I believe I have all the prerequisites:<br>
> ><br>
> > - shared storage volume exists and is mounted<br>
> > - all nodes in hosts files<br>
> > - Gluster-NFS disabled<br>
> > - corosync, pacemaker and nfs-ganesha rpm's installed<br>
> ><br>
> > Anything I missed?<br>
> ><br>
> > Everything has been installed by RPM so is in the default locations:<br>
> > /usr/libexec/ganesha/ganesha-ha.sh<br>
> > /etc/ganesha/ganesha.conf (empty)<br>
> > /etc/ganesha/ganesha-ha.conf<br>
> ><br>
><br>
> Looks fine for me.<br>
><br>
> > After I started the pcsd service manually, nfs-ganesha could be<br>
> > enabled successfully, but there was no virtual IP present on the<br>
> > interfaces and looking at the system log, I noticed corosync failed to<br>
> > start:<br>
> ><br>
> > - on the host where I issued the gluster nfs-ganesha enable command:<br>
> ><br>
> > Sep 21 17:07:18 iron systemd: Starting NFS-Ganesha file server...<br>
> > Sep 21 17:07:19 iron systemd: Started NFS-Ganesha file server.<br>
> > Sep 21 17:07:19 iron rpc.statd[2409]: Received SM_UNMON_ALL request<br>
</div></div>> > from <a href="http://iron.int.rdmedia.com" rel="noreferrer" target="_blank">iron.int.rdmedia.com</a> <<a href="http://iron.int.rdmedia.com" rel="noreferrer" target="_blank">http://iron.int.rdmedia.com</a>> while not<br>
<div><div class="h5">> > monitoring any hosts<br>
> > Sep 21 17:07:20 iron systemd: Starting Corosync Cluster Engine...<br>
> > Sep 21 17:07:20 iron corosync[3426]: [MAIN ] Corosync Cluster Engine<br>
> > ('2.3.4'): started and ready to provide service.<br>
> > Sep 21 17:07:20 iron corosync[3426]: [MAIN ] Corosync built-in<br>
> > features: dbus systemd xmlconf snmp pie relro bindnow<br>
> > Sep 21 17:07:20 iron corosync[3427]: [TOTEM ] Initializing transport<br>
> > (UDP/IP Unicast).<br>
> > Sep 21 17:07:20 iron corosync[3427]: [TOTEM ] Initializing<br>
> > transmit/receive security (NSS) crypto: none hash: none<br>
> > Sep 21 17:07:20 iron corosync[3427]: [TOTEM ] The network interface<br>
> > [10.100.30.38] is now up.<br>
> > Sep 21 17:07:20 iron corosync[3427]: [SERV ] Service engine loaded:<br>
> > corosync configuration map access [0]<br>
> > Sep 21 17:07:20 iron corosync[3427]: [QB ] server name: cmap<br>
> > Sep 21 17:07:20 iron corosync[3427]: [SERV ] Service engine loaded:<br>
> > corosync configuration service [1]<br>
> > Sep 21 17:07:20 iron corosync[3427]: [QB ] server name: cfg<br>
> > Sep 21 17:07:20 iron corosync[3427]: [SERV ] Service engine loaded:<br>
> > corosync cluster closed process group service v1.01 [2]<br>
> > Sep 21 17:07:20 iron corosync[3427]: [QB ] server name: cpg<br>
> > Sep 21 17:07:20 iron corosync[3427]: [SERV ] Service engine loaded:<br>
> > corosync profile loading service [4]<br>
> > Sep 21 17:07:20 iron corosync[3427]: [QUORUM] Using quorum provider<br>
> > corosync_votequorum<br>
> > Sep 21 17:07:20 iron corosync[3427]: [VOTEQ ] Waiting for all cluster<br>
> > members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:20 iron corosync[3427]: [SERV ] Service engine loaded:<br>
> > corosync vote quorum service v1.0 [5]<br>
> > Sep 21 17:07:20 iron corosync[3427]: [QB ] server name: votequorum<br>
> > Sep 21 17:07:20 iron corosync[3427]: [SERV ] Service engine loaded:<br>
> > corosync cluster quorum service v0.1 [3]<br>
> > Sep 21 17:07:20 iron corosync[3427]: [QB ] server name: quorum<br>
> > Sep 21 17:07:20 iron corosync[3427]: [TOTEM ] adding new UDPU member<br>
> > {10.100.30.38}<br>
> > Sep 21 17:07:20 iron corosync[3427]: [TOTEM ] adding new UDPU member<br>
> > {10.100.30.37}<br>
> > Sep 21 17:07:20 iron corosync[3427]: [TOTEM ] A new membership<br>
</div></div>> > (<a href="http://10.100.30.38:104" rel="noreferrer" target="_blank">10.100.30.38:104</a> <<a href="http://10.100.30.38:104" rel="noreferrer" target="_blank">http://10.100.30.38:104</a>>) was formed. Members joined: 1<br>
<span class="">> > Sep 21 17:07:20 iron corosync[3427]: [VOTEQ ] Waiting for all cluster<br>
> > members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:20 iron corosync[3427]: [VOTEQ ] Waiting for all cluster<br>
> > members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:20 iron corosync[3427]: [VOTEQ ] Waiting for all cluster<br>
> > members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:20 iron corosync[3427]: [QUORUM] Members[1]: 1<br>
> > Sep 21 17:07:20 iron corosync[3427]: [MAIN ] Completed service<br>
> > synchronization, ready to provide service.<br>
> > Sep 21 17:07:20 iron corosync[3427]: [TOTEM ] A new membership<br>
</span>> > (<a href="http://10.100.30.37:108" rel="noreferrer" target="_blank">10.100.30.37:108</a> <<a href="http://10.100.30.37:108" rel="noreferrer" target="_blank">http://10.100.30.37:108</a>>) was formed. Members joined: 1<br>
<div><div class="h5">> > Sep 21 17:08:21 iron corosync: Starting Corosync Cluster Engine<br>
> > (corosync): [FAILED]<br>
> > Sep 21 17:08:21 iron systemd: corosync.service: control process<br>
> > exited, code=exited status=1<br>
> > Sep 21 17:08:21 iron systemd: Failed to start Corosync Cluster Engine.<br>
> > Sep 21 17:08:21 iron systemd: Unit corosync.service entered failed state.<br>
> ><br>
> ><br>
> > - on the other host:<br>
> ><br>
> > Sep 21 17:07:19 cobalt systemd: Starting Preprocess NFS configuration...<br>
> > Sep 21 17:07:19 cobalt systemd: Starting RPC Port Mapper.<br>
> > Sep 21 17:07:19 cobalt systemd: Reached target RPC Port Mapper.<br>
> > Sep 21 17:07:19 cobalt systemd: Starting Host and Network Name Lookups.<br>
> > Sep 21 17:07:19 cobalt systemd: Reached target Host and Network Name<br>
> > Lookups.<br>
> > Sep 21 17:07:19 cobalt systemd: Starting RPC bind service...<br>
> > Sep 21 17:07:19 cobalt systemd: Started Preprocess NFS configuration.<br>
> > Sep 21 17:07:19 cobalt systemd: Started RPC bind service.<br>
> > Sep 21 17:07:19 cobalt systemd: Starting NFS status monitor for<br>
> > NFSv2/3 locking....<br>
> > Sep 21 17:07:19 cobalt rpc.statd[2662]: Version 1.3.0 starting<br>
> > Sep 21 17:07:19 cobalt rpc.statd[2662]: Flags: TI-RPC<br>
> > Sep 21 17:07:19 cobalt systemd: Started NFS status monitor for NFSv2/3<br>
> > locking..<br>
> > Sep 21 17:07:19 cobalt systemd: Starting NFS-Ganesha file server...<br>
> > Sep 21 17:07:19 cobalt systemd: Started NFS-Ganesha file server.<br>
> > Sep 21 17:07:19 cobalt kernel: warning: `ganesha.nfsd' uses 32-bit<br>
> > capabilities (legacy support in use)<br>
> > Sep 21 17:07:19 cobalt logger: setting up rd-ganesha-ha<br>
> > Sep 21 17:07:19 cobalt rpc.statd[2662]: Received SM_UNMON_ALL request<br>
</div></div>> > from <a href="http://cobalt.int.rdmedia.com" rel="noreferrer" target="_blank">cobalt.int.rdmedia.com</a> <<a href="http://cobalt.int.rdmedia.com" rel="noreferrer" target="_blank">http://cobalt.int.rdmedia.com</a>> while not<br>
<div><div class="h5">> > monitoring any hosts<br>
> > Sep 21 17:07:19 cobalt logger: setting up cluster rd-ganesha-ha with<br>
> > the following cobalt iron<br>
> > Sep 21 17:07:20 cobalt systemd: Stopped Pacemaker High Availability<br>
> > Cluster Manager.<br>
> > Sep 21 17:07:20 cobalt systemd: Stopped Corosync Cluster Engine.<br>
> > Sep 21 17:07:20 cobalt systemd: Reloading.<br>
> > Sep 21 17:07:20 cobalt systemd:<br>
> > [/usr/lib/systemd/system/dm-event.socket:10] Unknown lvalue<br>
> > 'RemoveOnStop' in section 'Socket'<br>
> > Sep 21 17:07:20 cobalt systemd:<br>
> > [/usr/lib/systemd/system/lvm2-lvmetad.socket:9] Unknown lvalue<br>
> > 'RemoveOnStop' in section 'Socket'<br>
> > Sep 21 17:07:20 cobalt systemd: Reloading.<br>
> > Sep 21 17:07:20 cobalt systemd:<br>
> > [/usr/lib/systemd/system/dm-event.socket:10] Unknown lvalue<br>
> > 'RemoveOnStop' in section 'Socket'<br>
> > Sep 21 17:07:20 cobalt systemd:<br>
> > [/usr/lib/systemd/system/lvm2-lvmetad.socket:9] Unknown lvalue<br>
> > 'RemoveOnStop' in section 'Socket'<br>
> > Sep 21 17:07:20 cobalt systemd: Starting Corosync Cluster Engine...<br>
> > Sep 21 17:07:20 cobalt corosync[2816]: [MAIN ] Corosync Cluster<br>
> > Engine ('2.3.4'): started and ready to provide service.<br>
> > Sep 21 17:07:20 cobalt corosync[2816]: [MAIN ] Corosync built-in<br>
> > features: dbus systemd xmlconf snmp pie relro bindnow<br>
> > Sep 21 17:07:20 cobalt corosync[2817]: [TOTEM ] Initializing transport<br>
> > (UDP/IP Unicast).<br>
> > Sep 21 17:07:20 cobalt corosync[2817]: [TOTEM ] Initializing<br>
> > transmit/receive security (NSS) crypto: none hash: none<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [TOTEM ] The network interface<br>
> > [10.100.30.37] is now up.<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [SERV ] Service engine loaded:<br>
> > corosync configuration map access [0]<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QB ] server name: cmap<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [SERV ] Service engine loaded:<br>
> > corosync configuration service [1]<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QB ] server name: cfg<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [SERV ] Service engine loaded:<br>
> > corosync cluster closed process group service v1.01 [2]<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QB ] server name: cpg<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [SERV ] Service engine loaded:<br>
> > corosync profile loading service [4]<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QUORUM] Using quorum provider<br>
> > corosync_votequorum<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [VOTEQ ] Waiting for all<br>
> > cluster members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [SERV ] Service engine loaded:<br>
> > corosync vote quorum service v1.0 [5]<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QB ] server name: votequorum<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [SERV ] Service engine loaded:<br>
> > corosync cluster quorum service v0.1 [3]<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QB ] server name: quorum<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [TOTEM ] adding new UDPU member<br>
> > {10.100.30.37}<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [TOTEM ] adding new UDPU member<br>
> > {10.100.30.38}<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [TOTEM ] A new membership<br>
</div></div>> > (<a href="http://10.100.30.37:100" rel="noreferrer" target="_blank">10.100.30.37:100</a> <<a href="http://10.100.30.37:100" rel="noreferrer" target="_blank">http://10.100.30.37:100</a>>) was formed. Members joined: 1<br>
<span class="">> > Sep 21 17:07:21 cobalt corosync[2817]: [VOTEQ ] Waiting for all<br>
> > cluster members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [VOTEQ ] Waiting for all<br>
> > cluster members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [VOTEQ ] Waiting for all<br>
> > cluster members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QUORUM] Members[1]: 1<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [MAIN ] Completed service<br>
> > synchronization, ready to provide service.<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [TOTEM ] A new membership<br>
</span>> > (<a href="http://10.100.30.37:108" rel="noreferrer" target="_blank">10.100.30.37:108</a> <<a href="http://10.100.30.37:108" rel="noreferrer" target="_blank">http://10.100.30.37:108</a>>) was formed. Members joined: 1<br>
<div><div class="h5">> > Sep 21 17:07:21 cobalt corosync[2817]: [VOTEQ ] Waiting for all<br>
> > cluster members. Current votes: 1 expected_votes: 2<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [QUORUM] Members[1]: 1<br>
> > Sep 21 17:07:21 cobalt corosync[2817]: [MAIN ] Completed service<br>
> > synchronization, ready to provide service.<br>
> > Sep 21 17:08:50 cobalt systemd: corosync.service operation timed out.<br>
> > Terminating.<br>
> > Sep 21 17:08:50 cobalt corosync: Starting Corosync Cluster Engine<br>
> > (corosync):<br>
> > Sep 21 17:08:50 cobalt systemd: Failed to start Corosync Cluster Engine.<br>
> > Sep 21 17:08:50 cobalt systemd: Unit corosync.service entered failed<br>
> > state.<br>
> > Sep 21 17:08:55 cobalt logger: warning: pcs property set<br>
> > no-quorum-policy=ignore failed<br>
> > Sep 21 17:08:55 cobalt logger: warning: pcs property set<br>
> > stonith-enabled=false failed<br>
> > Sep 21 17:08:55 cobalt logger: warning: pcs resource create nfs_start<br>
> > ganesha_nfsd ha_vol_mnt=/var/run/gluster/shared_storage --clone failed<br>
> > Sep 21 17:08:56 cobalt logger: warning: pcs resource delete<br>
> > nfs_start-clone failed<br>
> > Sep 21 17:08:56 cobalt logger: warning: pcs resource create nfs-mon<br>
> > ganesha_mon --clone failed<br>
> > Sep 21 17:08:56 cobalt logger: warning: pcs resource create nfs-grace<br>
> > ganesha_grace --clone failed<br>
> > Sep 21 17:08:57 cobalt logger: warning pcs resource create<br>
> > cobalt-cluster_ip-1 ocf:heartbeat:IPaddr ip= cidr_netmask=32 op<br>
> > monitor interval=15s failed<br>
> > Sep 21 17:08:57 cobalt logger: warning: pcs resource create<br>
> > cobalt-trigger_ip-1 ocf:heartbeat:Dummy failed<br>
> > Sep 21 17:08:57 cobalt logger: warning: pcs constraint colocation add<br>
> > cobalt-cluster_ip-1 with cobalt-trigger_ip-1 failed<br>
> > Sep 21 17:08:57 cobalt logger: warning: pcs constraint order<br>
> > cobalt-trigger_ip-1 then nfs-grace-clone failed<br>
> > Sep 21 17:08:57 cobalt logger: warning: pcs constraint order<br>
> > nfs-grace-clone then cobalt-cluster_ip-1 failed<br>
> > Sep 21 17:08:57 cobalt logger: warning pcs resource create<br>
> > iron-cluster_ip-1 ocf:heartbeat:IPaddr ip= cidr_netmask=32 op monitor<br>
> > interval=15s failed<br>
> > Sep 21 17:08:57 cobalt logger: warning: pcs resource create<br>
> > iron-trigger_ip-1 ocf:heartbeat:Dummy failed<br>
> > Sep 21 17:08:57 cobalt logger: warning: pcs constraint colocation add<br>
> > iron-cluster_ip-1 with iron-trigger_ip-1 failed<br>
> > Sep 21 17:08:57 cobalt logger: warning: pcs constraint order<br>
> > iron-trigger_ip-1 then nfs-grace-clone failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: pcs constraint order<br>
> > nfs-grace-clone then iron-cluster_ip-1 failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: pcs constraint location<br>
> > cobalt-cluster_ip-1 rule score=-INFINITY ganesha-active ne 1 failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: pcs constraint location<br>
> > cobalt-cluster_ip-1 prefers iron=1000 failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: pcs constraint location<br>
> > cobalt-cluster_ip-1 prefers cobalt=2000 failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: pcs constraint location<br>
> > iron-cluster_ip-1 rule score=-INFINITY ganesha-active ne 1 failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: pcs constraint location<br>
> > iron-cluster_ip-1 prefers cobalt=1000 failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: pcs constraint location<br>
> > iron-cluster_ip-1 prefers iron=2000 failed<br>
> > Sep 21 17:08:58 cobalt logger: warning pcs cluster cib-push<br>
> > /tmp/tmp.nXTfyA1GMR failed<br>
> > Sep 21 17:08:58 cobalt logger: warning: scp ganesha-ha.conf to cobalt<br>
> > failed<br>
> ><br>
> > BTW, I'm using CentOS 7. There are multiple network interfaces on the<br>
> > servers, could that be a problem?<br>
> ><br>
> ><br>
> ><br>
> ><br>
> > On 21 September 2015 at 11:48, Jiffin Tony Thottan<br>
</div></div><div><div class="h5">> > <<a href="mailto:jthottan@redhat.com">jthottan@redhat.com</a> <mailto:<a href="mailto:jthottan@redhat.com">jthottan@redhat.com</a>>> wrote:<br>
> ><br>
> ><br>
> ><br>
> > On 21/09/15 13:56, Tiemen Ruiten wrote:<br>
> >> Hello Soumya, Kaleb, list,<br>
> >><br>
> >> This Friday I created the gluster_shared_storage volume manually,<br>
> >> I just tried it with the command you supplied, but both have the<br>
> >> same result:<br>
> >><br>
> >> from etc-glusterfs-glusterd.vol.log on the node where I issued<br>
> >> the command:<br>
> >><br>
> >> [2015-09-21 07:59:47.756845] I [MSGID: 106474]<br>
> >> [glusterd-ganesha.c:403:check_host_list] 0-management: ganesha<br>
> >> host found Hostname is cobalt<br>
> >> [2015-09-21 07:59:48.071755] I [MSGID: 106474]<br>
> >> [glusterd-ganesha.c:349:is_ganesha_host] 0-management: ganesha<br>
> >> host found Hostname is cobalt<br>
> >> [2015-09-21 07:59:48.653879] E [MSGID: 106470]<br>
> >> [glusterd-ganesha.c:264:glusterd_op_set_ganesha] 0-management:<br>
> >> Initial NFS-Ganesha set up failed<br>
> ><br>
> > As far as what I understand from the logs, it called<br>
> > setup_cluser()[calls `ganesha-ha.sh` script ] but script failed.<br>
> > Can u please provide following details :<br>
> > -Location of ganesha.sh file??<br>
> > -Location of ganesha-ha.conf, ganesha.conf files ?<br>
> ><br>
> ><br>
> > And also can u cross check whether all the prerequisites before HA<br>
> > setup satisfied ?<br>
> ><br>
> > --<br>
> > With Regards,<br>
> > Jiffin<br>
> ><br>
> ><br>
> >> [2015-09-21 07:59:48.653912] E [MSGID: 106123]<br>
> >> [glusterd-syncop.c:1404:gd_commit_op_phase] 0-management: Commit<br>
> >> of operation 'Volume (null)' failed on localhost : Failed to set<br>
> >> up HA config for NFS-Ganesha. Please check the log file for details<br>
> >> [2015-09-21 07:59:45.402458] I [MSGID: 106006]<br>
> >> [glusterd-svc-mgmt.c:323:glusterd_svc_common_rpc_notify]<br>
> >> 0-management: nfs has disconnected from glusterd.<br>
> >> [2015-09-21 07:59:48.071578] I [MSGID: 106474]<br>
> >> [glusterd-ganesha.c:403:check_host_list] 0-management: ganesha<br>
> >> host found Hostname is cobalt<br>
> >><br>
> >> from etc-glusterfs-glusterd.vol.log on the other node:<br>
> >><br>
> >> [2015-09-21 08:12:50.111877] E [MSGID: 106062]<br>
> >> [glusterd-op-sm.c:3698:glusterd_op_ac_unlock] 0-management:<br>
> >> Unable to acquire volname<br>
> >> [2015-09-21 08:14:50.548087] E [MSGID: 106062]<br>
> >> [glusterd-op-sm.c:3635:glusterd_op_ac_lock] 0-management: Unable<br>
> >> to acquire volname<br>
> >> [2015-09-21 08:14:50.654746] I [MSGID: 106132]<br>
> >> [glusterd-proc-mgmt.c:83:glusterd_proc_stop] 0-management: nfs<br>
> >> already stopped<br>
> >> [2015-09-21 08:14:50.655095] I [MSGID: 106474]<br>
> >> [glusterd-ganesha.c:403:check_host_list] 0-management: ganesha<br>
> >> host found Hostname is cobalt<br>
> >> [2015-09-21 08:14:51.287156] E [MSGID: 106062]<br>
> >> [glusterd-op-sm.c:3698:glusterd_op_ac_unlock] 0-management:<br>
> >> Unable to acquire volname<br>
> >><br>
> >><br>
> >> from etc-glusterfs-glusterd.vol.log on the arbiter node:<br>
> >><br>
> >> [2015-09-21 08:18:50.934713] E [MSGID: 101075]<br>
> >> [common-utils.c:3127:gf_is_local_addr] 0-management: error in<br>
> >> getaddrinfo: Name or service not known<br>
> >> [2015-09-21 08:18:51.504694] E [MSGID: 106062]<br>
> >> [glusterd-op-sm.c:3698:glusterd_op_ac_unlock] 0-management:<br>
> >> Unable to acquire volname<br>
> >><br>
> >> I have put the hostnames of all servers in my /etc/hosts file,<br>
> >> including the arbiter node.<br>
> >><br>
> >><br>
> >> On 18 September 2015 at 16:52, Soumya Koduri <<a href="mailto:skoduri@redhat.com">skoduri@redhat.com</a><br>
</div></div><div><div class="h5">> >> <mailto:<a href="mailto:skoduri@redhat.com">skoduri@redhat.com</a>>> wrote:<br>
> >><br>
> >> Hi Tiemen,<br>
> >><br>
> >> One of the pre-requisites before setting up nfs-ganesha HA is<br>
> >> to create and mount shared_storage volume. Use below CLI for that<br>
> >><br>
> >> "gluster volume set all cluster.enable-shared-storage enable"<br>
> >><br>
> >> It shall create the volume and mount in all the nodes<br>
> >> (including the arbiter node). Note this volume shall be<br>
> >> mounted on all the nodes of the gluster storage pool (though<br>
> >> in this case it may not be part of nfs-ganesha cluster).<br>
> >><br>
> >> So instead of manually creating those directory paths, please<br>
> >> use above CLI and try re-configuring the setup.<br>
> >><br>
> >> Thanks,<br>
> >> Soumya<br>
> >><br>
> >> On 09/18/2015 07:29 PM, Tiemen Ruiten wrote:<br>
> >><br>
> >> Hello Kaleb,<br>
> >><br>
> >> I don't:<br>
> >><br>
> >> # Name of the HA cluster created.<br>
> >> # must be unique within the subnet<br>
> >> HA_NAME="rd-ganesha-ha"<br>
> >> #<br>
> >> # The gluster server from which to mount the shared data<br>
> >> volume.<br>
> >> HA_VOL_SERVER="iron"<br>
> >> #<br>
> >> # N.B. you may use short names or long names; you may not<br>
> >> use IP addrs.<br>
> >> # Once you select one, stay with it as it will be mildly<br>
> >> unpleasant to<br>
> >> # clean up if you switch later on. Ensure that all names<br>
> >> - short and/or<br>
> >> # long - are in DNS or /etc/hosts on all machines in the<br>
> >> cluster.<br>
> >> #<br>
> >> # The subset of nodes of the Gluster Trusted Pool that<br>
> >> form the ganesha<br>
> >> # HA cluster. Hostname is specified.<br>
> >> HA_CLUSTER_NODES="cobalt,iron"<br>
> >> #HA_CLUSTER_NODES="<a href="http://server1.lab.redhat.com" rel="noreferrer" target="_blank">server1.lab.redhat.com</a><br>
> >> <<a href="http://server1.lab.redhat.com" rel="noreferrer" target="_blank">http://server1.lab.redhat.com</a>><br>
> >> <<a href="http://server1.lab.redhat.com" rel="noreferrer" target="_blank">http://server1.lab.redhat.com</a>>,<a href="http://server2.lab.redhat.com" rel="noreferrer" target="_blank">server2.lab.redhat.com</a><br>
> >> <<a href="http://server2.lab.redhat.com" rel="noreferrer" target="_blank">http://server2.lab.redhat.com</a>><br>
> >> <<a href="http://server2.lab.redhat.com" rel="noreferrer" target="_blank">http://server2.lab.redhat.com</a>>,..."<br>
> >> #<br>
> >> # Virtual IPs for each of the nodes specified above.<br>
> >> VIP_server1="10.100.30.101"<br>
> >> VIP_server2="10.100.30.102"<br>
> >> #VIP_server1_lab_redhat_com="10.0.2.1"<br>
> >> #VIP_server2_lab_redhat_com="10.0.2.2"<br>
> >><br>
> >> hosts cobalt & iron are the data nodes, the arbiter<br>
> >> ip/hostname (neon)<br>
> >> isn't mentioned anywhere in this config file.<br>
> >><br>
> >><br>
> >> On 18 September 2015 at 15:56, Kaleb S. KEITHLEY<br>
> >> <<a href="mailto:kkeithle@redhat.com">kkeithle@redhat.com</a> <mailto:<a href="mailto:kkeithle@redhat.com">kkeithle@redhat.com</a>><br>
</div></div>> >> <mailto:<a href="mailto:kkeithle@redhat.com">kkeithle@redhat.com</a><br>
<div><div class="h5">> >> <mailto:<a href="mailto:kkeithle@redhat.com">kkeithle@redhat.com</a>>>> wrote:<br>
> >><br>
> >> On 09/18/2015 09:46 AM, Tiemen Ruiten wrote:<br>
> >> > Hello,<br>
> >> ><br>
> >> > I have a Gluster cluster with a single replica 3,<br>
> >> arbiter 1 volume (so<br>
> >> > two nodes with actual data, one arbiter node). I<br>
> >> would like to setup<br>
> >> > NFS-Ganesha HA for this volume but I'm having some<br>
> >> difficulties.<br>
> >> ><br>
> >> > - I needed to create a directory<br>
> >> /var/run/gluster/shared_storage<br>
> >> > manually on all nodes, or the command 'gluster<br>
> >> nfs-ganesha enable would<br>
> >> > fail with the following error:<br>
> >> > [2015-09-18 13:13:34.690416] E [MSGID: 106032]<br>
> >> > [glusterd-ganesha.c:708:pre_setup] 0-THIS->name:<br>
> >> mkdir() failed on path<br>
> >> > /var/run/gluster/shared_storage/nfs-ganesha, [No<br>
> >> such file or directory]<br>
> >> ><br>
> >> > - Then I found out that the command connects to the<br>
> >> arbiter node as<br>
> >> > well, but obviously I don't want to set up<br>
> >> NFS-Ganesha there. Is it<br>
> >> > actually possible to setup NFS-Ganesha HA with an<br>
> >> arbiter node? If it's<br>
> >> > possible, is there any documentation on how to do that?<br>
> >> ><br>
> >><br>
> >> Please send the /etc/ganesha/ganesha-ha.conf file<br>
> >> you're using.<br>
> >><br>
> >> Probably you have included the arbiter in your HA<br>
> >> config; that would be<br>
> >> a mistake.<br>
> >><br>
> >> --<br>
> >><br>
> >> Kaleb<br>
> >><br>
> >><br>
> >><br>
> >><br>
> >> --<br>
> >> Tiemen Ruiten<br>
> >> Systems Engineer<br>
> >> R&D Media<br>
> >><br>
> >><br>
> >> _______________________________________________<br>
> >> Gluster-users mailing list<br>
</div></div>> >> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a> <mailto:<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>><br>
<span class="">> >> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
> >><br>
> >><br>
> >><br>
> >><br>
> >> --<br>
> >> Tiemen Ruiten<br>
> >> Systems Engineer<br>
> >> R&D Media<br>
> >><br>
> >><br>
> >> _______________________________________________<br>
> >> Gluster-users mailing list<br>
</span>> >> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a> <mailto:<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>><br>
<span class="">> >> <a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > Gluster-users mailing list<br>
</span>> > <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a> <mailto:<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>><br>
<span class="im HOEnZb">> > <a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > Tiemen Ruiten<br>
> > Systems Engineer<br>
> > R&D Media<br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > Tiemen Ruiten<br>
> > Systems Engineer<br>
> > R&D Media<br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > Gluster-users mailing list<br>
> > <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> > <a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
><br>
><br>
</span><div class="HOEnZb"><div class="h5">_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">Tiemen Ruiten<br>Systems Engineer<br>R&D Media<br></div></div>
</div>