<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
On 5/27/2015 1:17 PM, Atin Mukherjee wrote:<br>
<blockquote cite="mid:5565FC2F.703@redhat.com" type="cite">
<pre wrap="">On 05/27/2015 07:33 PM, Ted Miller wrote:
</pre>
<blockquote type="cite">
<pre wrap="">responses below
Ted Miller
On 5/26/2015 12:01 AM, Atin Mukherjee wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On 05/26/2015 03:12 AM, Ted Miller wrote:
</pre>
<blockquote type="cite">
<pre wrap="">From: Niels de Vos <a class="moz-txt-link-rfc2396E" href="mailto:ndevos@redhat.com"><ndevos@redhat.com></a>
Sent: Monday, May 25, 2015 4:44 PM
On Mon, May 25, 2015 at 06:49:26PM +0000, Ted Miller wrote:
</pre>
<blockquote type="cite">
<pre wrap="">From: Humble Devassy Chirammal <a class="moz-txt-link-rfc2396E" href="mailto:humble.devassy@gmail.com"><humble.devassy@gmail.com></a>
Sent: Monday, May 18, 2015 9:37 AM
Hi All,
GlusterFS 3.7.0 RPMs for RHEL, CentOS, Fedora and packages for
Debian are available at
download.gluster.org<a class="moz-txt-link-rfc2396E" href="http://download.gluster.org"><http://download.gluster.org></a> [1].
[1] <a class="moz-txt-link-freetext" href="http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.0/">http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.0/</a>
--Humble
On Thu, May 14, 2015 at 2:49 PM, Vijay Bellur
<<a class="moz-txt-link-abbreviated" href="mailto:vbellur@redhat.com">vbellur@redhat.com</a><a class="moz-txt-link-rfc2396E" href="mailto:vbellur@redhat.com"><mailto:vbellur@redhat.com></a>> wrote:
Hi All,
I am happy to announce that Gluster 3.7.0 is now generally
available. 3.7.0 contains several
[snip]
Cheers,
Vijay
[snip]
</pre>
</blockquote>
<pre wrap="">[snip]
I have no idea about the problem below, it sounds like something the
GlusterD developers could help with.
Niels
</pre>
<blockquote type="cite">
<pre wrap="">Command 'gluster volume status' on the C5 machine makes everything
look fine:
Status of volume: ISO2
Gluster process Port
Online Pid
------------------------------------------------------------------------------
Brick 10.x.x.2:/bricks/01/iso2 49162
Y 4679
Brick 10.x.x.4:/bricks/01/iso2 49183
Y 6447
Brick 10.x.x.9:/bricks/01/iso2 49169
Y 1985
But the same command on either of the C6 machines shows the C5 machine
(10.x.x.2) missing in action (though it does recognize that there are
NFS and heal daemons there):
Status of volume: ISO2
Gluster process TCP Port RDMA Port
Online Pid
------------------------------------------------------------------------------
Brick 10.41.65.4:/bricks/01/iso2 49183 0
Y 6447
Brick 10.41.65.9:/bricks/01/iso2 49169 0
Y 1985
NFS Server on localhost 2049 0
Y 2279
Self-heal Daemon on localhost N/A N/A
Y 2754
NFS Server on 10.41.65.2 2049 0
Y 4757
Self-heal Daemon on 10.41.65.2 N/A N/A
Y 4764
NFS Server on 10.41.65.4 2049 0
Y 6543
Self-heal Daemon on 10.41.65.4 N/A N/A
Y 6551
So, is this just an oversight (I hope), or has support for C5 been
dropped?
If support for C5 is gone, how do I downgrade my Centos6 machines back
to 3.6.x? (I know how to change the repo, but the actual sequence of
yum commands and gluster commands is unknown to me).
</pre>
</blockquote>
</blockquote>
<pre wrap="">Could you attach the glusterd log file of 10.x.x.2 machine
</pre>
</blockquote>
<pre wrap="">attached as etc-glusterfs-glusterd.vol.log.newer.2, starting from last
machine reboot
</pre>
<blockquote type="cite">
<pre wrap=""> and the node from where you triggered volume status.
</pre>
</blockquote>
<pre wrap="">attached as etc-glusterfs-glusterd.vol.log.newer4 starting same time as
.2 log
</pre>
<blockquote type="cite">
<pre wrap="">Could you also share gluster volume info output of all the nodes?
</pre>
</blockquote>
<pre wrap="">I have several volumes, so I chose the one that shows up first on the
listings:
*from 10.41.65.2:*
[root@office2 /var/log/glusterfs]$ gluster volume info
Volume Name: ISO2
Type: Replicate
Volume ID: 090da4b3-c666-41fe-8283-2c029228b3f7
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.41.65.2:/bricks/01/iso2
Brick2: 10.41.65.4:/bricks/01/iso2
Brick3: 10.41.65.9:/bricks/01/iso2
[root@office2 /var/log/glusterfs]$ gluster volume status ISO2
Status of volume: ISO2
Gluster process Port Online Pid
------------------------------------------------------------------------------
Brick 10.41.65.2:/bricks/01/iso2 49162 Y 4463
Brick 10.41.65.4:/bricks/01/iso2 49183 Y 6447
Brick 10.41.65.9:/bricks/01/iso2 49169 Y 1985
NFS Server on localhost 2049 Y 4536
Self-heal Daemon on localhost N/A Y 4543
NFS Server on 10.41.65.9 2049 Y 2279
Self-heal Daemon on 10.41.65.9 N/A Y 2754
NFS Server on 10.41.65.4 2049 Y 6543
Self-heal Daemon on 10.41.65.4 N/A Y 6551
Task Status of Volume ISO2
------------------------------------------------------------------------------
There are no active volume tasks
[root@office2 ~]$ gluster peer status
Number of Peers: 2
Hostname: 10.41.65.9
Uuid: cf2ae9c7-833e-4a73-a996-e72158011c69
State: Peer in Cluster (Connected)
Hostname: 10.41.65.4
Uuid: bd3ca8b7-f2da-44ce-8739-c0db5e40158c
State: Peer in Cluster (Connected)
*from 10.41.65.4:*
[root@office4b ~]# gluster volume info ISO2
Volume Name: ISO2
Type: Replicate
Volume ID: 090da4b3-c666-41fe-8283-2c029228b3f7
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.41.65.2:/bricks/01/iso2
Brick2: 10.41.65.4:/bricks/01/iso2
Brick3: 10.41.65.9:/bricks/01/iso2
[root@office4b ~]# gluster volume status ISO2
Status of volume: ISO2
Gluster process TCP Port RDMA Port Online
Pid
------------------------------------------------------------------------------
Brick 10.41.65.4:/bricks/01/iso2 49183 0 Y 6447
Brick 10.41.65.9:/bricks/01/iso2 49169 0 Y 1985
NFS Server on localhost 2049 0 Y 6543
Self-heal Daemon on localhost N/A N/A Y 6551
NFS Server on 10.41.65.2 2049 0 Y 4536
Self-heal Daemon on 10.41.65.2 N/A N/A Y 4543
NFS Server on 10.41.65.9 2049 0 Y 2279
Self-heal Daemon on 10.41.65.9 N/A N/A Y 2754
Task Status of Volume ISO2
------------------------------------------------------------------------------
There are no active volume tasks
[root@office4b ~]# gluster peer status
Number of Peers: 2
Hostname: 10.41.65.2
Uuid: 4a53ed8b-2b41-4a3c-acf7-2dabec431f57
State: Peer in Cluster (Connected)
Hostname: 10.41.65.9
Uuid: cf2ae9c7-833e-4a73-a996-e72158011c69
State: Peer in Cluster (Connected)
*from 10.41.65.9:*
[root@office9 ~]$ gluster volume info ISO2
Volume Name: ISO2
Type: Replicate
Volume ID: 090da4b3-c666-41fe-8283-2c029228b3f7
Status: Started
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: 10.41.65.2:/bricks/01/iso2
Brick2: 10.41.65.4:/bricks/01/iso2
Brick3: 10.41.65.9:/bricks/01/iso2
[root@office9 ~]$ gluster volume status ISO2
Status of volume: ISO2
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick 10.41.65.4:/bricks/01/iso2 49183 0 Y 6447
Brick 10.41.65.9:/bricks/01/iso2 49169 0 Y 1985
NFS Server on localhost 2049 0 Y 2279
Self-heal Daemon on localhost N/A N/A Y 2754
NFS Server on 10.41.65.2 2049 0 Y 4536
Self-heal Daemon on 10.41.65.2 N/A N/A Y 4543
NFS Server on 10.41.65.4 2049 0 Y 6543
Self-heal Daemon on 10.41.65.4 N/A N/A Y 6551
Task Status of Volume ISO2
------------------------------------------------------------------------------
There are no active volume tasks
[root@office9 ~]$ gluster peer status
Number of Peers: 2
Hostname: 10.41.65.2
Uuid: 4a53ed8b-2b41-4a3c-acf7-2dabec431f57
State: Peer in Cluster (Connected)
Hostname: 10.41.65.4
Uuid: bd3ca8b7-f2da-44ce-8739-c0db5e40158c
State: Peer in Cluster (Connected)
</pre>
</blockquote>
<pre wrap="">Would it be possible for you to kill and
restart glusterd with glusterd -LDEBUG at 2 & 4 and share the complete
log file for both of them?
</pre>
</blockquote>
First--an observation:<br>
<u>The issue seems to be with the gluster volume status command</u>
on the 3.7 machines, not with the actual gluster software. I use
gkrellm to monitor all three machines on my workstation (which is
node 4 {of nodes 2,4,9}). Over the weekend I dumped some files onto
the gluster file system from node 4 (version 3.7), and noticed that
I saw the usual pattern of network and disk activity <i>including
on node 2</i> (version 3.6).<br>
<br>
I had created a new directory, so I did an ls on the corresponding
directory on the brick, and found that my new directory had indeed
been created and populated with files on all three nodes. So, it
looks like glusterd and the 3.7 client are working properly, but the
'gluster volume status' command is not handling the 3.6 node
correctly. "gluster volume status" shows the NFS server and
self-heal daemon information for the 3.6 node, but not the Brick
information.<br>
<br>
The 3.6 node shows status for all three nodes correctly (as far as I
can tell).<br>
<br>
I am attaching the log files. They had both been rotated within the
previous 48 hours, so I did:<br>
<br>
on 2 + 4: service glusterd stop<br>
on 4: glusterd -LDEBUG<br>
on 2: glusterd -LDEBUG<br>
after a short time I copied the logs for attachment.<br>
<br>
<div class="moz-signature">-- <br>
<font color="#3366ff" face="Helvetica, Arial, sans-serif"><big><font
color="#000099"><b>Ted Miller</b></font>, Design Engineer<br>
<b>SonSet Solutions</b></big><br>
(formerly HCJB Global Technology Center)<br>
my desk +1 574.970.4272<br>
receptionist +1 574.972.4252<br>
<a href="http://sonsetsolutions.org">http://sonsetsolutions.org</a></font><br>
<br>
<font color="#996633"><i>Technology for abundant life!</i></font></div>
</body>
</html>