<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 23, 2016 at 4:17 AM, Steve Dainard <span dir="ltr"><<a href="mailto:sdainard@spd1.com" target="_blank">sdainard@spd1.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">About 5 hours after upgrading gluster 3.7.6 -> 3.7.13 on Centos 7, one of my gluster servers disconnected its volume. The other two volumes this host serves were not affected.<br><br><div><div># gluster volume status storage</div><div>Status of volume: storage</div><div>Gluster process TCP Port RDMA Port Online Pid</div><div>------------------------------<wbr>------------------------------<wbr>------------------</div><div>Brick 10.0.231.50:/mnt/raid6-<wbr>storage/storage 49159 0 Y 30743</div><div>Brick 10.0.231.51:/mnt/raid6-<wbr>storage/storage 49159 0 Y 676 </div><div><b>Brick 10.0.231.52:/mnt/raid6-<wbr>storage/storag</b><b>e N/A N/A N N/A </b></div><div>Brick 10.0.231.53:/mnt/raid6-<wbr>storage/storage 49154 0 Y 10253</div><div>Brick 10.0.231.54:/mnt/raid6-<wbr>storage/storage 49153 0 Y 2792 </div><div>Brick 10.0.231.55:/mnt/raid6-<wbr>storage/storage 49153 0 Y 13590</div><div>Brick 10.0.231.56:/mnt/raid6-<wbr>storage/storage 49152 0 Y 9281 </div><div>NFS Server on localhost 2049 0 Y 30775</div><div>Quota Daemon on localhost N/A N/A Y 30781</div><div>NFS Server on 10.0.231.54 2049 0 Y 2817 </div><div>Quota Daemon on 10.0.231.54 N/A N/A Y 2824 </div><div>NFS Server on 10.0.231.51 2049 0 Y 710 </div><div>Quota Daemon on 10.0.231.51 N/A N/A Y 719 </div><div>NFS Server on 10.0.231.52 2049 0 Y 9090 </div><div>Quota Daemon on 10.0.231.52 N/A N/A Y 9098 </div><div>NFS Server on 10.0.231.55 2049 0 Y 13611</div><div>Quota Daemon on 10.0.231.55 N/A N/A Y 13619</div><div>NFS Server on 10.0.231.56 2049 0 Y 9303 </div><div>Quota Daemon on 10.0.231.56 N/A N/A Y 9310 </div><div>NFS Server on 10.0.231.53 2049 0 Y 26304</div><div>Quota Daemon on 10.0.231.53 N/A N/A Y 26320</div><div> </div><div>Task Status of Volume storage</div><div>------------------------------<wbr>------------------------------<wbr>------------------</div><div>There are no active volume tasks</div><div><br></div><div>I see lots of logs related to trashcan (failed [file exists]), set xattrs (failed [no such file or directory]), quota (invalid arguments) in the brick logs, which I enabled as a feature after the upgrade this morning.</div></div></div></blockquote><div><br></div><div>Could you let us know the time (in UTC) around which this issue was seen such that we can look at the logs around that time and see if something went wrong.<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><br></div><div>After restarting glusterd on that host, the volume came back online.</div><div><br></div><div>I've attached logs from that host if someone can take a look.</div><div><br></div><div><div># gluster volume info storage</div><div> </div><div>Volume Name: storage</div><div>Type: Distribute</div><div>Volume ID: 6f95525a-94d7-4174-bac4-<wbr>e1a18fe010a2</div><div>Status: Started</div><div>Number of Bricks: 7</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: 10.0.231.50:/mnt/raid6-<wbr>storage/storage</div><div>Brick2: 10.0.231.51:/mnt/raid6-<wbr>storage/storage</div><div>Brick3: 10.0.231.52:/mnt/raid6-<wbr>storage/storage</div><div>Brick4: 10.0.231.53:/mnt/raid6-<wbr>storage/storage</div><div>Brick5: 10.0.231.54:/mnt/raid6-<wbr>storage/storage</div><div>Brick6: 10.0.231.55:/mnt/raid6-<wbr>storage/storage</div><div>Brick7: 10.0.231.56:/mnt/raid6-<wbr>storage/storage</div><div>Options Reconfigured:</div><div>nfs.disable: no</div><div>features.trash-max-filesize: 1GB</div><div>features.trash: on</div><div>features.quota-deem-statfs: on</div><div>features.inode-quota: on</div><div>features.quota: on</div><div>performance.readdir-ahead: on</div></div><div><br></div><div><div># rpm -qa |grep glusterfs</div><div>glusterfs-fuse-3.7.13-1.el7.<wbr>x86_64</div><div>glusterfs-cli-3.7.13-1.el7.<wbr>x86_64</div><div>glusterfs-3.7.13-1.el7.x86_64</div><div>glusterfs-server-3.7.13-1.el7.<wbr>x86_64</div><div>glusterfs-api-3.7.13-1.el7.<wbr>x86_64</div><div>glusterfs-libs-3.7.13-1.el7.<wbr>x86_64</div><div>glusterfs-client-xlators-3.7.<wbr>13-1.el7.x86_64</div></div><div><br></div><div><br></div><div>Thanks,</div><div>Steve</div><div><br></div></div><div><br></div></div>
<br>______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><br></div>--Atin<br></div></div>
</div></div>