<div dir="ltr"><br><div>Hi,</div><div><br></div><div>Thanks for the snapd log. Can you please attach all the gluster logs? i.e. the contents of /var/log/glusterfs.</div><div><br></div><div>Regards,</div><div>Raghavendra</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 25, 2016 at 11:27 AM, Alastair Neil <span dir="ltr"><<a href="mailto:ajneil.tech@gmail.com" target="_blank">ajneil.tech@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">attached compressed log</div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 22 April 2016 at 20:15, FNU Raghavendra Manjunath <span dir="ltr"><<a href="mailto:rabhat@redhat.com" target="_blank">rabhat@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div>Hi Alastair,</div><div><br></div><div>Can you please provide the snap daemon logs. It is present in /var/log/glusterfs/snaps/snapd.log.</div><div><br></div><div>Provide the snapd logs of the node from which you have mounted the volume (i.e. the node whose ip address/hostname you have given while mounting the volume).</div><div><br></div><div>Regards,</div><div>Raghavendra</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Fri, Apr 22, 2016 at 5:19 PM, Alastair Neil <span dir="ltr"><<a href="mailto:ajneil.tech@gmail.com" target="_blank">ajneil.tech@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">I just upgraded my cluster to 3.7.11 from 3.7.10 and access to the .snaps directories now fail with <div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">bash: cd: .snaps: Transport endpoint is not connected</blockquote></div><div><br></div><div>in the volume log file on the client I see:</div><div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">016-04-22 21:08:28.005854] I [rpc-clnt.c:1847:rpc_clnt_reconfig] 2-homes-snapd-client: changing port to 49493 (from 0)<br>[2016-04-22 21:08:28.009558] E [socket.c:2278:socket_connect_finish] 2-homes-snapd-client: connection to xx.xx.xx.xx.xx:49493 failed (No route to host)</blockquote></div><div><br></div><div>I'm quite perplexed, now it's not a network issue or DNS as far as I can tell, the glusterfs client is working fine, and the gluster servers all resolve ok. It seems to be happening on all the clients I have tried different systems with 3.7.8, 3.7.10, and 3.7.11 version clients and see the same failure on all of them.</div><div><br></div><div>On the servers the snapshots are being taken as expected and they are started:</div><div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Snapshot : Scheduled-Homes_Hourly-homes_GMT-2016.04.22-16.00.01<br>Snap UUID : 91ba50b0-d8f2-4135-9ea5-edfdfe2ce61d<br>Created : 2016-04-22 16:00:01<br>Snap Volumes:<br><span style="white-space:pre-wrap"> </span>Snap Volume Name : 5170144102814026a34f8f948738406f<br><span style="white-space:pre-wrap"> </span>Origin Volume name : homes<br><span style="white-space:pre-wrap"> </span>Snaps taken for homes : 16<br><span style="white-space:pre-wrap"> </span>Snaps available for homes : 240<br><span style="white-space:pre-wrap"> </span>Status : Started</blockquote></div><div><br></div><div><br></div><div>the homes volume is replica 3 all the peers are up and so are all the bricks and services:</div><div><font face="arial narrow, sans-serif"><br></font></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><font face="arial narrow, sans-serif">glv status homes<br>Status of volume: homes<br>Gluster process TCP Port RDMA Port Online Pid<br>------------------------------------------------------------------------------<br>Brick gluster-2:/export/brick2/home 49171 0 Y 38298<br>Brick gluster0:/export/brick2/home 49154 0 Y 23519<br>Brick gluster1.vsnet.gmu.edu:/export/brick2<br>/home 49154 0 Y 23794<br>Snapshot Daemon on localhost 49486 0 Y 23699<br>NFS Server on localhost 2049 0 Y 23486<br>Self-heal Daemon on localhost N/A N/A Y 23496<br>Snapshot Daemon on gluster-2 49261 0 Y 38479<br>NFS Server on gluster-2 2049 0 Y 39640<br>Self-heal Daemon on gluster-2 N/A N/A Y 39709<br>Snapshot Daemon on gluster1 49480 0 Y 23982<br>NFS Server on gluster1 2049 0 Y 23766<br>Self-heal Daemon on gluster1 N/A N/A Y 23776<br> <br>Task Status of Volume homes<br>------------------------------------------------------------------------------<br>There are no active volume tasks</font></blockquote></div><div><br></div><div>I'd appreciate any ideas about troubleshooting this. I tried disable .snaps access on the volume and re-enabling it but is made no difference.</div><div><br></div></div>
<br></div></div>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>