<div dir="ltr"><div class="gmail_extra"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>On Sat, Aug 27, 2016 at 9:55 PM, David Gossage <span dir="ltr"><<a href="mailto:dgossage@carouselchecks.com" target="_blank">dgossage@carouselchecks.com</a>></span> wrote:<br></div></div></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div><div data-smartmail="gmail_signature"><div dir="ltr"><div>On Sat, Aug 27, 2016 at 5:35 PM, David Gossage <span dir="ltr"><<a href="mailto:dgossage@carouselchecks.com" target="_blank">dgossage@carouselchecks.com</a>></span> wrote:<br></div></div></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p dir="ltr"></p>
<p dir="ltr">On Aug 27, 2016 4:37 PM, "Lindsay Mathieson" <<a href="mailto:lindsay.mathieson@gmail.com" target="_blank">lindsay.mathieson@gmail.com</a>> wrote:<br>
><br>
> On 28/08/2016 6:07 AM, David Gossage wrote:<br>
>><br>
>> 7 hours after starting full heal shards still haven't started healing, and count from heal statistics heal-count has only reached 1800 out of 19000 shards. shards dir hasn't even been recreated yet. Creation of the non sharded stubs (do they have a more official term?) in the visible mount point was as speedy as expected. shards are painfully slow.<br>
><br>
><br>
><br>
> Is your CPU usage through the roof?</p>
<p dir="ltr">Currently it has almost no activity. First node yesterday got a bit high. But 2nd node today that has issues is pretty low.<br>
><br>
> If you haven't already, I'd suggest<br>
><br>
> - changing "cluster.data-self-heal-algori<wbr>thm" to "full"<br>
><br>
> - And restarting the gluster volume if possible<br>
></p>
<p dir="ltr">I'll shut down vm's later tonight and see if that helps at all.</p></blockquote><div><br></div><div>applied "cluster.data-self-heal-algor<wbr>ithm" to "full"</div><div><br></div><div>stopped volume - started volume</div><div><br></div><div>cpu activity barely noticeable. heal count crawling at 1 new addition to list every minute or 2 and stil just building a list it hasnt started making a .shard directory</div></div></div></div></blockquote><div><br></div><div>logging into each linux vm and running from / 'find . | xargs stat ' seeme dto make count ump a bit faster. wasnt quite sure best way to repeatthat for windows vm's so just ran full system virus scans.</div><div><br></div><div>Still after 15+ plus hours now its listed 3600/19000 shards in to be healed list and started healing none.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<p dir="ltr">><br>
> I have a suspicion something changed recently with heal, I've noticed that it takes a long time (hours) to kick in when the diff algorithm is used. I don't recall it doing this with 3.7.11<br>
><span><font color="#888888"><br>
><br>
> -- <br>
> Lindsay Mathieson<br>
></font></span></p>
</blockquote></div><br></div></div>
</blockquote></div><br></div></div>