<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<br>
<div class="moz-cite-prefix">On 10/14/2015 10:43 PM, Mohamed Pakkeer
wrote:<br>
</div>
<blockquote
cite="mid:CAN7LZ4EwPaYc6dR+xzXgu7HAUBjCSa+4Q84Qiy-1B-Bro+dJ+Q@mail.gmail.com"
type="cite">
<div dir="ltr">Hi Pranith,
<div><br>
</div>
<div>Will this patch improve the heal performance on distributed
disperse volume?. Currently we are getting 10MB/s heal
performance on 10G backed network. SHD daemon takes 5 days to
complete the heal operation for single 4TB( 3.5 TB data) disk
failure.</div>
</div>
</blockquote>
I will try to see if the patch for replication can be generalized
for disperse volume as well. Keep watching the mailing list for
updates :-)<br>
<br>
Pranith<br>
<blockquote
cite="mid:CAN7LZ4EwPaYc6dR+xzXgu7HAUBjCSa+4Q84Qiy-1B-Bro+dJ+Q@mail.gmail.com"
type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Regards,</div>
<div>Backer</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Oct 14, 2015 at 9:08 PM, Ben
Turner <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:bturner@redhat.com" target="_blank">bturner@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb">
<div class="h5">----- Original Message -----<br>
> From: "Pranith Kumar Karampuri" <<a
moz-do-not-send="true"
href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>><br>
> To: "Ben Turner" <<a moz-do-not-send="true"
href="mailto:bturner@redhat.com">bturner@redhat.com</a>>,
"Humble Devassy Chirammal" <<a
moz-do-not-send="true"
href="mailto:humble.devassy@gmail.com">humble.devassy@gmail.com</a>>,
"Atin Mukherjee"<br>
> <<a moz-do-not-send="true"
href="mailto:atin.mukherjee83@gmail.com">atin.mukherjee83@gmail.com</a>><br>
> Cc: "gluster-users" <<a moz-do-not-send="true"
href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
> Sent: Wednesday, October 14, 2015 1:39:14 AM<br>
> Subject: Re: [Gluster-users] Speed up heal
performance<br>
><br>
><br>
><br>
> On 10/13/2015 07:11 PM, Ben Turner wrote:<br>
> > ----- Original Message -----<br>
> >> From: "Humble Devassy Chirammal" <<a
moz-do-not-send="true"
href="mailto:humble.devassy@gmail.com">humble.devassy@gmail.com</a>><br>
> >> To: "Atin Mukherjee" <<a
moz-do-not-send="true"
href="mailto:atin.mukherjee83@gmail.com">atin.mukherjee83@gmail.com</a>><br>
> >> Cc: "Ben Turner" <<a
moz-do-not-send="true"
href="mailto:bturner@redhat.com">bturner@redhat.com</a>>,
"gluster-users"<br>
> >> <<a moz-do-not-send="true"
href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
> >> Sent: Tuesday, October 13, 2015 6:14:46
AM<br>
> >> Subject: Re: [Gluster-users] Speed up
heal performance<br>
> >><br>
> >>> Good news is we already have a WIP
patch <a moz-do-not-send="true"
href="http://review.glusterd.org/10851"
rel="noreferrer" target="_blank">review.glusterd.org/10851</a>
to<br>
> >> introduce multi threaded shd. Credits to
Richard/Shreyas from facebook for<br>
> >> this. IIRC, we also have a BZ for the
same<br>
> >> Isnt it the same bugzilla (<br>
> >> <a moz-do-not-send="true"
href="https://bugzilla.redhat.com/show_bug.cgi?id=1221737"
rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1221737</a>)
mentioned in the<br>
> >> commit log?<br>
> > @Lindsay - No need for a BZ, the above BZ
should suffice.<br>
> ><br>
> > @Anyone - In the commit I see:<br>
> ><br>
> > { .key =
"cluster.shd-max-threads",<br>
> > .voltype =
"cluster/replicate",<br>
> > .option = "shd-max-threads",<br>
> > .op_version = 1,<br>
> > .flags = OPT_FLAG_CLIENT_OPT<br>
> > },<br>
> > { .key =
"cluster.shd-thread-batch-size",<br>
> > .voltype =
"cluster/replicate",<br>
> > .option =
"shd-thread-batch-size",<br>
> > .op_version = 1,<br>
> > .flags = OPT_FLAG_CLIENT_OPT<br>
> > },<br>
> ><br>
> > So we can tune max threads and thread batch
size? I understand max<br>
> > threads, but what is batch size? In my
testing on 10G NICs with a backend<br>
> > that will service 10G throughput I see about
1.5 GB per minute of SH<br>
> > throughput. To Lindsay's other point, will
this patch improve SH<br>
> > throughput? My systems can write at 1.5 GB
/ Sec and NICs can to 1.2 GB /<br>
> > sec but I only see ~1.5 GB per _minute_ of
SH throughput. If we can not<br>
> > only make SH multi threaded, but improve the
performance of a single<br>
> > thread that would be awesome. Super bonus
points if we can have some sort<br>
> > of tunible that can limit the bandwidth each
thread can consume. It would<br>
> > be great to be able to crank things up when
the systems aren't busy and<br>
> > slow things down when load increases.<br>
> This patch is not merged because I thought we
needed throttling feature<br>
> to go in before we can merge this for better
control of the self-heal<br>
> speed. We are doing that for 3.8. So expect to
see both of these for 3.8.<br>
<br>
</div>
</div>
Great news! You da man Pranith, next time I am on your
side of the world beers are on me :)<br>
<span class="HOEnZb"><font color="#888888"><br>
-b<br>
</font></span>
<div class="HOEnZb">
<div class="h5"><br>
><br>
> Pranith<br>
> ><br>
> > -b<br>
> ><br>
> ><br>
> >> --Humble<br>
> >><br>
> >><br>
> >> On Tue, Oct 13, 2015 at 7:26 AM, Atin
Mukherjee<br>
> >> <<a moz-do-not-send="true"
href="mailto:atin.mukherjee83@gmail.com">atin.mukherjee83@gmail.com</a>><br>
> >> wrote:<br>
> >><br>
> >>> -Atin<br>
> >>> Sent from one plus one<br>
> >>> On Oct 13, 2015 3:16 AM, "Ben
Turner" <<a moz-do-not-send="true"
href="mailto:bturner@redhat.com">bturner@redhat.com</a>>
wrote:<br>
> >>>> ----- Original Message -----<br>
> >>>>> From: "Lindsay Mathieson"
<<a moz-do-not-send="true"
href="mailto:lindsay.mathieson@gmail.com">lindsay.mathieson@gmail.com</a>><br>
> >>>>> To: "gluster-users" <<a
moz-do-not-send="true"
href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>><br>
> >>>>> Sent: Friday, October 9,
2015 9:18:11 AM<br>
> >>>>> Subject: [Gluster-users]
Speed up heal performance<br>
> >>>>><br>
> >>>>> Is there any way to max out
heal performance? My cluster is unused<br>
> >>> overnight,<br>
> >>>>> and lightly used at
lunchtimes, it would be handy to speed up a heal.<br>
> >>>>><br>
> >>>>> The only tuneable I found
was cluster.self-heal-window-size, which<br>
> >>> doesn't<br>
> >>>>> seem to make much
difference.<br>
> >>>> I don't know of any way to speed
this up, maybe someone else could chime<br>
> >>> in here that knows the heal daemon
better than me. Maybe you could open<br>
> >>> an<br>
> >>> RFE on this? In my testing I only
see 2 files getting healed at a time<br>
> >>> per<br>
> >>> replica pair. I would like to see
this be multi threaded(if its not<br>
> >>> already) with the ability to tune it
to control resource usage(similar to<br>
> >>> what we did in the rebalance
refactoring done recently). If you let me<br>
> >>> know the BZ # I'll add my data +
suggestions, I have been testing this<br>
> >>> pretty extensively in recent weeks
and good data + some ideas on how to<br>
> >>> speed things up.<br>
> >>> Good news is we already have a WIP
patch <a moz-do-not-send="true"
href="http://review.glusterd.org/10851"
rel="noreferrer" target="_blank">review.glusterd.org/10851</a>
to<br>
> >>> introduce multi threaded shd.
Credits to Richard/Shreyas from facebook<br>
> >>> for<br>
> >>> this. IIRC, we also have a BZ for
the same but the patch is in rfc as of<br>
> >>> now. AFAIK, this is a candidate to
land in 3.8 as well, Vijay can correct<br>
> >>> me otherwise.<br>
> >>>> -b<br>
> >>>><br>
> >>>>> thanks,<br>
> >>>>> --<br>
> >>>>> Lindsay<br>
> >>>>><br>
> >>>>>
_______________________________________________<br>
> >>>>> Gluster-users mailing list<br>
> >>>>> <a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> >>>>> <a moz-do-not-send="true"
href="http://www.gluster.org/mailman/listinfo/gluster-users"
rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
> >>>>
_______________________________________________<br>
> >>>> Gluster-users mailing list<br>
> >>>> <a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> >>>> <a moz-do-not-send="true"
href="http://www.gluster.org/mailman/listinfo/gluster-users"
rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
> >>>
_______________________________________________<br>
> >>> Gluster-users mailing list<br>
> >>> <a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> >>> <a moz-do-not-send="true"
href="http://www.gluster.org/mailman/listinfo/gluster-users"
rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
> >>><br>
> >
_______________________________________________<br>
> > Gluster-users mailing list<br>
> > <a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> > <a moz-do-not-send="true"
href="http://www.gluster.org/mailman/listinfo/gluster-users"
rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
><br>
><br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a moz-do-not-send="true"
href="http://www.gluster.org/mailman/listinfo/gluster-users"
rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
</div>
</div>
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
<div class="gmail_signature"><br>
<br>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>