<html><head></head><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px"><div id="yui_3_16_0_ym19_1_1470326912688_10093" dir="ltr"><span id="yui_3_16_0_ym19_1_1470326912688_10118">i got 1.2 gb/s on seagate sshd </span><span style="color: rgb(51, 51, 51); font-family: Helvetica, Arial, sans-serif; font-size: 15px; font-weight: bold; line-height: 22px;" id="yui_3_16_0_ym19_1_1470326912688_10096">ST1000LX001 raid 5 x3 (but with the dreaded cache array on) and 1.1 gb/s on samsung pro ssd 1tb x3 raid5 (no array caching on for it's not compatible on proliant---not enterprise ssd).</span></div> <div class="qtdSeparateBR"><br><br></div><div class="yahoo_quoted" style="display: block;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 10px;"> <div style="font-family: HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif; font-size: 16px;"> <div dir="ltr"><font size="2" face="Arial"> On Thursday, August 4, 2016 5:23 AM, Kaamesh Kamalaaharan <kaamesh@novocraft.com> wrote:<br></font></div> <br><br> <div class="y_msg_container"><div id="yiv3955646572"><div><div dir="ltr">hi, <div>thanks for the reply. I have hardware raid 5 storage servers with 4TB WD red drives. I think they are capable of 6GB/s transfers so it shouldnt be a drive speed issue. Just for testing i tried to do a dd test directy into the brick mounted from the storage server itself and got around 800mb/s transfer rate which is double what i get when the brick is mounted on the client. Are there any other options or tests that i can perform to figure out the root cause of my problem as i have exhaused most google searches and tests. </div><div><br clear="none"></div><div>Kaamesh</div></div><div class="yiv3955646572yqt9536278327" id="yiv3955646572yqt49617"><div class="yiv3955646572gmail_extra"><br clear="none"><div class="yiv3955646572gmail_quote">On Wed, Aug 3, 2016 at 10:58 PM, Leno Vo <span dir="ltr"><<a rel="nofollow" shape="rect" ymailto="mailto:lenovolastname@yahoo.com" target="_blank" href="mailto:lenovolastname@yahoo.com">lenovolastname@yahoo.com</a>></span> wrote:<br clear="none"><blockquote class="yiv3955646572gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div style="color:#000;background-color:#fff;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"><div><span>your 10G nic is capable, the problem is the disk speed, fix ur disk speed first, use ssd or sshd or sas 15k in a raid 0 or raid 5/6 x4 at least.</span></div> <div><br clear="none"><br clear="none"></div><div style="display:block;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10px;"> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;"><div><div class="yiv3955646572h5"> <div dir="ltr"><font size="2" face="Arial"> On Wednesday, August 3, 2016 2:40 AM, Kaamesh Kamalaaharan <<a rel="nofollow" shape="rect" ymailto="mailto:kaamesh@novocraft.com" target="_blank" href="mailto:kaamesh@novocraft.com">kaamesh@novocraft.com</a>> wrote:<br clear="none"></font></div> <br clear="none"><br clear="none"> </div></div><div><div><div class="yiv3955646572h5"><div><div dir="ltr">Hi , <div>I have gluster 3.6.2 installed on my server network. Due to internal issues we are not allowed to upgrade the gluster version. All the clients are on the same version of gluster. When transferring files to/from the clients or between my nodes over the 10gb network, the transfer rate is capped at 450Mb/s .Is there any way to increase the transfer speeds for gluster mounts? </div><div><br clear="none"></div><div>Our server setup is as following:</div><div><br clear="none"></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px;"><div>2 gluster servers -gfs1 and gfs2</div><div> volume name : gfsvolume</div><div>3 clients - hpc1, hpc2,hpc3</div><div>gluster volume mounted on /export/gfsmount/</div></blockquote><div><br clear="none"></div><div><br clear="none"></div><div><br clear="none"></div><div>The following is the average results what i did so far:</div><div><br clear="none"></div><div>1) test bandwith with iperf between all machines - 9.4 GiB/s</div><div>2) test write speed with dd <br clear="none"></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px;"><div>dd if=/dev/zero of=/export/gfsmount/testfile bs=1G count=1</div><div><br clear="none"></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px;"><div>result=399Mb/s</div></blockquote><div><br clear="none"></div>3) test read speed with dd</div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px;"><div>dd if=/export/gfsmount/testfile of=/dev/zero bs=1G count=1</div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px;"><div><br clear="none"></div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px;">result=284MB/s<br clear="none"></blockquote><br clear="none"></div><div>My gluster volume configuration:</div><div> </div><blockquote style="margin:0 0 0 40px;border:none;padding:0px;"><div><pre style="color:rgb(0,0,0);">Volume Name: gfsvolume</pre></div><div><pre style="color:rgb(0,0,0);">Type: Replicate</pre></div><div><pre style="color:rgb(0,0,0);">Volume ID: a29bd2fb-b1ef-4481-be10-c2f4faf4059b</pre></div><div><pre style="color:rgb(0,0,0);">Status: Started</pre></div><div><pre style="color:rgb(0,0,0);">Number of Bricks: 1 x 2 = 2</pre></div><div><pre style="color:rgb(0,0,0);">Transport-type: tcp</pre></div><div><pre style="color:rgb(0,0,0);">Bricks:</pre></div><div><pre style="color:rgb(0,0,0);">Brick1: gfs1:/export/sda/brick</pre></div><div><pre style="color:rgb(0,0,0);">Brick2: gfs2:/export/sda/brick</pre></div><div><pre style="color:rgb(0,0,0);">Options Reconfigured:</pre></div><div><pre style="color:rgb(0,0,0);">performance.quick-read: off</pre></div><div><pre style="color:rgb(0,0,0);">network.ping-timeout: 30</pre></div><div><pre style="color:rgb(0,0,0);">network.frame-timeout: 90</pre></div><div><pre style="color:rgb(0,0,0);">performance.cache-max-file-size: 2MB</pre></div><div><pre style="color:rgb(0,0,0);">cluster.server-quorum-type: none</pre></div><div><pre style="color:rgb(0,0,0);">nfs.addr-namelookup: off</pre></div><div><pre style="color:rgb(0,0,0);">nfs.trusted-write: off</pre></div><div><pre style="color:rgb(0,0,0);">performance.write-behind-window-size: 4MB</pre></div><div><pre style="color:rgb(0,0,0);">cluster.data-self-heal-algorithm: diff</pre></div><div><pre style="color:rgb(0,0,0);">performance.cache-refresh-timeout: 60</pre></div><div><pre style="color:rgb(0,0,0);">performance.cache-size: 1GB</pre></div><div><pre style="color:rgb(0,0,0);">cluster.quorum-type: fixed</pre></div><div><pre style="color:rgb(0,0,0);">auth.allow: 172.*</pre></div><div><pre style="color:rgb(0,0,0);">cluster.quorum-count: 1</pre></div><div><pre style="color:rgb(0,0,0);">diagnostics.latency-measurement: on</pre></div><div><pre style="color:rgb(0,0,0);">diagnostics.count-fop-hits: on</pre></div><div><pre style="color:rgb(0,0,0);">cluster.server-quorum-ratio: 50%</pre></div></blockquote><div><pre style="color:rgb(0,0,0);"><br clear="none"></pre><pre style="color:rgb(0,0,0);">Any help would be appreciated. </pre><pre style="color:rgb(0,0,0);">Thanks,</pre><pre style="color:rgb(0,0,0);">Kaamesh</pre><pre style="color:rgb(0,0,0);"><br clear="none"></pre></div></div></div><br clear="none"></div></div>_______________________________________________<br clear="none">Gluster-users mailing list<br clear="none"><a rel="nofollow" shape="rect" ymailto="mailto:Gluster-users@gluster.org" target="_blank" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br clear="none"><a rel="nofollow" shape="rect" target="_blank" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a><br clear="none"><br clear="none"></div> </div> </div> </div></div></div></blockquote></div><br clear="none"></div></div></div></div><br><br></div> </div> </div> </div></div></body></html>