<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><meta http-equiv="Content-Type" content="text/html charset=windows-1252" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Pranith,<div class=""><br class=""></div><div class="">I’m sorry but I cannot bring you any comparison because comparison will be distorted by the fact in my HPC cluster in production the network technology is InfiniBand QDR and my volumes are quite different (brick in RAID6 (12x2TB), 2 bricks per server and 4 servers into my pool)</div><div class=""><br class=""></div><div class="">Concerning your demand, in attachments you can find all expected results hoping it can help you to solve this serious performance issue (maybe I need play with glusterfs parameters?).</div><div class=""><br class=""></div><div class="">Thank you very much by advance,</div><div class="">Geoffrey<br class=""><div apple-content-edited="true" class="">
<div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">------------------------------------------------------<br class="">Geoffrey Letessier<br class="">Responsable informatique & ingénieur système<br class="">UPR 9080 - CNRS - Laboratoire de Biochimie Théorique<br class="">Institut de Biologie Physico-Chimique<br class="">13, rue Pierre et Marie Curie - 75005 Paris<br class="">Tel: 01 58 41 50 93 - eMail: <a href="mailto:geoffrey.letessier@ibpc.fr" class="">geoffrey.letessier@ibpc.fr</a></div>
</div>
<br class=""><div class=""><blockquote type="cite" class=""><div class="">Le 2 juin 2015 à 10:09, Pranith Kumar Karampuri <<a href="mailto:pkarampu@redhat.com" class="">pkarampu@redhat.com</a>> a écrit :</div><br class="Apple-interchange-newline"><div class="">
<meta content="text/html; charset=windows-1252" http-equiv="Content-Type" class="">
<div bgcolor="#FFFFFF" text="#000000" class="">
hi Geoffrey,<br class="">
Since you are saying it happens on all types of
volumes, lets do the following:<br class="">
1) Create a dist-repl volume<br class="">
2) Set the options etc you need.<br class="">
3) enable gluster volume profile using "gluster volume profile
<volname> start"<br class="">
4) run the work load<br class="">
5) give output of "gluster volume profile <volname> info"<br class="">
<br class="">
Repeat the steps above on new and old version you are comparing this
with. That should give us insight into what could be causing the
slowness.<br class="">
<br class="">
Pranith<br class="">
<div class="moz-cite-prefix">On 06/02/2015 03:22 AM, Geoffrey
Letessier wrote:<br class="">
</div>
<blockquote cite="mid:DD72172A-2E2B-4141-B424-4474CE4D79F8@cnrs.fr" type="cite" class="">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252" class="">
Dear all,
<div class=""><br class="">
</div>
<div class="">I have a crash test cluster where i’ve tested the new version
of GlusterFS (v3.7) before upgrading my HPC cluster in
production. </div>
<div class="">But… all my tests show me very very low performances.</div>
<div class=""><br class="">
</div>
<div class="">For my benches, as you can read below, I do some actions
(untar, du, find, tar, rm) with linux kernel sources, dropping
cache, each on distributed, replicated, distributed-replicated,
single (single brick) volumes and the native FS of one brick.</div>
<div class=""><br class="">
</div>
<div class="">
<div class=""># time (echo 3 > /proc/sys/vm/drop_caches; tar xJf
~/linux-4.1-rc5.tar.xz; sync; echo 3 >
/proc/sys/vm/drop_caches)</div>
<div class=""># time (echo 3 > /proc/sys/vm/drop_caches; du -sh
linux-4.1-rc5/; echo 3 > /proc/sys/vm/drop_caches)</div>
<div class=""># time (echo 3 > /proc/sys/vm/drop_caches; find
linux-4.1-rc5/|wc -l; echo 3 > /proc/sys/vm/drop_caches)</div>
<div class=""># time (echo 3 > /proc/sys/vm/drop_caches; tar czf
linux-4.1-rc5.tgz linux-4.1-rc5/; echo 3 >
/proc/sys/vm/drop_caches)</div>
<div class=""># time (echo 3 > /proc/sys/vm/drop_caches; rm -rf
linux-4.1-rc5.tgz linux-4.1-rc5/; echo 3 >
/proc/sys/vm/drop_caches)</div>
<div class=""><br class="">
</div>
<div class="">And here are the process times:</div>
<div class=""><br class="">
</div>
<div class="">
<div class="">
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">| | UNTAR | DU | FIND | TAR
| RM |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">| single | ~3m45s | ~43s | ~47s |
~3m10s | ~3m15s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">| replicated | ~5m10s | ~59s | ~1m6s |
~1m19s | ~1m49s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">| distributed | ~4m18s | ~41s | ~57s |
~2m24s | ~1m38s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">| dist-repl | ~8m18s | ~1m4s | ~1m11s |
~1m24s | ~2m40s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">| native FS | ~11s | ~4s | ~2s |
~56s | ~10s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);" class="">---------------------------------------------------------------</div>
</div>
</div>
<div class=""><br class="">
</div>
<div class="">I get the same results, whether with default configurations
with custom configurations.</div>
<div class=""><br class="">
</div>
<div class="">if I look at the side of the ifstat command, I can note my
IO write processes never exceed 3MBs...</div>
<div class=""><br class="">
</div>
<div class="">EXT4 native FS seems to be faster (roughly 15-20% but no
more) than XFS one</div>
<div class=""><br class="">
</div>
<div class="">My [test] storage cluster config is composed by 2 identical
servers (biCPU Intel Xeon X5355, 8GB of RAM, 2x2TB HDD
(no-RAID) and Gb ethernet)</div>
<div class=""><br class="">
</div>
<div class="">My volume settings:</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>single:
1server 1 brick</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>replicated:
2 servers 1 brick each</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>distributed:
2 servers 2 bricks each</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>dist-repl:
2 bricks in the same server and replica 2</div>
<div class=""><br class="">
</div>
<div class="">All seems to be OK in gluster status command line.</div>
<div class=""><br class="">
</div>
<div class="">Do you have an idea why I obtain so bad results?</div>
<div class="">Thanks in advance.</div>
<div class="">Geoffrey</div>
<div class="">-----------------------------------------------</div>
<div apple-content-edited="true" class="">
<div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Geoffrey Letessier<br class="">
<br class="">
Responsable informatique & ingénieur système<br class="">
CNRS - UPR 9080 - Laboratoire de Biochimie Théorique<br class="">
Institut de Biologie Physico-Chimique<br class="">
13, rue Pierre et Marie Curie - 75005 Paris<br class="">
Tel: 01 58 41 50 93 - eMail: <a moz-do-not-send="true" href="mailto:geoffrey.letessier@cnrs.fr" class="">geoffrey.letessier@cnrs.fr</a></div>
</div>
<br class="">
</div>
<br class="">
<fieldset class="mimeAttachmentHeader"></fieldset>
<br class="">
<pre wrap="" class="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br class="">
</div>
</div></blockquote></div></div></div></body></html>