<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
hi Geoffrey,<br>
Since you are saying it happens on all types of
volumes, lets do the following:<br>
1) Create a dist-repl volume<br>
2) Set the options etc you need.<br>
3) enable gluster volume profile using "gluster volume profile
<volname> start"<br>
4) run the work load<br>
5) give output of "gluster volume profile <volname> info"<br>
<br>
Repeat the steps above on new and old version you are comparing this
with. That should give us insight into what could be causing the
slowness.<br>
<br>
Pranith<br>
<div class="moz-cite-prefix">On 06/02/2015 03:22 AM, Geoffrey
Letessier wrote:<br>
</div>
<blockquote cite="mid:DD72172A-2E2B-4141-B424-4474CE4D79F8@cnrs.fr"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
Dear all,
<div><br>
</div>
<div>I have a crash test cluster where i’ve tested the new version
of GlusterFS (v3.7) before upgrading my HPC cluster in
production. </div>
<div>But… all my tests show me very very low performances.</div>
<div><br>
</div>
<div>For my benches, as you can read below, I do some actions
(untar, du, find, tar, rm) with linux kernel sources, dropping
cache, each on distributed, replicated, distributed-replicated,
single (single brick) volumes and the native FS of one brick.</div>
<div><br>
</div>
<div>
<div># time (echo 3 > /proc/sys/vm/drop_caches; tar xJf
~/linux-4.1-rc5.tar.xz; sync; echo 3 >
/proc/sys/vm/drop_caches)</div>
<div># time (echo 3 > /proc/sys/vm/drop_caches; du -sh
linux-4.1-rc5/; echo 3 > /proc/sys/vm/drop_caches)</div>
<div># time (echo 3 > /proc/sys/vm/drop_caches; find
linux-4.1-rc5/|wc -l; echo 3 > /proc/sys/vm/drop_caches)</div>
<div># time (echo 3 > /proc/sys/vm/drop_caches; tar czf
linux-4.1-rc5.tgz linux-4.1-rc5/; echo 3 >
/proc/sys/vm/drop_caches)</div>
<div># time (echo 3 > /proc/sys/vm/drop_caches; rm -rf
linux-4.1-rc5.tgz linux-4.1-rc5/; echo 3 >
/proc/sys/vm/drop_caches)</div>
<div><br>
</div>
<div>And here are the process times:</div>
<div><br>
</div>
<div>
<div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">| | UNTAR | DU | FIND | TAR
| RM |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">| single | ~3m45s | ~43s | ~47s |
~3m10s | ~3m15s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">| replicated | ~5m10s | ~59s | ~1m6s |
~1m19s | ~1m49s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">| distributed | ~4m18s | ~41s | ~57s |
~2m24s | ~1m38s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">| dist-repl | ~8m18s | ~1m4s | ~1m11s |
~1m24s | ~2m40s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">---------------------------------------------------------------</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">| native FS | ~11s | ~4s | ~2s |
~56s | ~10s |</div>
<div style="margin: 0px; font-size: 10px; font-family:
Menlo; color: rgb(255, 255, 255); background-color: rgb(0,
0, 0);">---------------------------------------------------------------</div>
</div>
</div>
<div><br>
</div>
<div>I get the same results, whether with default configurations
with custom configurations.</div>
<div><br>
</div>
<div>if I look at the side of the ifstat command, I can note my
IO write processes never exceed 3MBs...</div>
<div><br>
</div>
<div>EXT4 native FS seems to be faster (roughly 15-20% but no
more) than XFS one</div>
<div><br>
</div>
<div>My [test] storage cluster config is composed by 2 identical
servers (biCPU Intel Xeon X5355, 8GB of RAM, 2x2TB HDD
(no-RAID) and Gb ethernet)</div>
<div><br>
</div>
<div>My volume settings:</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>single:
1server 1 brick</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>replicated:
2 servers 1 brick each</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>distributed:
2 servers 2 bricks each</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>dist-repl:
2 bricks in the same server and replica 2</div>
<div><br>
</div>
<div>All seems to be OK in gluster status command line.</div>
<div><br>
</div>
<div>Do you have an idea why I obtain so bad results?</div>
<div>Thanks in advance.</div>
<div>Geoffrey</div>
<div>-----------------------------------------------</div>
<div apple-content-edited="true">
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;
word-wrap: break-word; -webkit-nbsp-mode: space;
-webkit-line-break: after-white-space;">Geoffrey Letessier<br>
<br>
Responsable informatique & ingénieur système<br>
CNRS - UPR 9080 - Laboratoire de Biochimie Théorique<br>
Institut de Biologie Physico-Chimique<br>
13, rue Pierre et Marie Curie - 75005 Paris<br>
Tel: 01 58 41 50 93 - eMail: <a moz-do-not-send="true"
href="mailto:geoffrey.letessier@cnrs.fr">geoffrey.letessier@cnrs.fr</a></div>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>