<div dir="ltr">Hi Pranith,<div><br></div><div><br></div><div><div>Gluster Version :</div><div>glusterfs 3.6.2 built on Jan 22 2015 12:58:10</div></div><div><ul><li>100,000 file test in XFS directly<br></li></ul></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>[root@glusterfs-node1 gfs_data]# java FileCreationAndSearchtest 10w_test 100000</div><div><div>Creation finished!! Time: <b>8961(ms)</b></div></div><div><div>File Search Finished!! 33223 searches, time: 124ms, misses:0</div></div><div><div>NIO Search Finished!! 33223 searches, time: 367ms, misses:0</div></div></blockquote><div><br></div><div><ul><li>100,000 file test from client (Distributed Replicated)<br></li></ul></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>[root@glusterfs-client01 drvol]# java FileCreationAndSearchtest 10w_test01 100000</div></div><div><div>Start!!</div></div><div><div>Creation finished!! Time: <b>265713(ms)</b></div></div><div><div>File Search Finished!! 33453 searches, time: 18403(ms), misses:0</div></div><div><div>NIO Search Finished!! 33453 searches, time: 17827(ms), misses:0</div></div></blockquote><div><br></div><div><br></div><div>after many small file created (<span style="color:rgb(84,84,84);line-height:18.2000007629395px">≒</span>200 million), it cost more than 1000000(ms) to finish 100,000 file test</div><div><br></div><div>I've flushed all volumes and create new one to reproduce the condition again.</div><div><br></div><div><br></div><div><div class="gmail_extra"><br><div class="gmail_quote">2015-02-05 19:19 GMT+08:00 Pranith Kumar Karampuri <span dir="ltr"><<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span>
<br>
<div>On 02/02/2015 02:42 PM, Ason Hu wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>In my small file test, 3 types of volume are created in
same GlusterFS Cluster (4 nodes)</div>
<div>
<ul>
<li>Distritubed<br>
</li>
<li>Distributed Replicated<br>
</li>
<li>Distributed Striped</li>
</ul>
<br>
10 million small files (8 byte) create and search each test
cycle life <br>
<table style="border-spacing:0px;color:rgb(119,119,119);font-size:13px">
<tbody>
</tbody>
</table>
<div><br>
</div>
<div><br>
</div>
<div>
<div>Nodes info : </div>
<div>
<ul>
<li>OS : CentOS 6.6 64bit</li>
<li>RAM : 64GB<br>
</li>
<li>HDD : 4TB, 7200RPM, SAS *6 RAID 0<br>
</li>
<li>FileSystem : XFS, BS=4K</li>
</ul>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Problem :</div>
<div><br>
</div>
<div>It occur when file reach <span style="color:rgb(84,84,84);line-height:18.2000007629395px">≒</span>200
million (sum of 3 volumes)</div>
</div>
<div>files created speed drop from <span style="color:rgb(84,84,84);line-height:18.2000007629395px"> </span><span style="color:rgb(84,84,84);line-height:18.2000007629395px">≒</span>1500
to <span style="color:rgb(84,84,84);line-height:18.2000007629395px"> </span><span style="color:rgb(84,84,84);line-height:18.2000007629395px">≒</span>100</div>
</div>
</blockquote></span>
If you do this test directly on the XFS partition, what is the kind
of performance difference you see?<br>
<br>
Pranith<span><br>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>Though the speed of creation become slow but search speed
and throughput still good </div>
<div>Seems each types of volume have same problem, I don't know
what cause this problem..<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Workaround :</div>
<div><br>
</div>
<div>I try to create new volume ( Distributed |
Distributed Replicated | Distributed Striped ), mount it, </div>
<div>problem solved!</div>
<div>But the old volume write speed still sad..</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>I'd try to increase below volume setting, but don't work
at all</div>
<div>
<ul>
<li>performance.cache-size <br>
</li>
<li>performance.io-thread-count</li>
</ul>
</div>
<div>
<div><br>
</div>
<div>I didn't enable the volume qouta option.</div>
<div>Does glusterfs volume have some kind of limitation of
file creation? <br>
</div>
</div>
</div>
</blockquote>
</span><blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div><br>
</div>
<div>Regards</div>
<div>Nosa Hu,</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</div>
</blockquote></div><br></div></div></div>