<div dir="ltr"><div><div><div><div><div><div><div><div>Hmm I don't have an answer to that question right now because I'm no on-disk fs expert.<br></div>As far as gluster is concerned, I don't actually see any issue except for the following:<br></div>as of today, gluster's entry self-heal algorithm crawls the whole parent directory and compares the<br></div>source and sink to figure out what files need to be created/deleted on the sink brick.<br><br></div>If the shards were out into multiple directories as opposed to just /.shard, entry self-heal would take<br></div>lesser time and resources to complete in certain cases since the crawling could possibly be confined to only those few directories and they would all within them contain fewer shards.<br></div>But with granular entry self-heal feature which is going to be introduced in 3.8, this will also be a non-issue since<br></div>there AFR will record the exact shards that were created/deleted while a brick is down and only sync them to the sink brick once it is back up.<br><br><a href="https://github.com/gluster/glusterfs-specs/blob/master/accepted/granular-entry-self-healing.md">https://github.com/gluster/glusterfs-specs/blob/master/accepted/granular-entry-self-healing.md</a><br><br></div><div>-Krutika<br></div><div><br></div>-Krutika<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Apr 16, 2016 at 5:38 AM, Lindsay Mathieson <span dir="ltr"><<a href="mailto:lindsay.mathieson@gmail.com" target="_blank">lindsay.mathieson@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Looking at migrating all our VM's to sharded gluster storage, 2.5TB of data in total. Underlying storage is zfs (on linux).<br>
<br>
Small scale testing has been going well but one thing I am unsure of is whether a *large* number of small files in one directory is going to be a problem. If I migrate everything to a 4MB shard size there is going to be 655360 files. Will that be a problem for gluster? or for zfs?<span class="HOEnZb"><font color="#888888"><br>
<br>
-- <br>
Lindsay Mathieson<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
</font></span></blockquote></div><br></div>