<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 29, 2016 at 2:26 PM, Yannick Perret <span dir="ltr"><<a href="mailto:yannick.perret@liris.cnrs.fr" target="_blank">yannick.perret@liris.cnrs.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Ok, last try:<br>
after investigating more versions I found that FUSE client leaks memory on all of them.<br>
I tested:<br>
- 3.6.7 client on debian 7 32bit and on debian 8 64bit (with 3.6.7 serveurs on debian 8 64bit)<br>
- 3.6.9 client on debian 7 32bit and on debian 8 64bit (with 3.6.7 serveurs on debian 8 64bit)=<br>
- 3.7.13 client on debian 8 64bit (with 3.8.1 serveurs on debian 8 64bit)<br>
- 3.8.1 client on debian 8 64bit (with 3.8.1 serveurs on debian 8 64bit)<br>
In all cases compiled from sources, appart for 3.8.1 where .deb were used (due to a configure runtime error).<br>
For 3.7 it was compiled with --disable-tiering. I also tried to compile with --disable-fusermount (no change).<br>
<br>
In all of these cases the memory (resident & virtual) of glusterfs process on client grows on each activity and never reach a max (and never reduce).<br>
"Activity" for these tests is cp -Rp and ls -lR.<br>
The client I let grows the most overreached ~4Go RAM. On smaller machines it ends by OOM killer killing glusterfs process or glusterfs dying due to allocation error.<br>
<br>
In 3.6 mem seems to grow continusly, whereas in 3.8.1 it grows by "steps" (430400 ko → 629144 (~1min) → 762324 (~1min) → 827860…).<br>
<br>
All tests performed on a single test volume used only by my test client. Volume in a basic x2 replica. The only parameters I changed on this volume (without any effect) are diagnostics.client-log-level set to ERROR and network.inode-lru-limit set to 1024.<br></blockquote><div><br></div><div>Could you attach statedumps of your runs?<br>The following link has steps to capture this(<a href="https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/">https://gluster.readthedocs.io/en/latest/Troubleshooting/statedump/</a> ). We basically need to see what are the memory types that are increasing. If you could help find the issue, we can send the fixes for your workload. There is a 3.8.2 release in around 10 days I think. We can probably target this issue for that?<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
<br>
This clearly prevent us to use glusterfs on our clients. Any way to prevent this to happen? I still switched back to NFS mounts but it is not what we're looking for.<br>
<br>
Regards,<br>
--<br>
Y.<br>
<br>
<br>
<br>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Pranith<br></div></div>
</div></div>