<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Any help/clues about that?<br>
<br>
BTW I saw that used memory increase when performing ls -lR but not
when performing ls -R.<br>
<br>
This problem prevent us to use glusterfs client.<br>
<br>
Regards,<br>
--<br>
Y.<br>
<br>
Le 25/07/2016 16:08, Yannick Perret a écrit :<br>
</div>
<blockquote cite="mid:57961D5C.9040903@liris.cnrs.fr" type="cite">
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
<div class="moz-cite-prefix">Hello,<br>
checking for related problems in bugreports I found this
discussion:<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="https://www.gluster.org/pipermail/gluster-users/2015-September/023709.html">https://www.gluster.org/pipermail/gluster-users/2015-September/023709.html</a><br>
<br>
The memleaks from <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="https://bugzilla.redhat.com/show_bug.cgi?id=1126831">https://bugzilla.redhat.com/show_bug.cgi?id=1126831</a>
are supposed to be corrected in 3.7.9. Does 3.6.9 (that I'm
using) also has these corrections?<br>
<br>
But in the 1st discussion they talk about
network.inode-lru-limit that may influence the used memory.<br>
Is that true? I set the value to 512 but I'm still able to go
far away over ~512Mo (I reached ~1.3Go of VSZ and ~1Go of RSS).<br>
I also tested with 1024 and I'm at 1171140/987024 and it is
still growing slowly (performing cp -Rp + ls -lR on the mounted
volume).<br>
BTW if this value can influence memory usage on clients is it
possible to set it on a per-client base? Because it may depend
on available memory on each client, not on the volume itself.<br>
<br>
Any help would be appreciated has it prevent us to use FUSE
mount.<br>
<br>
Regards,<br>
--<br>
Y.<br>
<br>
Le 22/07/2016 14:06, Yannick Perret a écrit :<br>
</div>
<blockquote cite="mid:57920C48.5070208@liris.cnrs.fr" type="cite">Hello,
<br>
some times ago I posted about a memory leak in client process,
but it was on a very old 32bit machine (both kernel and OS) and
I don't found evidences about a similar problem on our recent
machines. <br>
But I performed more tests and I have the same problem. <br>
<br>
Clients are 64bit Debian 8.2 machines. Glusterfs client on these
machines is compiled from sources with activated stuff: <br>
FUSE client : yes <br>
Infiniband verbs : no <br>
epoll IO multiplex : yes <br>
argp-standalone : no <br>
fusermount : yes <br>
readline : yes <br>
georeplication : yes <br>
Linux-AIO : no <br>
Enable Debug : no <br>
systemtap : no <br>
Block Device xlator : no <br>
glupy : no <br>
Use syslog : yes <br>
XML output : yes <br>
QEMU Block formats : no <br>
Encryption xlator : yes <br>
Erasure Code xlator : yes <br>
<br>
I tested both 3.6.7 and 3.6.9 version on client (3.6.7 is the
one installed on our machines, even on servers, 3.6.9 is for
testing with last 3.6 version). <br>
<br>
Here are the operations on the client (also performed with
similar results with 3.6.7 version): <br>
# /usr/local/sbin/glusterfs --version <br>
glusterfs 3.6.9 built on Jul 22 2016 13:27:42 <br>
(…) <br>
# mount -t glusterfs sto1.my.domain:BACKUP-ADMIN-DATA /zog/ <br>
# cd /usr/ <br>
# cp -Rp * /zog/TEMP/ <br>
Then monitoring memory used by glusterfs process while 'cp' is
running (resp. VSZ and RSS from 'ps'): <br>
284740 70232 <br>
284740 70232 <br>
284876 71704 <br>
285000 72684 <br>
285136 74008 <br>
285416 75940 <br>
(…) <br>
368684 151980 <br>
369324 153768 <br>
369836 155576 <br>
370092 156192 <br>
370092 156192 <br>
Here both sizes are stable and correspond to the end of 'cp'
command. <br>
If I restart an other 'cp' (even on the same directories) size
starts again to increase. <br>
If I perform a 'ls -lR' in the directory size also increase: <br>
370756 192488 <br>
389964 212148 <br>
390948 213232 <br>
(here I ^C the 'ls') <br>
<br>
When doing nothing the size don't increase but never decrease
(calling 'sync' don't change the situation). <br>
Sending a HUP signal to glusterfs process also increases memory
(390948 213324 → 456484 213320). <br>
Changing volume configuration (changing
diagnostics.client-sys-log-level value) don't change anything. <br>
<br>
Here the actual ps: <br>
root 17041 4.9 5.2 456484 213320 ? Ssl 13:29 1:21
/usr/local/sbin/glusterfs --volfile-server=sto1.my.domain
--volfile-id=BACKUP-ADMIN-DATA /zog <br>
<br>
Of course umouting/remounting fall back to "start" size: <br>
# umount /zog <br>
# mount -t glusterfs sto1.my.domain:BACKUP-ADMIN-DATA /zog/ <br>
→ root 28741 0.3 0.7 273320 30484 ? Ssl 13:57
0:00 /usr/local/sbin/glusterfs --volfile-server=sto1.my.domain
--volfile-id=BACKUP-ADMIN-DATA /zog <br>
<br>
<br>
I didn't saw this before because most of our volumes are mounted
"on demand" for some storage activities or are permanently
mounted but with very few activity. <br>
But clearly this memory usage driff is a long-term problem. On
the old 32bit machine I had this problem ("solved" by using NFS
mounts in order to wait for this old machine to be replaced) and
it lead to glusterfs being killed by OS when out of free memory.
It was faster than what I describe here but it's just a question
of time. <br>
<br>
<br>
Thanks for any help about that. <br>
<br>
Regards, <br>
-- <br>
Y. <br>
<br>
<br>
The corresponding volume on servers is (if it can help): <br>
Volume Name: BACKUP-ADMIN-DATA <br>
Type: Replicate <br>
Volume ID: 306d57f3-fb30-4bcc-8687-08bf0a3d7878 <br>
Status: Started <br>
Number of Bricks: 1 x 2 = 2 <br>
Transport-type: tcp <br>
Bricks: <br>
Brick1: sto1.my.domain:/glusterfs/backup-admin/data <br>
Brick2: sto2.my.domain:/glusterfs/backup-admin/data <br>
Options Reconfigured: <br>
diagnostics.client-sys-log-level: WARNING <br>
<br>
<br>
<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>