<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Hi Thibault,<br>
<br>
There are a few tuneables that have helped boost ganesha
performance and I suspect these tuneables on the OS side apply to
improve performance for several workloads where ganesha is
concerned. (And they don't seem to be necessary for gluster-nfs at
all).<br>
<br>
Adding Manoj here, who may be able to point you to these
configurables (as he has experimented with the ganesha
performance).<br>
<br>
Thanks,<br>
Anand<br>
<br>
On 08/13/2015 07:13 PM, Niels de Vos wrote:<br>
</div>
<blockquote
cite="mid:20150813134305.GD10840@ndevos-x240.usersys.redhat.com"
type="cite">
<pre wrap="">On Mon, Aug 10, 2015 at 09:19:25AM +0100, Thibault Godouet wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Thanks Niel for your helpful answer.
Regarding the locking, indeed that solves my issue. Now I'm wondering how
to monitor this. The best I have so far is get the list of RPC binds and
the TCP/UDP port in particular, and then run a lsof to find out if it is
Gluster. Should work, but a bit indirect. If someone knows a better way
I'd be interested to know.
</pre>
</blockquote>
<pre wrap="">
That is almost how I do it as well. Instead of 'lsof' I use 'netstat' or
'ss', depending on the Linux distribution.
</pre>
<blockquote type="cite">
<pre wrap="">As for Ganesha, I saw articles explaining that it effectively removes
layers, hence why I thought NFS v3 via Ganesha would be faster than native
Gluster NFS. Given your answer I take it there are other moving parts /
differences. Is there a general known guideline on which is best when?
E.g. does one handle better small files than the other one or something
like that?
</pre>
</blockquote>
<pre wrap="">
I am not aware of any guidelines for this. The difference in performance
is highly dependent on the workload and use-case. There is little
difference in the layers between Gluster/NFS and NFS-Ganesha, both are
userspace nfs-server implementations (neither has context switches for
the Linux-VFS like fuse mounts have).
If you need the best performance, you should problaby just try both
configurations, and run your intended workload against the servers.
Artificial/standard tests most often do not emulate a real workload.
HTH,
Niels
</pre>
<blockquote type="cite">
<pre wrap="">On 5 Aug 2015 7:06 pm, "Niels de Vos" <a class="moz-txt-link-rfc2396E" href="mailto:ndevos@redhat.com"><ndevos@redhat.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">On Wed, Aug 05, 2015 at 04:11:47PM +0100, Thibault Godouet wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Looking around I get the impression that file locking (NLM) may simply
</pre>
</blockquote>
<pre wrap="">not
</pre>
<blockquote type="cite">
<pre wrap="">be supported in glusterfs's built-in NFS server.
</pre>
</blockquote>
<pre wrap="">
This is actually supported. But note that you can not run a userspace
NLM implementation provided by a NFS-server (Gluster/NFS or NFS-Ganesha)
on a system that acts as an NFS-client. The Linux kernel NFS-client uses
the lockd kernel module, and there can be only one NLM implementation be
registered at rpcbind. Whichever service (nfs-client or nfs-server)
starts first, will be able to register itself, the 2nd one will (mostly
silently) fail.
</pre>
<blockquote type="cite">
<pre wrap="">I get the impression that Ganesha is aimed at supporting NFS better, and
presumably supports locking well, so I should give it a try (If I
understand well the performance is also likely to be higher, which is a
nice bonus!)
</pre>
</blockquote>
<pre wrap="">
NFS-Ganesha offers more features than Gluster/NFS. The performance is
highly dependent on the workload, Gluster/NFS can be faster for many of
them.
Cheers,
Niels
</pre>
<blockquote type="cite">
<pre wrap="">
If someone could confirm this that would be useful to make sure I'm going
in the right direction.
Thanks,
Thibault.
On 4 Aug 2015 1:23 pm, "Thibault Godouet" <a class="moz-txt-link-rfc2396E" href="mailto:tibo92@godouet.net"><tibo92@godouet.net></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
I have a cluster of 2 servers running 3.7.3 with replication, and
</pre>
</blockquote>
</blockquote>
<pre wrap="">standard
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">NFS (no ganesha). This in on CentOS 6.
I use CTDB with 2 virtual IPs (one for each server in a normal
situation) to share the volume over NFS and CIFS (samba).
fnctl() file locking doesn't seem to work when the volume is mounted
</pre>
</blockquote>
</blockquote>
<pre wrap="">over
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">NFS.
This is apparent with a 'svn info' (svn 1.8 if it made any difference)
</pre>
</blockquote>
</blockquote>
<pre wrap="">in
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">a local working copy:
$ svn info
svn: E200033: Another process is blocking the working copy database, or
the underlying filesystem does not support file locking; if the working
copy is on a network filesystem, make sure file locking has been
</pre>
</blockquote>
</blockquote>
<pre wrap="">enabled on
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">the file server
svn: E200033: sqlite[S5]: database is locked, executing statement
</pre>
</blockquote>
</blockquote>
<pre wrap="">'PRAGMA
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">synchronous=OFF;PRAGMA recursive_triggers=ON;PRAGMA
</pre>
</blockquote>
</blockquote>
<pre wrap="">foreign_keys=OFF;PRAGMA
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">locking_mode = NORMAL;'
a strace shows:
$ svn info
svn: E200033: Another process is blocking the working copy database, or
the underlying filesystem does not support file locking; if the working
copy is on a network filesystem, make sure file locking has been
</pre>
</blockquote>
</blockquote>
<pre wrap="">enabled on
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">the file server
svn: E200033: sqlite[S5]: database is locked, executing statement
</pre>
</blockquote>
</blockquote>
<pre wrap="">'PRAGMA
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">synchronous=OFF;PRAGMA recursive_triggers=ON;PRAGMA
</pre>
</blockquote>
</blockquote>
<pre wrap="">foreign_keys=OFF;PRAGMA
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">locking_mode = NORMAL;'
Everything seems to work fine on native Gluster (FUSE) mounts: the same
'svn info' works nicely.
I can't really use native mounts due to the performance hit (many small
files) and the fact I would need to install the gluster client
</pre>
</blockquote>
</blockquote>
<pre wrap="">software on
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">every server.
Is fnctl() file locking supported in Gluster NFS mounts? If so, any
</pre>
</blockquote>
</blockquote>
<pre wrap="">idea
</pre>
<blockquote type="cite">
<blockquote type="cite">
<pre wrap="">why it doesn't work for me?
Thanks,
Thibault.
</pre>
</blockquote>
</blockquote>
<pre wrap="">
</pre>
<blockquote type="cite">
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a>
</pre>
</blockquote>
<pre wrap="">
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
</blockquote>
</blockquote>
<br>
</body>
</html>