[Gluster-users] Two nodes both as server+client

Daniel Jordan Bambach dan at lateral.net
Tue Jun 10 15:44:13 UTC 2008


Thanks to Anand, I have some serious speed ups on local machine  
performance, by combining the server and client within the client  
config. This removed a few overheads, and both write and read speeds  
are up on each individual machine

However - using the attached spec files, neither server is able to  
connect to the other, and I am stumped as to why, each log file reads  
the equiv of:

2008-06-10 13:07:32 E [tcp-client.c:171:tcp_connect] latsrv2-local:  
non-blocking connect() returned: 111 (Connection refused)

This simply looks like there is no protocol/sever to connect to for  
the other client.

Is anyone able to spot a howler in here, or is it something more  
fundamental?

P.S. Apologies to Anand for sending this to you twice!

The two client specs are here (as there are no longer any server specs!)

LATSRV1:
#Start with server defs in our client conf.
#We can save on the overhead of a seperate glusterfsd
#Becuase we are always running a server+client pair

volume posix
  type storage/posix
  option directory /home/export2
end-volume

volume plocks
   type features/posix-locks
   subvolumes posix
end-volume

volume latsrv1-local
  type performance/io-threads
  option thread-count 8
  option cache-size 64MB
  subvolumes plocks
end-volume

volume server
  type protocol/server
  option transport-type tcp/server	# For TCP/IP transport
  option auth.ip.latsrv1-local.allow *		# Allow access to "brick" volume
  subvolumes latsrv1-local
end-volume

#Continue with the client spec..

volume latsrv2-local
  type protocol/client
  option transport-type tcp/client
  option remote-host latsrv2
# option remote-subvolume 195.224.189.148 #fake to model unavailable  
server
  option remote-subvolume latsrv2-local
end-volume

volume data-afr
  type cluster/afr
  subvolumes latsrv1-local latsrv2-local
  option read-subvolume latsrv1-local
  option self-heal on
end-volume

volume data
  type performance/read-ahead
  option page-size 128kB		# 256KB is the default option
  option page-count 4			# 2 is default option
  option force-atime-update off	# default is off
  subvolumes data-afr
end-volume

#we will mount the volume data.



LATSRV2:
#Start with server defs in our client conf.
#We can save on the overhead of a seperate glusterfsd
#Becuase we are always running a server+client pair

volume posix
  type storage/posix
  option directory /home/export2
end-volume

volume plocks
   type features/posix-locks
   subvolumes posix
end-volume

volume latsrv2-local
  type performance/io-threads
  option thread-count 8
  option cache-size 64MB
  subvolumes plocks
end-volume

volume server
  type protocol/server
  option transport-type tcp/server	# For TCP/IP transport
  option auth.ip.latsrv2-local.allow *		# Allow access to "brick" volume
  subvolumes latsrv2-local
end-volume

#Continue with the client spec..

volume latsrv1-local
  type protocol/client
  option transport-type tcp/client
  option remote-host latsrv1
# option remote-subvolume 195.224.189.148 #fake to model unavailable  
server
  option remote-subvolume latsrv1-local
end-volume

volume data-afr
  type cluster/afr
  subvolumes latsrv1-local latsrv2-local
  option read-subvolume latsrv2-local
  option self-heal on
end-volume

volume data
  type performance/read-ahead
  option page-size 128kB		# 256KB is the default option
  option page-count 4			# 2 is default option
  option force-atime-update off	# default is off
  subvolumes data-afr
end-volume

#we will mount the volume data.



On 5 Jun 2008, at 20:51, Anand Babu Periasamy wrote:

> There is lot of scope for improvement in performance and simplicity.
>
> Booster translator will help only when you LD_PRELOAD glusterfs- 
> booster.so
> before launching your applications. It bypasses kernel-fuse for  
> reads and
> writes. Even in that case, it makes sense to load booster translator  
> on
> the
> server side.
>
> In your setup, you have 2 servers acting as complete mirror for each  
> other
> (server and client for each other). You can merge client and server  
> into
> one process by loading protocol/server into the client space. It  
> will be a
> lot simpler and faster. Just 2 vol spec files.
>
> In upcoming 1.4, you will also be able to use the web embeddable  
> glusterfs
> client to directly access the storage from apache addresses space  
> (or even
> run the whole file system inside apache or lighttpd). It also has  
> binary
> protocol (fast and efficient) and non-blocking I/O functionalities.
>
> Please see the attached PDF. It will give you a good idea.
>
> --
> Anand Babu Periasamy
> GPG Key ID: 0x62E15A31
> Blog [http://ab.freeshell.org]
> The GNU Operating System [http://www.gnu.org]
> Z RESEARCH Inc [http://www.zresearch.com]
>
>
>
> Daniel Jordan Bambach wrote:
>> Hiya all..
>>
>> A scenario that seems to be a very neat solution to a basic high
> availability Webserver set up (Apache, Mysql, Python+Django) is to set
> up two machines, configure master<->master replication between the two
> MySQL databases, and then set up GlusterFS to mirror a filesystem
> between the machine that carries the Apache config, Django
>> applications, and file upload folders between the machines. You can
> pull the plug on either, and things should keep running on the other.
>>
>> With this in mind, I have set up an arrangement whereby each box runs
> GlusterFSD, and has a client running on them that connects the local
> server. AFR is set up at the server level, so that perhaps when/if the
> other machine goes down, the client happily carries on dealing with
> read/ write requests while the server deals with the non-existence of
> the other server.
>>
>> I've set this up in a test environment, and all is working peachy,  
>> and
> we are thinking of moving to deploy this to a new production
>> environment.
>>
>> With this in mind, I wanted to poll the collective knowledge of this
> list to see if there are any gotchas to this set up I might have
> missed, or any obvious performance features I should be using that I
> am not.
>>
>> Any help or advise would be greatly appreciated!!
>>
>> Here are the current server and client configs for the two machines:
>>
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>>
>
>
> <GlusterFS-Layout.pdf><GlusterFS-Layout.odp>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20080610/182ea28c/attachment.html>


More information about the Gluster-users mailing list