[Gluster-users] small write speed problem on EBS, distributed replica

[Mohit Anchlia]

Can you first run some test with no replica and see what results you
get? Also, can you look at network latency from client to each of your
4 servers and post the results?

On Mon, Mar 21, 2011 at 1:27 AM, karol skocik <karol.skocik at gmail.com> wrote:
> Hi,
>  I am in the process of evaluation of Gluster for major BI company,
> but I was surprised by very small write performance on Amazon EBS.
> Our setup is Gluster 3.1.2, distributed replica 2x2 on 64-bit m1.large
> instances. Every server node has 1 EBS volume attached to it.
> The configuration of the distributed replica is a default one, my
> small attemps to improve performance (io-threads, disabled io-stats
> and latency-measurement):
>
> volume EBSVolume-posix
>    type storage/posix
>    option directory /mnt/ebs
> end-volume
>
> volume EBSVolume-access-control
>    type features/access-control
>    subvolumes EBSVolume-posix
> end-volume
>
> volume EBSVolume-locks
>    type features/locks
>    subvolumes EBSVolume-access-control
> end-volume
>
> volume EBSVolume-io-threads
>    type performance/io-threads
>    option thread-count 4
>    subvolumes EBSVolume-locks
> end-volume
>
> volume /mnt/ebs
>    type debug/io-stats
>    option log-level NONE
>    option latency-measurement off
>    subvolumes EBSVolume-io-threads
> end-volume
>
> volume EBSVolume-server
>    type protocol/server
>    option transport-type tcp
>    option auth.addr./mnt/ebs.allow *
>    subvolumes /mnt/ebs
> end-volume
>
> In our test, all clients starts writing to different 1GB file at the same time.
> The measured write bandwidth, with 2x2 servers:
>
> 1 client: 6.5 MB/s
> 2 clients: 4.1 MB/s
> 3 clients: 2.4 MB/s
> 4 clients: 4.3 MB/s
>
> This is not acceptable for our needs. With PVFS2 (I know it's
> stripping which is very different from replica) we can get up to 35
> MB/s.
> 2-3 times slower than that would be understandable. But 5-15 times
> slower is not, and I would like to know whether there is something we
> could try out.
>
> Could anybody publish their write speeds on similar setup, and tips
> how to achieve better performance?
>
> Thank you,
>  Karol
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
>