<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi all, sorry for 2 threads today, but I felt like this deserved a separate thread…<div class=""><br class=""></div><div class="">I was trying to replace my replica 2 volumes with replica 3 arbiter 1 volumes… The new volumes though are 10x slower on direct writes than their replica 2 counterparts. Im wondering if this is to be expected or if I might have done something wrong? I confirmed that no data is being written into the arbiter bricks, just meta data.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""> Here’s the tests I ran. I did the dd tests multiple times with different file names, and they all had the same speeds ….</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 wtg002]# dd if=/dev/zero of=test bs=1M count=100 oflag=direct</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">100+0 records in</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">100+0 records out</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">104857600 bytes (105 MB) copied, 9.49974 s, 11.0 MB/s</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 wtg002]# cd ..</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 home]# cd wtg001</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 wtg001]# dd if=/dev/zero of=test bs=1M count=100 oflag=direct</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">100+0 records in</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">100+0 records out</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">104857600 bytes (105 MB) copied, 0.888929 s, 118 MB/s</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 wtg001]# gluster volume info wtg001</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Volume Name: wtg001</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Type: Distributed-Replicate</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Volume ID: 53179cfe-9896-4c94-9f1d-01dd474e027e</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Status: Started</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Number of Bricks: 2 x 2 = 4</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Transport-type: tcp</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Bricks:</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick1: xs141:/brick1/wtg001p0r0</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick2: xs138:/brick1/wtg001p0r1</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick3: xs139:/brick1/wtg001p1r0</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick4: xs140:/brick1/wtg001p1r1</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Options Reconfigured:</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">nfs.disable: on</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">performance.readdir-ahead: on</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 wtg001]# gluster volume info wtg002</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Volume Name: wtg002</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Type: Distributed-Replicate</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Volume ID: 410b67ad-bc1e-473b-b98f-ad431d7c9831</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Status: Started</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Number of Bricks: 2 x 3 = 6</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Transport-type: tcp</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Bricks:</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick1: xs141:/brick1/wtg001d2r0</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick2: xs138:/brick1/wtg001d2r1</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick3: xs139:/brick1/wtg001d2ra</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick4: xs139:/brick1/wtg001d3r0</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick5: xs140:/brick1/wtg001d3r1</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Brick6: xs141:/brick1/wtg001d3ra</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">Options Reconfigured:</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">nfs.disable: on</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">performance.readdir-ahead: on</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 wtg001]# mount | grep wtg</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">0:/wtg001 on /home/wtg001 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">0:/wtg002 on /home/wtg002 type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)</div><div style="margin: 0px; line-height: normal; color: rgb(69, 69, 69);" class="">[root@fs134 wtg001]#</div></div><div class=""><br class=""></div><div class=""><br class=""></div><img src="http://t.sidekickopen04.com/e1t/o/5/f18dQhb0S7ks8dDMPbW2n0x6l2B9gXrN7sKj6v4LN9-W2z8_hM5wLWr-N7dSKlbd3_yKW6jq8hk1k1H6H0?si=6102096971038720&pi=E9747E1A-C2BC-4FFB-B159-E8BC5ADEC31B" width="1" height="1" style="display:none!important"></body></html>