<html><body><div style="font-family: garamond,new york,times,serif; font-size: 12pt; color: #000000"><div>OK. I am not sure what it is that we're doing differently. I tried the steps you shared and here's what I got:<br></div><div><br></div><div>[root@dhcp35-215 bricks]# gluster volume info<br> <br>Volume Name: rep<br>Type: Replicate<br>Volume ID: 3fd45a4b-0d02-4a44-b74a-41592d48e102<br>Status: Started<br>Number of Bricks: 1 x 3 = 3<br>Transport-type: tcp<br>Bricks:<br>Brick1: kdhananjay:/bricks/1<br>Brick2: kdhananjay:/bricks/2<br>Brick3: kdhananjay:/bricks/3<br>Options Reconfigured:<br>performance.strict-write-ordering: on<br>features.shard: on<br>features.shard-block-size: 512MB<br>cluster.quorum-type: auto<br>client.event-threads: 4<br>server.event-threads: 4<br>cluster.self-heal-window-size: 256<br>performance.write-behind: on<br>nfs.enable-ino32: on<br>nfs.addr-namelookup: off<br>nfs.disable: on<br>performance.cache-refresh-timeout: 4<br>performance.cache-size: 1GB<br>performance.write-behind-window-size: 128MB<br>performance.io-thread-count: 32<br>performance.readdir-ahead: on<br><br></div><div>[root@dhcp35-215 mnt]# gluster volume set rep strict-write-ordering on<br>volume set: success<br>[root@dhcp35-215 mnt]# dd if=/dev/sda of=test.bin bs=1MB count=8192<br>8192+0 records in<br>8192+0 records out<br>8192000000 bytes (8.2 GB) copied, 133.754 s, 61.2 MB/s<br>[root@dhcp35-215 mnt]# ls -l<br>total 8000000<br>-rw-r--r--. 1 root root 8192000000 Nov 5 16:40 test.bin<br>[root@dhcp35-215 mnt]# ls -lh<br>total 7.7G<br>-rw-r--r--. 1 root root 7.7G Nov 5 16:40 test.bin<br>[root@dhcp35-215 mnt]# du test.bin <br>8000000 test.bin<br><br></div><div>[root@dhcp35-215 bricks]# du /bricks/1/.shard/<br>7475780 /bricks/1/.shard/<br>[root@dhcp35-215 bricks]# du /bricks/1/<br>.glusterfs/ .shard/ test.bin .trashcan/ <br>[root@dhcp35-215 bricks]# du /bricks/1/test.bin <br>524292 /bricks/1/test.bin<br><br></div><div>Just to be sure, did you rerun the test on the already broken file (test.bin) which was written to when strict-write-ordering had been off?<br></div><div>Or did you try the new test with strict-write-ordering on a brand new file?<br></div><div><br></div><div>-Krutika<br></div><div><br></div><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;" data-mce-style="border-left: 2px solid #1010FF; margin-left: 5px; padding-left: 5px; color: #000; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Lindsay Mathieson" <lindsay.mathieson@gmail.com><br><b>To: </b>"Krutika Dhananjay" <kdhananj@redhat.com><br><b>Cc: </b>"gluster-users" <gluster-users@gluster.org><br><b>Sent: </b>Thursday, November 5, 2015 3:04:51 AM<br><b>Subject: </b>Re: [Gluster-users] Shard file size (gluster 3.7.5)<br><div><br></div><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On 5 November 2015 at 01:09, Krutika Dhananjay <span dir="ltr"><<a href="mailto:kdhananj@redhat.com" target="_blank" data-mce-href="mailto:kdhananj@redhat.com">kdhananj@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" data-mce-style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;">Ah! It's the same issue. Just saw your volume info output. Enabling strict-write-ordering should ensure both size and disk usage are accurate.</blockquote></div><br></div><div class="gmail_extra">Tested it - nope :( Size s accurate (27746172928 bytes), but disk usage is wildly inaccurate (698787).<br><div><br></div></div><div class="gmail_extra">I have compression disabled on the underlying storage now.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br clear="all"><br>-- <br><div class="gmail_signature">Lindsay</div></div></div></blockquote><div><br></div></div></body></html>