<html><body><div style="font-family: garamond,new york,times,serif; font-size: 12pt; color: #000000"><div>Ah! It's the same issue. Just saw your volume info output. Enabling strict-write-ordering should ensure both size and disk usage are accurate.<br></div><div><br></div><div>-Krutika<br></div><div><br></div><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;" data-mce-style="border-left: 2px solid #1010FF; margin-left: 5px; padding-left: 5px; color: #000; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Lindsay Mathieson" <lindsay.mathieson@gmail.com><br><b>To: </b>"Krutika Dhananjay" <kdhananj@redhat.com><br><b>Cc: </b>"gluster-users" <gluster-users@gluster.org><br><b>Sent: </b>Wednesday, November 4, 2015 6:07:10 PM<br><b>Subject: </b>Re: [Gluster-users] Shard file size (gluster 3.7.5)<br><div><br></div><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On 4 November 2015 at 20:45, Krutika Dhananjay <span dir="ltr"><<a href="mailto:kdhananj@redhat.com" target="_blank" data-mce-href="mailto:kdhananj@redhat.com">kdhananj@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" data-mce-style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;"><div>The block count in the xattr doesn't amount to 16GB of used space.<br></div><div><br></div><div>Is this consistently reproducible? If it is, then could you share the steps? That would help me recreate this in-house and debug it.<br></div><div><br></div></blockquote></div><br><div><br></div></div><div class="gmail_extra">100% of the time for me, and all I have to do is copy or create a file on the gluster mount.<br><div><br></div></div><div class="gmail_extra">My bricks are all sitting on ZFS filesystems with compression enabled, maube that is confusing things? I'll try a test with compression off.<br><div><br></div></div><div class="gmail_extra">In the mean time here are the steps and results for a from scract volume I created (datastore3) with just one file.<br><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><br>root@vnb:/mnt/pve/gluster3# gluster volume info<br></span></span><div><br></div><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;">Volume Name: datastore3<br>Type: Replicate<br>Volume ID: 96acb55b-b3c2-4940-b642-221dd1b88617<br>Status: Started<br>Number of Bricks: 1 x 3 = 3<br>Transport-type: tcp<br>Bricks:<br>Brick1: vna.proxmox.softlog:/zfs_vm/datastore3<br>Brick2: vnb.proxmox.softlog:/glusterdata/datastore3<br>Brick3: vng.proxmox.softlog:/glusterdata/datastore3<br>Options Reconfigured:<br>performance.io-thread-count: 32<br>performance.write-behind-window-size: 128MB<br>performance.cache-size: 1GB<br>performance.cache-refresh-timeout: 4<br>nfs.disable: on<br>nfs.addr-namelookup: off<br>nfs.enable-ino32: on<br>performance.write-behind: on<br>cluster.self-heal-window-size: 256<br>server.event-threads: 4<br>client.event-threads: 4<br>cluster.quorum-type: auto<br>features.shard-block-size: 512MB<br>features.shard: on<br>performance.readdir-ahead: on<br>cluster.server-quorum-ratio: 51%<br></span></span><div><br></div><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;">root@vnb:/mnt/pve/gluster3# dd if=/dev/sda of=test.bin bs=1MB count=8192<br>8192+0 records in<br>8192+0 records out<br>8192000000 bytes (8.2 GB) copied, 79.5335 s, 1<br></span></span><div><br></div><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;">ls -l<br>total 289925<br>drwxr-xr-x 2 root root 2 Nov 4 22:24 images<br>-rw-r--r-- 1 root root 72357920896 Nov 4 22:26 test.bin<br></span></span><div><br></div><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;">ls -lh<br>total 284M<br>drwxr-xr-x 2 root root 2 Nov 4 22:24 images<br>-rw-r--r-- 1 root root 68G Nov 4 22:26 test.bin<br></span></span><div><br></div><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;">du test.bin<br>289924 test.bin<br></span></span><div><br></div><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;">du /glusterdata/datastore3/.shard/<br>2231508 /glusterdata/datastore3/.shard/<br></span></span><div><br></div><span style="font-family:monospace,monospace" data-mce-style="font-family: monospace,monospace;"><br>getfattr -d -m . -e hex /glusterdata/datastore3/test.bin<br>getfattr: Removing leading '/' from absolute path names<br># file: glusterdata/datastore3/test.bin<br>trusted.afr.dirty=0x000000000000000000000000<br>trusted.bit-rot.version=0x02000000000000005639f915000f2b76<br>trusted.gfid=0xa1ecf4c8ab0a4ecc8bd8d4f3affe0bfb<br>trusted.glusterfs.shard.block-size=0x0000000020000000<br>trusted.glusterfs.shard.file-size=0x00000010d8de40800000000000000000000000000008d9080000000000000000<br></span><br></div><div class="gmail_extra"><br clear="all"><br>-- <br><div class="gmail_signature">Lindsay</div></div></div></blockquote><div><br></div></div></body></html>