<html><body><div style="font-family: garamond,new york,times,serif; font-size: 12pt; color: #000000"><div>The logs are at /var/log/glusterfs/<hyphenated-path-to-the-mountpoint>.log<br></div><div><br></div><div>OK. So what do you observe when you set group virt to on?<br></div><div><br></div><div># gluster volume set <VOL> group virt<br></div><div><br></div><div>-Krutika<br></div><div><br></div><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Lindsay Mathieson" <lindsay.mathieson@gmail.com><br><b>To: </b>"Krutika Dhananjay" <kdhananj@redhat.com><br><b>Cc: </b>"gluster-users" <gluster-users@gluster.org><br><b>Sent: </b>Friday, November 13, 2015 11:57:15 AM<br><b>Subject: </b>Re: [Gluster-users] File Corruption with shards - 100% reproducable<br><div><br></div><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On 12 November 2015 at 15:46, Krutika Dhananjay <span dir="ltr"><<a href="mailto:kdhananj@redhat.com" target="_blank">kdhananj@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>OK. What do the client logs say?<br></div></blockquote><div><br></div><div>Dumb question - Which logs are those? <br><div><br></div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div></div><div>Could you share the exact steps to recreate this, and I will try it locally on my setup?<br></div></blockquote><div><br></div><div>I'm running this on a 3 node proxmox cluster, which makes the vm creation & migration easy to test.<br><div><br></div></div><div>Steps:<br></div><div>- Create 3 node gluster datastore using proxmox vm host nodes<br></div><div><br>- Add gluster datastore as a storage dvice to proxmox<br></div><div> * qemu vms use the gfapi to access the datastore<br></div><div> * proxmox also adds a fuse mount for easy acces<br><div><br></div></div><div>- create a VM on the gluster storage, QCOW2 format. I just created a simple debain Mate vm<br><div><br></div></div><div>- start the vm, open a console to it.<br><div><br></div></div><div>- live migrate the VM to a another node<br><div><br></div></div><div>- It will rapdily barf itself with disk errors<br><div><br></div></div><div>- stop the VM<br></div><div><br></div><div>- qemu will show file corruption (many many errors)<br></div><div> * qemu-img check <vm disk image><br> * qemu-img info <vm disk image><br></div><div><br><div><br></div></div><div>Repeating the process with sharding off has no errors.<br></div><div><br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div></div><div><br></div><div>Also, want to see the output of 'gluster volume info'.</div></blockquote></div><br></div><div class="gmail_extra"><br>I've trimmed settings down to a bare minimum. This is a test gluster cluster so I can do with it as I wish.<br><div><br></div><br></div><div class="gmail_extra"><span style="font-family:monospace,monospace"><br>gluster volume info<br> <br>Volume Name: datastore1<br>Type: Replicate<br>Volume ID: 238fddd0-a88c-4edb-8ac5-ef87c58682bf<br>Status: Started<br>Number of Bricks: 1 x 3 = 3<br>Transport-type: tcp<br>Bricks:<br>Brick1: vnb.proxmox.softlog:/mnt/ext4<br>Brick2: vng.proxmox.softlog:/mnt/ext4<br>Brick3: vna.proxmox.softlog:/mnt/ext4<br>Options Reconfigured:<br>performance.strict-write-ordering: on<br>performance.readdir-ahead: off<br>cluster.quorum-type: auto<br>features.shard: on</span><br><div><br></div><br clear="all"><br>-- <br><div class="gmail_signature">Lindsay</div>
</div></div>
</blockquote><div><br></div></div></body></html>