<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    Okay so i have enabled shard in my test volume and it did not help,
    stupidly enough, i have enabled it in a production volume
    "Distributed-Replicate" and it currpted  half of my VMs.<br>
    I have updated Gluster to the latest and nothing seems to be changed
    in my situation.<br>
    below the info of my volume;<br>
    <br>
    Number of Bricks: 3 x 2 = 6<br>
    Transport-type: tcp<br>
    Bricks:<br>
    Brick1: gfs001:/bricks/b001/vmware<br>
    Brick2: gfs002:/bricks/b004/vmware<br>
    Brick3: gfs001:/bricks/b002/vmware<br>
    Brick4: gfs002:/bricks/b005/vmware<br>
    Brick5: gfs001:/bricks/b003/vmware<br>
    Brick6: gfs002:/bricks/b006/vmware<br>
    Options Reconfigured:<br>
    performance.strict-write-ordering: on<br>
    cluster.server-quorum-type: server<br>
    cluster.quorum-type: auto<br>
    network.remote-dio: enable<br>
    performance.stat-prefetch: disable<br>
    performance.io-cache: off<br>
    performance.read-ahead: off<br>
    performance.quick-read: off<br>
    cluster.eager-lock: enable<br>
    features.shard-block-size: 16MB<br>
    features.shard: on<br>
    performance.readdir-ahead: off<br>
    <br>
    <br>
    <div class="moz-cite-prefix">On 03/12/2016 08:11 PM, David Gossage
      wrote:<br>
    </div>
    <blockquote
cite="mid:CAJXeYEXh5jg22zKAYvLNXQbvkVH7fmttm7X15Q+gJ2NEfFp3zA@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Sat, Mar 12, 2016 at 10:21 AM,
            Mahdi Adnan <span dir="ltr">&lt;<a moz-do-not-send="true"
                href="mailto:mahdi.adnan@earthlinktele.com"
                target="_blank">mahdi.adnan@earthlinktele.com</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF"> Both servers have
                HBA no RAIDs and i can setup a replicated or dispensers
                without any issues.<br>
                Logs are clean and when i tried to migrate a vm and got
                the error, nothing showed up in the logs.<br>
                i tried mounting the volume into my laptop and it
                mounted fine but, if i use dd to create a data file it
                just hang and i cant cancel it, and i cant unmount it or
                anything, i just have to reboot.<br>
                The same servers have another volume on other bricks in
                a distributed replicas, works fine.<br>
                I have even tried the same setup in a virtual
                environment (created two vms and install gluster and
                created a replicated striped) and again same thing, data
                corruption.<br>
              </div>
            </blockquote>
            <div><br>
            </div>
            <div>I'd look through mail archives for a topic "Shard in
              Production" I think it's called.  The shard portion may
              not be relevant but it does discuss certain settings that
              had to be applied with regards to avoiding corruption with
              VM's.  You may want to try and disable the
               performance.readdir-ahead also.</div>
            <div><br>
            </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF"> <br>
                <br>
                <div>On 03/12/2016 07:02 PM, David Gossage wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div class="gmail_extra">
                      <div>
                        <div>
                          <div dir="ltr">
                            <div><span><font color="#888888"><font><font
                                      size="1"><span
                                        style="color:rgb(51,51,51)"><span
                                          style="color:rgb(153,153,153)"><a
                                            moz-do-not-send="true"
                                            value="+17086132426"><font
                                              color="#888888"><font
                                                size="1"><br>
                                              </font></font></a></span></span></font></font></font></span></div>
                          </div>
                        </div>
                      </div>
                      <br>
                      <div class="gmail_quote">On Sat, Mar 12, 2016 at
                        9:51 AM, Mahdi Adnan <span dir="ltr">&lt;<a
                            moz-do-not-send="true"
                            href="mailto:mahdi.adnan@earthlinktele.com"
                            target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mahdi.adnan@earthlinktele.com">mahdi.adnan@earthlinktele.com</a></a>&gt;</span>
                        wrote:<br>
                        <blockquote class="gmail_quote"
                          style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                          <div text="#000000" bgcolor="#FFFFFF"> Thanks
                            David,<br>
                            <br>
                            My settings are all defaults, i have just
                            created the pool and started it.<br>
                            I have set the settings as your
                            recommendation and it seems to be the same
                            issue;<br>
                            <br>
                            Type: Striped-Replicate<br>
                            Volume ID:
                            44adfd8c-2ed1-4aa5-b256-d12b64f7fc14<br>
                            Status: Started<br>
                            Number of Bricks: 1 x 2 x 2 = 4<br>
                            Transport-type: tcp<br>
                            Bricks:<br>
                            Brick1: gfs001:/bricks/t1/s<br>
                            Brick2: gfs002:/bricks/t1/s<br>
                            Brick3: gfs001:/bricks/t2/s<br>
                            Brick4: gfs002:/bricks/t2/s<br>
                            Options Reconfigured:<br>
                            performance.stat-prefetch: off<br>
                            network.remote-dio: on<br>
                            cluster.eager-lock: enable<br>
                            performance.io-cache: off<br>
                            performance.read-ahead: off<br>
                            performance.quick-read: off<br>
                            performance.readdir-ahead: on<br>
                          </div>
                        </blockquote>
                        <div> </div>
                        <div><br>
                        </div>
                        <div>Is their a raid controller perhaps doing
                          any caching?  <br>
                        </div>
                        <div><br>
                        </div>
                        <div>In the gluster logs any errors being
                          reported during migration process?</div>
                        <div>Since they aren't in use yet have you
                          tested making just mirrored bricks using
                          different pairings of servers two at a time to
                          see if problem follows certain machine or
                          network ports?</div>
                        <div><br>
                        </div>
                        <div> </div>
                        <blockquote class="gmail_quote"
                          style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                          <div text="#000000" bgcolor="#FFFFFF"> <br>
                            <br>
                            <br>
                            <br>
                            <br>
                            <div>On 03/12/2016 03:25 PM, David Gossage
                              wrote:<br>
                            </div>
                            <blockquote type="cite">
                              <div dir="ltr"><br>
                                <div class="gmail_extra"><br>
                                  <div class="gmail_quote">On Sat, Mar
                                    12, 2016 at 1:55 AM, Mahdi Adnan <span
                                      dir="ltr">&lt;<a
                                        moz-do-not-send="true"
                                        href="mailto:mahdi.adnan@earthlinktele.com"
                                        target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:mahdi.adnan@earthlinktele.com">mahdi.adnan@earthlinktele.com</a></a>&gt;</span>
                                    wrote:<br>
                                    <blockquote class="gmail_quote"
                                      style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                      <p dir="ltr">Dears,</p>
                                      <p dir="ltr">I have created a
                                        replicated striped volume with
                                        two bricks and two servers but I
                                        can't use it because when I
                                        mount it in ESXi and try to
                                        migrate a VM to it, the data get
                                        corrupted.<br>
                                        Is any one have any idea why is
                                        this happening ?</p>
                                      <p dir="ltr">Dell 2950 x2<br>
                                        Seagate 15k 600GB<br>
                                        CentOS 7.2<br>
                                        Gluster 3.7.8</p>
                                      <p dir="ltr">Appreciate your help.</p>
                                    </blockquote>
                                    <div><br>
                                    </div>
                                    <div>Most reports of this I have
                                      seen end up being settings
                                      related.  Post gluster volume
                                      info. Below is what I have seen as
                                      most common recommended settings.</div>
                                    <div>I'd hazard a guess you may have
                                      some the read ahead cache or
                                      prefetch on.</div>
                                    <div><br>
                                    </div>
                                    <div>quick-read=off</div>
                                    <div>read-ahead=off</div>
                                    <div>io-cache=off</div>
                                    <div>stat-prefetch=off</div>
                                    <div>eager-lock=enable</div>
                                    <div>remote-dio=on</div>
                                    <blockquote class="gmail_quote"
                                      style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                      <p dir="ltr"><span><font
                                            color="#888888"><br>
                                            Mahdi Adnan<br>
                                            System Admin<br>
                                            <br>
                                          </font></span></p>
                                      <br>
_______________________________________________<br>
                                      Gluster-users mailing list<br>
                                      <a moz-do-not-send="true"
                                        href="mailto:Gluster-users@gluster.org"
                                        target="_blank">Gluster-users@gluster.org</a><br>
                                      <a moz-do-not-send="true"
                                        href="http://www.gluster.org/mailman/listinfo/gluster-users"
                                        rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
                                    </blockquote>
                                  </div>
                                  <br>
                                </div>
                              </div>
                            </blockquote>
                            <br>
                          </div>
                        </blockquote>
                      </div>
                      <br>
                    </div>
                  </div>
                </blockquote>
                <br>
              </div>
            </blockquote>
          </div>
          <br>
        </div>
      </div>
    </blockquote>
    <br>
  </body>
</html>