<div dir="ltr"><br><div class="gmail_extra"><div><div class="gmail_signature"><div dir="ltr"><div><span><font color="#888888"><font><font size="1"><span style="color:rgb(51,51,51)"><span style="color:rgb(153,153,153)"><a value="+17086132426"><font color="#888888"><font size="1"><br></font></font></a></span></span></font></font></font></span></div></div></div></div>
<br><div class="gmail_quote">On Sun, Mar 13, 2016 at 8:16 AM, Mahdi Adnan <span dir="ltr">&lt;<a href="mailto:mahdi.adnan@earthlinktele.com" target="_blank">mahdi.adnan@earthlinktele.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    Okay so i have enabled shard in my test volume and it did not help,
    stupidly enough, i have enabled it in a production volume
    &quot;Distributed-Replicate&quot; and it currpted  half of my VMs.<br>
    I have updated Gluster to the latest and nothing seems to be changed
    in my situation.<br>
    below the info of my volume;<br></div></blockquote><div><br></div><div>I was pointing at the settings in that email as an example for corruption fixing. I wouldn&#39;t recommend enabling sharding if you haven&#39;t gotten the base working yet on that cluster. What HBA&#39;s are you using and what is layout of filesystem for bricks?  </div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
    <br>
    Number of Bricks: 3 x 2 = 6<br>
    Transport-type: tcp<br>
    Bricks:<br>
    Brick1: gfs001:/bricks/b001/vmware<br>
    Brick2: gfs002:/bricks/b004/vmware<br>
    Brick3: gfs001:/bricks/b002/vmware<br>
    Brick4: gfs002:/bricks/b005/vmware<br>
    Brick5: gfs001:/bricks/b003/vmware<br>
    Brick6: gfs002:/bricks/b006/vmware<br>
    Options Reconfigured:<br>
    performance.strict-write-ordering: on<br>
    cluster.server-quorum-type: server<br>
    cluster.quorum-type: auto<br>
    network.remote-dio: enable<br>
    performance.stat-prefetch: disable<br>
    performance.io-cache: off<br>
    performance.read-ahead: off<br>
    performance.quick-read: off<br>
    cluster.eager-lock: enable<br>
    features.shard-block-size: 16MB<br>
    features.shard: on<br>
    performance.readdir-ahead: off<br>
    <br>
    <br>
    <div>On 03/12/2016 08:11 PM, David Gossage
      wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Sat, Mar 12, 2016 at 10:21 AM,
            Mahdi Adnan <span dir="ltr">&lt;<a href="mailto:mahdi.adnan@earthlinktele.com" target="_blank">mahdi.adnan@earthlinktele.com</a>&gt;</span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF"> Both servers have
                HBA no RAIDs and i can setup a replicated or dispensers
                without any issues.<br>
                Logs are clean and when i tried to migrate a vm and got
                the error, nothing showed up in the logs.<br>
                i tried mounting the volume into my laptop and it
                mounted fine but, if i use dd to create a data file it
                just hang and i cant cancel it, and i cant unmount it or
                anything, i just have to reboot.<br>
                The same servers have another volume on other bricks in
                a distributed replicas, works fine.<br>
                I have even tried the same setup in a virtual
                environment (created two vms and install gluster and
                created a replicated striped) and again same thing, data
                corruption.<br>
              </div>
            </blockquote>
            <div><br>
            </div>
            <div>I&#39;d look through mail archives for a topic &quot;Shard in
              Production&quot; I think it&#39;s called.  The shard portion may
              not be relevant but it does discuss certain settings that
              had to be applied with regards to avoiding corruption with
              VM&#39;s.  You may want to try and disable the
               performance.readdir-ahead also.</div>
            <div><br>
            </div>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
              <div text="#000000" bgcolor="#FFFFFF"> <br>
                <br>
                <div>On 03/12/2016 07:02 PM, David Gossage wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div class="gmail_extra">
                      <div>
                        <div>
                          <div dir="ltr">
                            <div><span><font color="#888888"><font><font size="1"><span style="color:rgb(51,51,51)"><span style="color:rgb(153,153,153)"><a value="+17086132426"><font color="#888888"><font size="1"><br>
                                              </font></font></a></span></span></font></font></font></span></div>
                          </div>
                        </div>
                      </div>
                      <br>
                      <div class="gmail_quote">On Sat, Mar 12, 2016 at
                        9:51 AM, Mahdi Adnan <span dir="ltr">&lt;<a href="mailto:mahdi.adnan@earthlinktele.com" target="_blank"></a><a href="mailto:mahdi.adnan@earthlinktele.com" target="_blank">mahdi.adnan@earthlinktele.com</a>&gt;</span>
                        wrote:<br>
                        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                          <div text="#000000" bgcolor="#FFFFFF"> Thanks
                            David,<br>
                            <br>
                            My settings are all defaults, i have just
                            created the pool and started it.<br>
                            I have set the settings as your
                            recommendation and it seems to be the same
                            issue;<br>
                            <br>
                            Type: Striped-Replicate<br>
                            Volume ID:
                            44adfd8c-2ed1-4aa5-b256-d12b64f7fc14<br>
                            Status: Started<br>
                            Number of Bricks: 1 x 2 x 2 = 4<br>
                            Transport-type: tcp<br>
                            Bricks:<br>
                            Brick1: gfs001:/bricks/t1/s<br>
                            Brick2: gfs002:/bricks/t1/s<br>
                            Brick3: gfs001:/bricks/t2/s<br>
                            Brick4: gfs002:/bricks/t2/s<br>
                            Options Reconfigured:<br>
                            performance.stat-prefetch: off<br>
                            network.remote-dio: on<br>
                            cluster.eager-lock: enable<br>
                            performance.io-cache: off<br>
                            performance.read-ahead: off<br>
                            performance.quick-read: off<br>
                            performance.readdir-ahead: on<br>
                          </div>
                        </blockquote>
                        <div> </div>
                        <div><br>
                        </div>
                        <div>Is their a raid controller perhaps doing
                          any caching?  <br>
                        </div>
                        <div><br>
                        </div>
                        <div>In the gluster logs any errors being
                          reported during migration process?</div>
                        <div>Since they aren&#39;t in use yet have you
                          tested making just mirrored bricks using
                          different pairings of servers two at a time to
                          see if problem follows certain machine or
                          network ports?</div>
                        <div><br>
                        </div>
                        <div> </div>
                        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                          <div text="#000000" bgcolor="#FFFFFF"> <br>
                            <br>
                            <br>
                            <br>
                            <br>
                            <div>On 03/12/2016 03:25 PM, David Gossage
                              wrote:<br>
                            </div>
                            <blockquote type="cite">
                              <div dir="ltr"><br>
                                <div class="gmail_extra"><br>
                                  <div class="gmail_quote">On Sat, Mar
                                    12, 2016 at 1:55 AM, Mahdi Adnan <span dir="ltr">&lt;<a href="mailto:mahdi.adnan@earthlinktele.com" target="_blank"></a><a href="mailto:mahdi.adnan@earthlinktele.com" target="_blank">mahdi.adnan@earthlinktele.com</a>&gt;</span>
                                    wrote:<br>
                                    <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                      <p dir="ltr">Dears,</p>
                                      <p dir="ltr">I have created a
                                        replicated striped volume with
                                        two bricks and two servers but I
                                        can&#39;t use it because when I
                                        mount it in ESXi and try to
                                        migrate a VM to it, the data get
                                        corrupted.<br>
                                        Is any one have any idea why is
                                        this happening ?</p>
                                      <p dir="ltr">Dell 2950 x2<br>
                                        Seagate 15k 600GB<br>
                                        CentOS 7.2<br>
                                        Gluster 3.7.8</p>
                                      <p dir="ltr">Appreciate your help.</p>
                                    </blockquote>
                                    <div><br>
                                    </div>
                                    <div>Most reports of this I have
                                      seen end up being settings
                                      related.  Post gluster volume
                                      info. Below is what I have seen as
                                      most common recommended settings.</div>
                                    <div>I&#39;d hazard a guess you may have
                                      some the read ahead cache or
                                      prefetch on.</div>
                                    <div><br>
                                    </div>
                                    <div>quick-read=off</div>
                                    <div>read-ahead=off</div>
                                    <div>io-cache=off</div>
                                    <div>stat-prefetch=off</div>
                                    <div>eager-lock=enable</div>
                                    <div>remote-dio=on</div>
                                    <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
                                      <p dir="ltr"><span><font color="#888888"><br>
                                            Mahdi Adnan<br>
                                            System Admin<br>
                                            <br>
                                          </font></span></p>
                                      <br>
_______________________________________________<br>
                                      Gluster-users mailing list<br>
                                      <a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
                                      <a href="http://www.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
                                    </blockquote>
                                  </div>
                                  <br>
                                </div>
                              </div>
                            </blockquote>
                            <br>
                          </div>
                        </blockquote>
                      </div>
                      <br>
                    </div>
                  </div>
                </blockquote>
                <br>
              </div>
            </blockquote>
          </div>
          <br>
        </div>
      </div>
    </blockquote>
    <br>
  </div>

</blockquote></div><br></div></div>