<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<div class="moz-cite-prefix">On 02/18/2015 11:43 AM,
<a class="moz-txt-link-abbreviated" href="mailto:tbenzvi@3vgeomatics.com">tbenzvi@3vgeomatics.com</a> wrote:<br>
</div>
<blockquote
cite="mid:20150218124308.b2b02683b6fce9ed61e10e2e9bfae354.a3d1725a9b.mailapi@email04.secureserver.net"
type="cite">
<div>Hi Olav,</div>
<div><br class="Apple-interchange-newline">
I have a hunch that our problem was caused by improper
unmounting of the gluster volume, and have since found that the
proper order should be: kill all jobs using volume -> unmount
volume on clients -> gluster volume stop -> stop gluster
service (if necessary)</div>
<div> </div>
<div>In my case, I wrote a Python script to find duplicate files
on the mounted volume, then delete the corresponding link files
on the bricks (making sure to also delete files in the
.glusterfs directory)</div>
<div> </div>
<div>However, your find command was also suggested to me and I
think it's a simpler solution. I believe removing all link files
(even ones that are not causing duplicates) is fine since the
next file access gluster will do a lookup on all bricks and
recreate any link files if necessary. Hopefully a gluster expert
can chime in on this point as I'm not completely sure.</div>
</blockquote>
<br>
You are correct.<br>
<br>
<blockquote
cite="mid:20150218124308.b2b02683b6fce9ed61e10e2e9bfae354.a3d1725a9b.mailapi@email04.secureserver.net"
type="cite">
<div> </div>
<div>Keep in mind your setup is somewhat different than mine as I
have only 5 bricks with no replication.</div>
<div> </div>
<div>Regards,</div>
<div>Tom</div>
<div> </div>
<blockquote class="threadBlockQuote" style="border-left: 2px solid
#C2C2C2; padding-left: 3px; margin-left: 4px;">---------
Original Message ---------
<div>Subject: Re: [Gluster-users] Hundreds of duplicate files<br>
From: "Olav Peeters" <a class="moz-txt-link-rfc2396E" href="mailto:opeeters@gmail.com"><opeeters@gmail.com></a><br>
Date: 2/18/15 10:52 am<br>
To: <a class="moz-txt-link-abbreviated" href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a>, <a class="moz-txt-link-abbreviated" href="mailto:tbenzvi@3vgeomatics.com">tbenzvi@3vgeomatics.com</a><br>
<br>
<div class="moz-cite-prefix">Hi all,<br>
I'm have this problem after upgrading from 3.5.3 to 3.6.2.<br>
At the moment I am still waiting for a heal to finish (on a
31TB volume with 42 bricks, replicated over three nodes).<br>
<br>
Tom,<br>
how did you remove the duplicates?<br>
with 42 bricks I will not be able to do this manually..<br>
Did a:<br>
find $brick_root -type f -size 0 -perm 1000 -exec /bin/rm {}
\;<br>
work for you?<br>
<br>
Should this type of thing ideally not be checked and mended
by a heal?<br>
<br>
Does anyone have an idea yet how this happens in the first
place? Can it be connected to upgrading?<br>
<br>
Cheers,<br>
Olav<br>
<pre class="moz-signature"> </pre>
On 01/01/15 03:07, <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:tbenzvi@3vgeomatics.com">tbenzvi@3vgeomatics.com</a>
wrote:</div>
<blockquote
cite="mid:20141231190720.b2b02683b6fce9ed61e10e2e9bfae354.1100adfcd4.mailapi@email04.secureserver.net">
<div>No, the files can be read on a newly mounted client! I
went ahead and deleted all of the link files associated
with these duplicates, and then remounted the volume. The
problem is fixed!</div>
<div>Thanks again for the help, Joe and Vijay.</div>
<div> </div>
<div>Tom</div>
<div> </div>
<blockquote class="threadBlockQuote" style="border-left: 2px
solid #C2C2C2; padding-left: 3px; margin-left: 4px;">---------
Original Message ---------
<div>Subject: Re: [Gluster-users] Hundreds of duplicate
files<br>
From: "Vijay Bellur" <a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:vbellur@redhat.com"><vbellur@redhat.com></a><br>
Date: 12/28/14 3:23 am<br>
To: <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:tbenzvi@3vgeomatics.com">tbenzvi@3vgeomatics.com</a>,
<a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
<br>
On 12/28/2014 01:20 PM, <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:tbenzvi@3vgeomatics.com">tbenzvi@3vgeomatics.com</a>
wrote:<br>
> Hi Vijay,<br>
> Yes the files are still readable from the
.glusterfs path.<br>
> There is no explicit error. However, trying to read
a text file in<br>
> python simply gives me null characters:<br>
><br>
> >>> open('ott_mf_itab').readlines()<br>
>
['\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']<br>
><br>
> And reading binary files does the same<br>
><br>
<br>
Is this behavior seen with a freshly mounted client too?<br>
<br>
-Vijay<br>
<br>
> --------- Original Message ---------<br>
> Subject: Re: [Gluster-users] Hundreds of duplicate
files<br>
> From: "Vijay Bellur" <a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:vbellur@redhat.com"><vbellur@redhat.com></a><br>
> Date: 12/27/14 9:57 pm<br>
> To: <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:tbenzvi@3vgeomatics.com">tbenzvi@3vgeomatics.com</a>,
<a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
><br>
> On 12/28/2014 10:13 AM, <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:tbenzvi@3vgeomatics.com">tbenzvi@3vgeomatics.com</a>
wrote:<br>
> > Thanks Joe, I've read your blog post as well
as your post<br>
> regarding the<br>
> > .glusterfs directory.<br>
> > I found some unneeded duplicate files which
were not being read<br>
> > properly. I then deleted the link file from
the brick. This always<br>
> > removes the duplicate file from the listing,
but the file does not<br>
> > always become readable. If I also delete the
associated file in the<br>
> > .glusterfs directory on that brick, then some
more files become<br>
> > readable. However this solution still doesn't
work for all files.<br>
> > I know the file on the brick is not corrupt as
it can be read<br>
> directly<br>
> > from the brick directory.<br>
><br>
> For files that are not readable from the client,
can you check if the<br>
> file is readable from the .glusterfs/ path?<br>
><br>
> What is the specific error that is seen while
trying to read one such<br>
> file from the client?<br>
><br>
> Thanks,<br>
> Vijay<br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-users mailing list<br>
> <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a><br>
></div>
</blockquote>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Gluster-users mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
</div>
</blockquote>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://www.gluster.org/mailman/listinfo/gluster-users">http://www.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br>
</body>
</html>