<p dir="ltr"></p>
<p dir="ltr">-Atin<br>
Sent from one plus one<br>
On 05-Mar-2016 11:46 am, "Ajil Abraham" <<a href="mailto:ajil95.abraham@gmail.com">ajil95.abraham@gmail.com</a>> wrote:<br>
><br>
> Thanks for all the support. After handling the input validation in my code, Glusterd no longer crashes. I am still waiting for clearance from my superior to pass on all the details. Expecting him to revert by this Sunday.<br>
Great to know that and we appreciate your contribution, if you happen to find any issues feel free to send patches :) <br>
><br>
> - Ajil<br>
><br>
> On Fri, Mar 4, 2016 at 10:20 AM, Joseph Fernandes <<a href="mailto:josferna@redhat.com">josferna@redhat.com</a>> wrote:<br>
>><br>
>> Well that may not be completely correct !<br>
>><br>
>> Its "gluster volume status all", unlike volume maintenance operation which are rare.<br>
>><br>
>> Status can be issued multiple times in a day or might be put in a script/cron-job to check the health of the<br>
>> cluster.<br>
>> But anyways the fix is ready as the bug says.<br>
>><br>
>> Crash is what we need to worry about.<br>
>><br>
>> ~Joe<br>
>><br>
>> ----- Original Message -----<br>
>> > From: "Atin Mukherjee" <<a href="mailto:amukherj@redhat.com">amukherj@redhat.com</a>><br>
>> > To: "Joseph Fernandes" <<a href="mailto:josferna@redhat.com">josferna@redhat.com</a>>, "Atin Mukherjee" <<a href="mailto:atin.mukherjee83@gmail.com">atin.mukherjee83@gmail.com</a>><br>
>> > Cc: "Gluster Devel" <<a href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a>>, "Ajil Abraham" <<a href="mailto:ajil95.abraham@gmail.com">ajil95.abraham@gmail.com</a>><br>
>> > Sent: Friday, March 4, 2016 9:37:43 AM<br>
>> > Subject: Re: [Gluster-devel] glusterd crashing<br>
>> ><br>
>> ><br>
>> ><br>
>> > On 03/04/2016 07:10 AM, Joseph Fernandes wrote:<br>
>> > > Might be this bug can give some context on the mem-leak (fix recently<br>
>> > > merged on master but not on 3.7.x)<br>
>> > ><br>
>> > > <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1287517">https://bugzilla.redhat.com/show_bug.cgi?id=1287517</a><br>
>> > Yes, this is what we'd be fixing in 3.7.x too, but if you refer to [1]<br>
>> > the hike is seen when a command is run in a loop which is typically not<br>
>> > a use case in any production setup.<br>
>> ><br>
>> > [1] <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1287517#c15">https://bugzilla.redhat.com/show_bug.cgi?id=1287517#c15</a><br>
>> > ><br>
>> > > ~Joe<br>
>> > ><br>
>> > ><br>
>> > > ----- Original Message -----<br>
>> > >> From: "Atin Mukherjee" <<a href="mailto:atin.mukherjee83@gmail.com">atin.mukherjee83@gmail.com</a>><br>
>> > >> To: "Joseph Fernandes" <<a href="mailto:josferna@redhat.com">josferna@redhat.com</a>><br>
>> > >> Cc: "Gluster Devel" <<a href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a>>, "Ajil Abraham"<br>
>> > >> <<a href="mailto:ajil95.abraham@gmail.com">ajil95.abraham@gmail.com</a>><br>
>> > >> Sent: Friday, March 4, 2016 7:01:54 AM<br>
>> > >> Subject: Re: [Gluster-devel] glusterd crashing<br>
>> > >><br>
>> > >> -Atin<br>
>> > >> Sent from one plus one<br>
>> > >> On 04-Mar-2016 6:12 am, "Joseph Fernandes" <<a href="mailto:josferna@redhat.com">josferna@redhat.com</a>> wrote:<br>
>> > >>><br>
>> > >>> Hi Ajil,<br>
>> > >>><br>
>> > >>> Well few things,<br>
>> > >>><br>
>> > >>> 1. Whenever you see a crash its better to send across the Backtrace(BT)<br>
>> > >> using gdb and attach the log files (or share it via some cloud drive)<br>
>> > >>><br>
>> > >>> 2. About the memory leak, What kind of tools are you using for profiling<br>
>> > >> memory, valgrind ? if so please attach the valgrind reports.<br>
>> > >>> $> glusterd --xlator-option *.run-with-valgrind=yes<br>
>> > >>><br>
>> > >>> 3. Well I am not sure if glusterd uses any of the mempools as we do in<br>
>> > >> client and brick processes, Atin can shed some light on this.<br>
>> > >>> Well In that case you can used the statedump mechanism check for<br>
>> > >> mem-leaks check the glusterfs/doc/debugging/<a href="http://statedump.md">statedump.md</a><br>
>> > >> GlusterD does use mempool and it has infra for capturing statedump as<br>
>> > >> well.<br>
>> > >> I am aware of few bytes of memory leaks in few paths which is really not a<br>
>> > >> huge concern but it shouldn't crash.<br>
>> > >>><br>
>> > >>> Hope this helps<br>
>> > >>><br>
>> > >>> ~Joe<br>
>> > >>><br>
>> > >>><br>
>> > >>> ----- Original Message -----<br>
>> > >>>> From: "Ajil Abraham" <<a href="mailto:ajil95.abraham@gmail.com">ajil95.abraham@gmail.com</a>><br>
>> > >>>> To: "Atin Mukherjee" <<a href="mailto:atin.mukherjee83@gmail.com">atin.mukherjee83@gmail.com</a>><br>
>> > >>>> Cc: "Gluster Devel" <<a href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a>><br>
>> > >>>> Sent: Thursday, March 3, 2016 10:48:56 PM<br>
>> > >>>> Subject: Re: [Gluster-devel] glusterd crashing<br>
>> > >>>><br>
>> > >>>> Hi Atin,<br>
>> > >>>><br>
>> > >>>> The inputs I use are as per the requirements of a project I am working<br>
>> > >> on for<br>
>> > >>>> one of the large finance institutions in Dubai. I will try to handle the<br>
>> > >>>> input validation within my code. I uncovered some of the issues while<br>
>> > >> doing<br>
>> > >>>> a thorough testing of my code.<br>
>> > >>>><br>
>> > >>>> I tried with 3.7.6 and also my own build from master branch. I will<br>
>> > >> check<br>
>> > >>>> with my superiors before sending you backtrace and other details. So<br>
>> > >> far, I<br>
>> > >>>> have seen memory leak in 100s of KBs.<br>
>> > >>>><br>
>> > >>>> -Ajil<br>
>> > >>>><br>
>> > >>>><br>
>> > >>>> On Thu, Mar 3, 2016 at 10:17 PM, Atin Mukherjee <<br>
>> > >> <a href="mailto:atin.mukherjee83@gmail.com">atin.mukherjee83@gmail.com</a><br>
>> > >>>>> wrote:<br>
>> > >>>><br>
>> > >>>><br>
>> > >>>><br>
>> > >>>><br>
>> > >>>> Hi Ajil,<br>
>> > >>>><br>
>> > >>>> Its good to see that you are doing a thorough testing gluster. From<br>
>> > >> your mail<br>
>> > >>>> it looks like your automation focuses on mostly negative tests. I need<br>
>> > >> few<br>
>> > >>>> additional details to get to know whether they are known:<br>
>> > >>>><br>
>> > >>>> 1. Version of gluster<br>
>> > >>>> 2. Backtrace of the crash along with reproducer<br>
>> > >>>> 3. Amount of memory leak in terms of bytes/KB/MB?? Have you already<br>
>> > >>>> identified them?<br>
>> > >>>><br>
>> > >>>> -Atin<br>
>> > >>>> Sent from one plus one<br>
>> > >>>> On 03-Mar-2016 10:01 pm, "Ajil Abraham" < <a href="mailto:ajil95.abraham@gmail.com">ajil95.abraham@gmail.com</a> ><br>
>> > >> wrote:<br>
>> > >>>><br>
>> > >>>><br>
>> > >>>><br>
>> > >>>> For my project, I am trying to do some automation using glusterd. It is<br>
>> > >> very<br>
>> > >>>> frustrating to see it crashing frequently. Looks like input validation<br>
>> > >> is<br>
>> > >>>> the culprit. I also see lot of buffer overflow and memory leak issues.<br>
>> > >>>> Making a note of these and will try to fix them. Surprised to see such<br>
>> > >> basic<br>
>> > >>>> issues still existing in Gluster.<br>
>> > >>>><br>
>> > >>>> -Ajil<br>
>> > >>>><br>
>> > >>>> _______________________________________________<br>
>> > >>>> Gluster-devel mailing list<br>
>> > >>>> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
>> > >>>> <a href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
>> > >>>><br>
>> > >>>><br>
>> > >>>> _______________________________________________<br>
>> > >>>> Gluster-devel mailing list<br>
>> > >>>> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
>> > >>>> <a href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
>> > >><br>
>> > > _______________________________________________<br>
>> > > Gluster-devel mailing list<br>
>> > > <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
>> > > <a href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
>> > ><br>
>> ><br>
><br>
><br>
><br>
> _______________________________________________<br>
> Gluster-devel mailing list<br>
> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
</p>