<p dir="ltr"></p>
<p dir="ltr">Sent from one plus one<br>
On Jun 17, 2015 10:28 PM, "Raghavendra Talur" <<a href="mailto:rtalur@redhat.com">rtalur@redhat.com</a>> wrote:<br>
><br>
><br>
> On Jun 17, 2015 17:18, Atin Mukherjee <<a href="mailto:amukherj@redhat.com">amukherj@redhat.com</a>> wrote:<br>
> ><br>
> ><br>
> ><br>
> > On 06/17/2015 04:26 PM, Raghavendra Talur wrote:<br>
> > > Hi,<br>
> > ><br>
> > ><br>
> > > MSV Bhat and I had presented in Gluster Design Summit some ideas about<br>
> > > improving our testing infrastructure.<br>
> > ><br>
> > > Here is the link to the slides: <a href="http://redhat.slides.com/rtalur/distaf#">http://redhat.slides.com/rtalur/distaf#</a><br>
> > ><br>
> > > Here are the same suggestions,<br>
> > ><br>
> > > 1. *A .t file for a bug*<br>
> > > When a community user discovers a bug in Gluster, they contact us over<br>
> > > irc or email and eventually end up filling a bug in bugzilla.<br>
> > > Many times it so happens that we find a bug which we don't know the<br>
> > > fix for OR not a bug in our module and also end up filling a bug in<br>
> > > bugzilla.<br>
> > ><br>
> > > If we could rather write a .t test to reproduce the bug and add it to<br>
> > > say /tests/bug/yet-to-be-fixed/ folder in gluster repo it would be<br>
> > > more helpful. As part of bug-triage we could try doing the same for bugs<br>
> > > filed by community users.<br>
> > ><br>
> > > *What do we get?*<br>
> > ><br>
> > > a. very easy for a new developer to pick up that bug and fix it.<br>
> > > If .t passes then the bug is fixed.<br>
> > ><br>
> > > b. The regression on daily patch sets would skip this folder; but on a<br>
> > > nightly basis we could run a test on this folder to see if any of these<br>
> > > tests got fixed while we were fixing some other tests. Yay!<br>
> > Attaching a reproducer in the form of .t might be difficult, specially<br>
> > for the race conditions. It might pass pre and post fix as well. So it<br>
> > *should not* be a must criteria to have .t file.<br>
><br>
> Agreed, it is only a good to have thing. For easy fix and/or easy reproducible bugs.<br>
> > ><br>
> > ><br>
> > ><br>
> > > 2. *New gerrit/review work flow*<br>
> > ><br>
> > > Our gerrit setup currently has a 2 hour average for regression run.<br>
> > > Due to long queue of commits the round about time is around 4-6 hours.<br>
> > ><br>
> > > Kaushal has proposed on how to reduce round about time more in this<br>
> > > thread <a href="http://www.spinics.net/lists/gluster-devel/msg15798.html">http://www.spinics.net/lists/gluster-devel/msg15798.html</a>.<br>
> > ><br>
> > ><br>
> > > 3. *Make sure tests can be done in docker and run in parallel*<br>
> > ><br>
> > > To reduce time for one test run from 2 hours we can look at running<br>
> > > tests in parallel. I did a prototype and got test time down to 40 mins<br>
> > > on a 16 GB RAM and 4 core VM.<br>
> > ><br>
> > > Current blocked at :<br>
> > > Some of the tests fail in docker while they pass in a VM.<br>
> > > Note that it is .t failing, Gluster works fine in docker.<br>
> > > Need some help on this. More on this in a mail I will be sending later<br>
> > > today at gluster-devel.<br>
> > ><br>
> > ><br>
> > > *what do we get?*<br>
> > > Running 4 docker containers on our Laptops itself can reduce time<br>
> > > taken by test runs down to 90 mins. Running them on powerful machines,<br>
> > > it is down to 40 mins as seen in the prototype.<br>
> > How about NetBSD, yesterday Niels point out to me that there is no<br>
> > docker service for NetBSD.<br>
><br>
> There are two take aways here,<br>
> 1. Reducing regression time on Jenkins<br>
> 2. Reducing regression time on our laptops.<br>
><br>
> For 1 we will still have NETBSD bottleneck. Haven't thought of how to avoid that.<br>
><br>
> At least getting 2 is still a win.<br>
I am bit ambitious for (1 && 2) ;)<br>
> > ><br>
> > ><br>
> > > 4. *Test definitions for every .t*<br>
> > ><br>
> > > May be the time has come to upgrade our test infra to have tests with<br>
> > > test definitions. Every .t file could have a corresponding .def file<br>
> > > which is<br>
> > > A JSON/YAML/XML config<br>
> > > Defines the requirements of test<br>
> > > Type of volume<br>
> > > Special knowledge of brick size required?<br>
> > > Which repo source folders should trigger this test<br>
> > > Running time<br>
> > > Test RUN level<br>
> > ><br>
> > > *what do we get?*<br>
> > > a. Run a partial set of tests on a commit based on git log and test<br>
> > > definitions and run complete regression as nightly.<br>
> > > b. Order test run based on run times. This combined with fail on first<br>
> > > test setting we have, we will fail as early as possible.<br>
> > > c. Order tests based on functionality level, which means a mount.t basic<br>
> > > test should run before a complex DHT test that makes use of FUSE mount.<br>
> > > Again, this will help us to fail as early as possible in failure scenarios.<br>
> > > d. With knowledge of type of volume required and number of bricks<br>
> > > required, we can re-use volumes that are created for subsequent tests.<br>
> > > Even the cleanup() function we have takes time. DiSTAF already has a<br>
> > > function equivalent to use_existing_else_create_new.<br>
> > ><br>
> > ><br>
> > > 5. *Testing GFAPI*<br>
> > > We don't have a good test framework for gfapi as of today.<br>
> > ><br>
> > > However, with the recent design proposal at<br>
> > > <a href="https://docs.google.com/document/d/1yuRLRbdccx_0V0UDAxqWbz4g983q5inuINHgM1YO040/edit?usp=sharing">https://docs.google.com/document/d/1yuRLRbdccx_0V0UDAxqWbz4g983q5inuINHgM1YO040/edit?usp=sharing</a><br>
> > ><br>
> > ><br>
> > > and<br>
> > ><br>
> > > Craig Cabrey from Facebook developing a set of coreutils using<br>
> > > GFAPI as mentioned here<br>
> > > <a href="http://www.spinics.net/lists/gluster-devel/msg15753.html">http://www.spinics.net/lists/gluster-devel/msg15753.html</a><br>
> > ><br>
> > > I guess we have it well covered :)<br>
> > ><br>
> > ><br>
> > > Reviews and suggestions welcome!<br>
> > ><br>
> > > Thanks,<br>
> > > Raghavendra Talur<br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > ><br>
> > > _______________________________________________<br>
> > > Gluster-devel mailing list<br>
> > > <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
> > > <a href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
> ><br>
> > --<br>
> > ~Atin<br>
> _______________________________________________<br>
> Gluster-devel mailing list<br>
> <a href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a><br>
> <a href="http://www.gluster.org/mailman/listinfo/gluster-devel">http://www.gluster.org/mailman/listinfo/gluster-devel</a><br>
</p>