<div dir="ltr"><div>This looks to be an SSL issue. GlusterD is not coming up after enabling mgmt encryption with the following error log:<br><br>[2016-08-09 11:37:08.854932] E [socket.c:4107:socket_init] 0-socket.management: could not load our cert<br><br></div>I believe the prerequisite of this test to have all the certificates to be available in the right location which is missing here, is this a new slave by any chance where the certs are yet to be created?<br><br>P.S : I am not an expert on this part, but Jeff/Kaushal/Mohit can chime in if I am wrong.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 9, 2016 at 1:37 PM, Ashish Pandey <span dir="ltr"><<a href="mailto:aspandey@redhat.com" target="_blank">aspandey@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div style="font-family:times new roman,new york,times,serif;font-size:12pt;color:#000000">I could see that this test is failing on my laptop with and without my unlink patch.<br>So, I don't think unlink patch is responsible for failure.<br><br><br><br><br><hr><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt"><b>From: </b>"Pranith Kumar Karampuri" <<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>><br><b>To: </b><a href="mailto:jenkins@build.gluster.org" target="_blank">jenkins@build.gluster.org</a><br><b>Cc: </b><a href="mailto:maintainers@gluster.org" target="_blank">maintainers@gluster.org</a>, "Ashish Pandey" <<a href="mailto:aspandey@redhat.com" target="_blank">aspandey@redhat.com</a>><br><b>Sent: </b>Tuesday, August 9, 2016 11:05:58 AM<br><b>Subject: </b>Re: [Gluster-Maintainers] Build failed in Jenkins: regression-test-burn-in #1481<div><div class="h5"><br><br><div dir="ltr">Ashish could you take a look at this failure to see if it is because of the change in posix xlator we did for unlink dir?<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 8, 2016 at 7:08 PM, <span dir="ltr"><<a href="mailto:jenkins@build.gluster.org" target="_blank">jenkins@build.gluster.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">See <<a href="http://build.gluster.org/job/regression-test-burn-in/1481/changes" rel="noreferrer" target="_blank">http://build.gluster.org/job/<wbr>regression-test-burn-in/1481/<wbr>changes</a>><br>
<br>
Changes:<br>
<br>
[Pranith Kumar K] posix: Do not move and recreate .glusterfs/unlink directory<br>
<br>
------------------------------<wbr>------------<br>
[...truncated 9683 lines...]<br>
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]<br>
sed: read error on /var/run/gluster/: Is a directory<br>
rm: cannot remove `/var/run/gluster/': Is a directory<br>
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]<br>
sed: read error on /var/run/gluster/: Is a directory<br>
rm: cannot remove `/var/run/gluster/': Is a directory<br>
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]<br>
sed: read error on /var/run/gluster/: Is a directory<br>
rm: cannot remove `/var/run/gluster/': Is a directory<br>
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]<br>
sed: read error on /var/run/gluster/: Is a directory<br>
rm: cannot remove `/var/run/gluster/': Is a directory<br>
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]<br>
sed: read error on /var/run/gluster/: Is a directory<br>
rm: cannot remove `/var/run/gluster/': Is a directory<br>
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]<br>
sed: read error on /var/run/gluster/: Is a directory<br>
rm: cannot remove `/var/run/gluster/': Is a directory<br>
kill: usage: kill [-s sigspec | -n signum | -sigspec] pid | jobspec ... or kill -l [sigspec]<br>
sed: read error on /var/run/gluster/: Is a directory<br>
rm: cannot remove `/var/run/gluster/': Is a directory<br>
./tests/bugs/cli/bug-1320388.t ..<br>
1..11<br>
not ok 1 , LINENUM:11<br>
FAILED COMMAND: glusterd<br>
not ok 2 , LINENUM:12<br>
FAILED COMMAND: pidof glusterd<br>
ok 3, LINENUM:13<br>
ok 4, LINENUM:14<br>
not ok 5 , LINENUM:15<br>
FAILED COMMAND: glusterfs --entry-timeout=0 --attribute-timeout=0 -s <a href="http://slave26.cloud.gluster.org" rel="noreferrer" target="_blank">slave26.cloud.gluster.org</a> --volfile-id patchy /mnt/glusterfs/0<br>
not ok 6 Got "" instead of "^6$", LINENUM:16<br>
FAILED COMMAND: ^6$ ec_child_up_count patchy 0<br>
not ok 7 , LINENUM:18<br>
FAILED COMMAND: kill_brick patchy <a href="http://slave26.cloud.gluster.org" rel="noreferrer" target="_blank">slave26.cloud.gluster.org</a> /d/backends/patchy5<br>
not ok 8 Got "" instead of "^5$", LINENUM:20<br>
FAILED COMMAND: ^5$ get_pending_heal_count patchy<br>
not ok 9 Got "" instead of "^6$", LINENUM:22<br>
FAILED COMMAND: ^6$ ec_child_up_count patchy 0<br>
ok 10, LINENUM:23<br>
not ok 11 Got "" instead of "^0$", LINENUM:24<br>
FAILED COMMAND: ^0$ get_pending_heal_count patchy<br>
Failed 8/11 subtests<br>
<br>
Test Summary Report<br>
-------------------<br>
./tests/bugs/cli/bug-1320388.t (Wstat: 0 Tests: 11 Failed: 8)<br>
Failed tests: 1-2, 5-9, 11<br>
Files=1, Tests=11, 201 wallclock secs ( 0.02 usr 0.00 sys + 7.79 cusr 2.70 csys = 10.51 CPU)<br>
Result: FAIL<br>
End of test ./tests/bugs/cli/bug-1320388.t<br>
==============================<wbr>==============================<wbr>====================<br>
<br>
<br>
Run complete<br>
==============================<wbr>==============================<wbr>====================<br>
Number of tests found: 158<br>
Number of tests selected for run based on pattern: 158<br>
Number of tests skipped as they were marked bad: 6<br>
Number of tests skipped because of known_issues: 1<br>
Number of tests that were run: 151<br>
<br>
1 test(s) failed<br>
./tests/bugs/cli/bug-1320388.t<br>
<br>
0 test(s) generated core<br>
<br>
<br>
Tests ordered by time taken, slowest to fastest:<br>
==============================<wbr>==============================<wbr>====================<br>
./tests/basic/afr/split-brain-<wbr>favorite-child-policy.t - 605 second<br>
./tests/basic/ec/ec-12-4.t - 335 second<br>
./tests/basic/ec/ec-<wbr>background-heals.t - 303 second<br>
./tests/bugs/cli/bug-1320388.<wbr>t - 201 second<br>
./tests/basic/ec/ec-7-3.t - 200 second<br>
./tests/basic/afr/entry-self-<wbr>heal.t - 176 second<br>
./tests/basic/ec/ec-6-2.t - 173 second<br>
./tests/basic/tier/tier-heald.<wbr>t - 166 second<br>
./tests/basic/ec/ec-5-2.t - 147 second<br>
./tests/basic/ec/ec-5-1.t - 145 second<br>
./tests/basic/glusterd/heald.<wbr>t - 138 second<br>
./tests/basic/afr/self-heal.t - 137 second<br>
./tests/basic/tier/legacy-<wbr>many.t - 124 second<br>
./tests/basic/tier/tier.t - 120 second<br>
./tests/basic/ec/ec-4-1.t - 120 second<br>
./tests/basic/ec/ec-root-heal.<wbr>t - 112 second<br>
./tests/basic/afr/granular-<wbr>esh/conservative-merge.t - 112 second<br>
./tests/basic/afr/granular-<wbr>esh/add-brick.t - 98 second<br>
./tests/basic/afr/granular-<wbr>esh/granular-esh.t - 97 second<br>
./tests/basic/afr/add-brick-<wbr>self-heal.t - 97 second<br>
./tests/basic/ec/ec-3-1.t - 93 second<br>
./tests/basic/ec/ec-new-entry.<wbr>t - 91 second<br>
./tests/basic/afr/split-brain-<wbr>heal-info.t - 77 second<br>
./tests/basic/afr/self-heald.<wbr>t - 77 second<br>
./tests/basic/afr/split-brain-<wbr>healing.t - 75 second<br>
./tests/basic/afr/metadata-<wbr>self-heal.t - 69 second<br>
./tests/basic/ec/self-heal.t - 67 second<br>
./tests/basic/quota.t - 66 second<br>
./tests/basic/tier/new-tier-<wbr>cmds.t - 63 second<br>
./tests/basic/tier/tierd_<wbr>check.t - 62 second<br>
./tests/basic/tier/frequency-<wbr>counters.t - 62 second<br>
./tests/basic/afr/sparse-file-<wbr>self-heal.t - 58 second<br>
./tests/basic/volume-snapshot-<wbr>clone.t - 53 second<br>
./tests/basic/uss.t - 47 second<br>
./tests/basic/tier/fops-<wbr>during-migration-pause.t - 47 second<br>
./tests/basic/ec/ec-notify.t - 41 second<br>
./tests/basic/mount-nfs-auth.<wbr>t - 40 second<br>
./tests/basic/ec/ec-readdir.t - 40 second<br>
./tests/basic/tier/locked_<wbr>file_migration.t - 39 second<br>
./tests/basic/volume-snapshot.<wbr>t - 38 second<br>
./tests/basic/ec/ec-anonymous-<wbr>fd.t - 37 second<br>
./tests/basic/afr/arbiter.t - 35 second<br>
./tests/basic/tier/unlink-<wbr>during-migration.t - 34 second<br>
./tests/basic/jbr/jbr.t - 34 second<br>
./tests/basic/ec/ec.t - 34 second<br>
./tests/basic/afr/data-self-<wbr>heal.t - 32 second<br>
./tests/basic/afr/quorum.t - 28 second<br>
./tests/basic/quota-ancestry-<wbr>building.t - 27 second<br>
./tests/basic/afr/durability-<wbr>off.t - 24 second<br>
./tests/basic/afr/arbiter-add-<wbr>brick.t - 24 second<br>
./tests/basic/tier/file_with_<wbr>spaces.t - 23 second<br>
./tests/basic/afr/heal-quota.<wbr>t - 23 second<br>
./tests/bitrot/bug-1294786.t - 22 second<br>
./tests/basic/mgmt_v3-locks.t - 21 second<br>
./tests/basic/geo-replication/<wbr>marker-xattrs.t - 21 second<br>
./tests/basic/ec/quota.t - 21 second<br>
./tests/basic/afr/gfid-self-<wbr>heal.t - 21 second<br>
./tests/bugs/bitrot/bug-<wbr>1227996.t - 20 second<br>
./tests/basic/tier/readdir-<wbr>during-migration.t - 20 second<br>
./tests/basic/op_errnos.t - 20 second<br>
./tests/basic/glusterd/<wbr>volfile_server_switch.t - 20 second<br>
./tests/bugs/cli/bug-1113476.<wbr>t - 19 second<br>
./tests/basic/0symbol-check.t - 19 second<br>
./tests/bugs/changelog/bug-<wbr>1225542.t - 18 second<br>
./tests/bugs/bitrot/bug-<wbr>1245981.t - 18 second<br>
./tests/basic/ec/statedump.t - 18 second<br>
./tests/basic/afr/granular-<wbr>esh/replace-brick.t - 18 second<br>
./tests/bugs/changelog/bug-<wbr>1321955.t - 16 second<br>
./tests/basic/afr/split-brain-<wbr>resolution.t - 16 second<br>
./tests/basic/afr/replace-<wbr>brick-self-heal.t - 16 second<br>
./tests/bugs/cli/bug-1077682.<wbr>t - 15 second<br>
./tests/bugs/changelog/bug-<wbr>1211327.t - 15 second<br>
./tests/bugs/bitrot/bug-<wbr>1228680.t - 15 second<br>
./tests/bitrot/br-state-check.<wbr>t - 15 second<br>
./tests/basic/afr/resolve.t - 15 second<br>
./tests/bugs/cli/bug-1047416.<wbr>t - 14 second<br>
./tests/basic/volume-status.t - 14 second<br>
./tests/basic/glusterd/<wbr>disperse-create.t - 14 second<br>
./tests/basic/glusterd/<wbr>arbiter-volume-probe.t - 14 second<br>
./tests/basic/bd.t - 14 second<br>
./tests/bugs/bug-1110262.t - 13 second<br>
./tests/bugs/bitrot/bug-<wbr>1288490.t - 13 second<br>
./tests/bugs/bitrot/1209751-<wbr>bitrot-scrub-tunable-reset.t - 13 second<br>
./tests/basic/afr/client-side-<wbr>heal.t - 13 second<br>
./tests/basic/tier/ctr-rename-<wbr>overwrite.t - 12 second<br>
./tests/basic/rpc-coverage.t - 12 second<br>
./tests/basic/pump.t - 12 second<br>
./tests/basic/nufa.t - 12 second<br>
./tests/basic/cdc.t - 12 second<br>
./tests/bugs/cli/bug-1030580.<wbr>t - 11 second<br>
./tests/basic/volume.t - 11 second<br>
./tests/basic/quota-anon-fd-<wbr>nfs.t - 11 second<br>
./tests/basic/inode-quota-<wbr>enforcing.t - 11 second<br>
./tests/basic/afr/stale-file-<wbr>lookup.t - 11 second<br>
./tests/basic/afr/root-squash-<wbr>self-heal.t - 11 second<br>
./tests/bugs/cli/bug-1087487.<wbr>t - 10 second<br>
./tests/bugs/changelog/bug-<wbr>1208470.t - 10 second<br>
./tests/bugs/bitrot/1209752-<wbr>volume-status-should-show-<wbr>bitrot-scrub-info.t - 10 second<br>
./tests/bugs/bitrot/1207029-<wbr>bitrot-daemon-should-start-on-<wbr>valid-node.t - 10 second<br>
./tests/bitrot/bug-1244613.t - 10 second<br>
./tests/bitrot/bug-1207627-<wbr>bitrot-scrub-status.t - 10 second<br>
./tests/basic/stats-dump.t - 10 second<br>
./tests/basic/quota-nfs.t - 10 second<br>
./tests/basic/glusterd/<wbr>arbiter-volume.t - 10 second<br>
./tests/basic/ec/ec-read-<wbr>policy.t - 10 second<br>
./tests/basic/afr/read-subvol-<wbr>data.t - 10 second<br>
./tests/basic/afr/arbiter-<wbr>mount.t - 10 second<br>
./tests/bugs/bitrot/1209818-<wbr>vol-info-show-scrub-process-<wbr>properly.t - 9 second<br>
./tests/bugs/access-control/<wbr>bug-958691.t - 9 second<br>
./tests/bugs/access-control/<wbr>bug-887098-gmount-crash.t - 9 second<br>
./tests/basic/mount.t - 9 second<br>
./tests/basic/meta.t - 9 second<br>
./tests/basic/gfapi/<wbr>bug1291259.t - 9 second<br>
./tests/basic/fop-sampling.t - 9 second<br>
./tests/basic/afr/read-subvol-<wbr>entry.t - 9 second<br>
./tests/bugs/cli/bug-1022905.<wbr>t - 8 second<br>
./tests/bugs/bug-1258069.t - 8 second<br>
./tests/bugs/bitrot/bug-<wbr>1229134-bitd-not-support-vol-<wbr>set.t - 8 second<br>
./tests/basic/distribute/bug-<wbr>1265677-use-readdirp.t - 8 second<br>
./tests/basic/afr/arbiter-<wbr>remove-brick.t - 8 second<br>
./tests/bitrot/br-stub.t - 7 second<br>
./tests/basic/quota-rename.t - 7 second<br>
./tests/basic/pgfid-feat.t - 7 second<br>
./tests/basic/gfapi/gfapi-dup.<wbr>t - 7 second<br>
./tests/basic/gfapi/bug-<wbr>1241104.t - 7 second<br>
./tests/basic/gfapi/anonymous_<wbr>fd.t - 7 second<br>
./tests/basic/ec/ec-internal-<wbr>xattrs.t - 7 second<br>
./tests/basic/ec/dht-rename.t - 7 second<br>
./tests/basic/distribute/<wbr>throttle-rebal.t - 7 second<br>
./tests/basic/afr/heal-info.t - 7 second<br>
./tests/basic/afr/gfid-heal.t - 7 second<br>
./tests/basic/afr/arbiter-<wbr>statfs.t - 7 second<br>
./tests/bugs/cli/bug-1004218.<wbr>t - 6 second<br>
./tests/bugs/bitrot/bug-<wbr>1210684-scrub-pause-resume-<wbr>error-handling.t - 6 second<br>
./tests/bugs/access-control/<wbr>bug-1051896.t - 6 second<br>
./tests/bitrot/bug-internal-<wbr>xattrs-check-1243391.t - 6 second<br>
./tests/bitrot/bug-1221914.t - 6 second<br>
./tests/basic/gfid-access.t - 6 second<br>
./tests/basic/gfapi/libgfapi-<wbr>fini-hang.t - 6 second<br>
./tests/basic/fops-sanity.t - 6 second<br>
./tests/basic/ec/nfs.t - 6 second<br>
./tests/basic/afr/gfid-<wbr>mismatch.t - 6 second<br>
./tests/basic/jbr/jbr-volgen.<wbr>t - 5 second<br>
./tests/basic/afr/arbiter-cli.<wbr>t - 5 second<br>
./tests/bugs/cli/bug-1047378.<wbr>t - 4 second<br>
./tests/basic/posixonly.t - 2 second<br>
./tests/basic/rpm.t - 1 second<br>
./tests/basic/netgroup_<wbr>parsing.t - 1 second<br>
./tests/basic/exports_parsing.<wbr>t - 1 second<br>
./tests/basic/gfapi/upcall-<wbr>cache-invalidate.t - 0 second<br>
./tests/basic/first-test.t - 0 second<br>
<br>
Result is 1<br>
<br>
+ RET=1<br>
++ wc -l<br>
++ ls -l /glusterfs-3048.core<br>
+ cur_count=1<br>
++ ls /glusterfs-3048.core<br>
+ cur_cores=/glusterfs-3048.core<br>
+ '[' 1 '!=' 1 ']'<br>
+ '[' 1 -ne 0 ']'<br>
+ filename=logs/glusterfs-logs-<wbr>20160808:11:54:44.tgz<br>
+ tar -czf /archives/logs/glusterfs-logs-<wbr>20160808:11:54:44.tgz /var/log/glusterfs /var/log/messages /var/log/messages-20160717 /var/log/messages-20160724 /var/log/messages-20160731 /var/log/messages-20160807<br>
tar: Removing leading `/' from member names<br>
+ echo Logs archived in <a href="http://slave26.cloud.gluster.org/logs/glusterfs-logs-20160808:11:54:44.tgz" rel="noreferrer" target="_blank">http://slave26.cloud.gluster.<wbr>org/logs/glusterfs-logs-<wbr>20160808:11:54:44.tgz</a><br>
Logs archived in <a href="http://slave26.cloud.gluster.org/logs/glusterfs-logs-20160808:11:54:44.tgz" rel="noreferrer" target="_blank">http://slave26.cloud.gluster.<wbr>org/logs/glusterfs-logs-<wbr>20160808:11:54:44.tgz</a><br>
+ case $(uname -s) in<br>
++ uname -s<br>
+ /sbin/sysctl -w kernel.core_pattern=/%e-%p.<wbr>core<br>
kernel.core_pattern = /%e-%p.core<br>
+ exit 1<br>
+ RET=1<br>
+ '[' 1 = 0 ']'<br>
+ V=-1<br>
+ VERDICT=FAILED<br>
+ '[' 0 -eq 1 ']'<br>
+ exit 1<br>
Build step 'Execute shell' marked build as failure<br>
______________________________<wbr>_________________<br>
maintainers mailing list<br>
<a href="mailto:maintainers@gluster.org" target="_blank">maintainers@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/maintainers" rel="noreferrer" target="_blank">http://www.gluster.org/<wbr>mailman/listinfo/maintainers</a><br>
</blockquote></div><br><br clear="all"><br>-- <br><div><div dir="ltr">Pranith<br></div></div>
</div>
</div></div></div><br></div></div><br>______________________________<wbr>_________________<br>
maintainers mailing list<br>
<a href="mailto:maintainers@gluster.org">maintainers@gluster.org</a><br>
<a href="http://www.gluster.org/mailman/listinfo/maintainers" rel="noreferrer" target="_blank">http://www.gluster.org/<wbr>mailman/listinfo/maintainers</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><br></div>--Atin<br></div></div>
</div>