<div dir="ltr">Hi Vijay,<div><br></div><div>I rerun the test with gluster 3.7.11and found that the utilization still high when i use libgfapi. The write performance is also no good. </div><div><br></div><div>Below are the info and results:</div><div><br></div><div>Hypervisor host:</div><div>libvirt 1.2.17-13.el7_2.4</div><div>qemu-kvm 2.1.2-23.el7.1</div><div><br></div><div><br></div><div>WIndows VM:</div><div>Windows 2008R2</div><div>IOmeter</div><div><br></div><div>Ubuntu VM:</div><div>Ubuntu 14.04</div><div>fio 2.1.3</div><div><br></div><div><br></div><div>Windows VM (libgfapi)</div><div><br></div><div>4k random read</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">22920.76</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">0.65</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">82.02</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU Privilegde time (%)</td>
<td align="right">77.76</td>
</tr></tbody></table><br></div><div>4k random write</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">4526.39</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">3.26</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">93.61</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU Privilegde time (%)</td>
<td align="right">90.24</td>
</tr></tbody></table><br></div><div>Windows VM (fuse)</div><div><br></div><div><div>4k random read</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">14662.86</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">1.08</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">27.66</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU Privilegde time (%)</td>
<td align="right">24.45</td>
</tr></tbody></table><br></div><div>4k random write</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">16911.66</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">0.94</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">26.74</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU Privilegde time (%)</td>
<td align="right">22.64</td>
</tr></tbody></table><br></div><div>Ubuntu VM (libgfapi)</div><div><br></div><div>4k random read</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">34364</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">0.46</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">6.09</td>
</tr></tbody></table><br></div><div>4k random write</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">4531</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">3.53</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">1.2</td>
</tr></tbody></table><br></div><div>Ubuntu VM (fuse)</div><div><br></div><div>4k random read</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">17341</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">0.92</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">4.22</td>
</tr></tbody></table><br></div><div>4k random write</div><div><br></div><div><table border="0" cellpadding="0" cellspacing="0" width="294" style="border-collapse:collapse;width:220pt">
<colgroup><col width="187" style="width:140pt">
<col width="107" style="width:80pt">
</colgroup><tbody><tr height="20" style="height:15pt">
<td height="20" width="187" style="height:15pt;width:140pt">IOPS</td>
<td align="right" width="107" style="width:80pt">17611</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">Avg. response time (ms)</td>
<td align="right">0.91</td>
</tr>
<tr height="20" style="height:15pt">
<td height="20" style="height:15pt">CPU utilization total (%)</td>
<td align="right">4.65</td>
</tr></tbody></table><br></div><div>Any comments on this or things i should try?</div><div><br></div><div>Thanks.</div><div><br></div><div>Cw</div><div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 18, 2016 at 10:00 PM, Vijay Bellur <span dir="ltr"><<a href="mailto:vbellur@redhat.com" target="_blank">vbellur@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 04/18/2016 06:50 AM, qingwei wei wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
I posted this on gluster-users mailing list but got no response so far.<br>
<br>
<a href="http://www.gluster.org/pipermail/gluster-users/2016-March/025727.html" rel="noreferrer" target="_blank">http://www.gluster.org/pipermail/gluster-users/2016-March/025727.html</a><br>
<br>
Basically, Windows guest OS CPU consumption is very high when i run some<br>
IO workload. This si not happening for NFS or fuse mount gluster volume.<br>
Anyone here experience the same also?<br>
<br>
</blockquote>
<br></span>
Is a similar behavior being observed with Linux guests too?<br>
<br>
I also observe that the setup consists of 3.7.8. I would recommend using 3.7.9 or higher as 3.7.8 had a performance regression problem.<br>
<br>
Some windows guests also need tuning to keep CPU consumption under control. One such article can be found in the proxmox wiki [1].<br>
<br>
HTH,<br>
Vijay<br>
<br>
[1] <a href="http://pve.proxmox.com/wiki/Performance_Tweaks#Windows" rel="noreferrer" target="_blank">http://pve.proxmox.com/wiki/Performance_Tweaks#Windows</a><br>
</blockquote></div><br></div>