<div dir="ltr"><div>Hi all</div><div><br></div><div>Gluster-3.7.6 in 'Quota' problem occurs in the following test case.</div><div><br></div><div>(it doesn't occur if don't enable the volume quota)</div><div><br></div><div>Volume usage mismatch occurs when using glusterfs-3.7.6 on ZFS.</div><div><br></div><div>Can you help with the following problems?</div><div><br></div><div><br></div><div>1. zfs disk pool information</div><div><br></div><div>root@server-1:~# zpool status</div><div> pool: pool</div><div> state: ONLINE</div><div> scan: none requested</div><div>config:</div><div><br></div><div><span class="" style="white-space:pre"> </span>NAME STATE READ WRITE CKSUM</div><div><span class="" style="white-space:pre"> </span>pool ONLINE 0 0 0</div><div><span class="" style="white-space:pre"> </span> pci-0000:00:10.0-scsi-0:0:1:0 ONLINE 0 0 0</div><div><span class="" style="white-space:pre"> </span> pci-0000:00:10.0-scsi-0:0:2:0 ONLINE 0 0 0</div><div><span class="" style="white-space:pre"> </span> pci-0000:00:10.0-scsi-0:0:3:0 ONLINE 0 0 0</div><div><br></div><div>errors: No known data errors</div><div><br></div><div>root@server-2:~# zpool status</div><div> pool: pool</div><div> state: ONLINE</div><div> scan: none requested</div><div>config:</div><div><br></div><div><span class="" style="white-space:pre"> </span>NAME STATE READ WRITE CKSUM</div><div><span class="" style="white-space:pre"> </span>pool ONLINE 0 0 0</div><div><span class="" style="white-space:pre"> </span> pci-0000:00:10.0-scsi-0:0:1:0 ONLINE 0 0 0</div><div><span class="" style="white-space:pre"> </span> pci-0000:00:10.0-scsi-0:0:2:0 ONLINE 0 0 0</div><div><span class="" style="white-space:pre"> </span> pci-0000:00:10.0-scsi-0:0:3:0 ONLINE 0 0 0</div><div><br></div><div>errors: No known data errors</div><div><br></div><div>2. zfs volume list information</div><div><br></div><div>root@server-1:~# zfs list</div><div>NAME USED AVAIL REFER MOUNTPOINT</div><div>pool 179K 11.3T 19K /pool</div><div>pool/tvol1 110K 11.3T 110K /pool/tvol1</div><div><br></div><div>root@server-2:~# zfs list</div><div>NAME USED AVAIL REFER MOUNTPOINT</div><div>pool 179K 11.3T 19K /pool</div><div>pool/tvol1 110K 11.3T 110K /pool/tvol1</div><div><br></div><div>3. gluster volume information</div><div><br></div><div>root@server-1:~# gluster volume info</div><div> </div><div>Volume Name: tvol1</div><div>Type: Distribute</div><div>Volume ID: 02d4c9de-e05f-4177-9e86-3b9b2195d7ab</div><div>Status: Started</div><div>Number of Bricks: 2</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: 38.38.38.101:/pool/tvol1</div><div>Brick2: 38.38.38.102:/pool/tvol1</div><div>Options Reconfigured:</div><div>features.quota-deem-statfs: on</div><div>features.inode-quota: on</div><div>features.quota: on</div><div>performance.readdir-ahead: on</div><div><br></div><div>4. gluster volume quota list</div><div><br></div><div>root@server-1:~# gluster volume quota tvol1 list</div><div> Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?</div><div>-------------------------------------------------------------------------------------------------------------------------------</div><div>/ 100.0GB 80%(80.0GB) 0Bytes 100.0GB No No</div><div><br></div><div>5. brick disk usage</div><div><br></div><div>root@server-1:~# df -k</div><div>Filesystem 1K-blocks Used Available Use% Mounted on</div><div>pool 12092178176 0 12092178176 0% /pool</div><div>pool/tvol1 12092178304 128 12092178176 1% /pool/tvol1</div><div>localhost:tvol1 104857600 0 104857600 0% /run/gluster/tvol1</div><div><br></div><div>root@server-2:~# df -k</div><div>Filesystem 1K-blocks Used Available Use% Mounted on</div><div>pool 12092178176 0 12092178176 0% /pool</div><div>pool/tvol1 12092178304 128 12092178176 1% /pool/tvol1</div><div><br></div><div>6. client mount / disk usage</div><div><br></div><div>root@client:~# mount -t glusterfs 38.38.38.101:/tvol1 /mnt</div><div>root@client:~# df -k</div><div>Filesystem 1K-blocks Used Available Use% Mounted on</div><div>38.38.38.101:/tvol1 104857600 0 104857600 0% /mnt</div><div><br></div><div>7. Write using the dd command file</div><div><br></div><div>root@client:~# dd if=/dev/urandom of=/mnt/10m bs=1048576 count=10</div><div>10+0 records in</div><div>10+0 records out</div><div>10485760 bytes (10 MB) copied, 0.751261 s, 14.0 MB/s</div><div><br></div><div>8. client disk usage</div><div><br></div><div>root@client:~# df -k</div><div>Filesystem 1K-blocks Used Available Use% Mounted on</div><div>38.38.38.101:/tvol1 104857600 0 104857600 0% /mnt</div><div><br></div><div>9. brick disk usage</div><div><br></div><div>root@server-1:~# df -k</div><div>Filesystem 1K-blocks Used Available Use% Mounted on</div><div>pool 12092167936 0 12092167936 0% /pool</div><div>pool/tvol1 12092178304 10368 12092167936 1% /pool/tvol1</div><div>localhost:tvol1 104857600 0 104857600 0% /run/gluster/tvol1</div><div><br></div><div>root@server-2:~# df -k</div><div>Filesystem 1K-blocks Used Available Use% Mounted on</div><div>pool 12092178176 0 12092178176 0% /pool</div><div>pool/tvol1 12092178304 128 12092178176 1% /pool/tvol1</div><div><br></div><div>10. zfs volume list information</div><div><br></div><div>root@server-1:~# zfs list</div><div>NAME USED AVAIL REFER MOUNTPOINT</div><div>pool 10.2M 11.3T 19K /pool</div><div>pool/tvol1 10.1M 11.3T 10.1M /pool/tvol1</div><div><br></div><div>root@server-2:~# zfs list</div><div>NAME USED AVAIL REFER MOUNTPOINT</div><div>pool 188K 11.3T 19K /pool</div><div>pool/tvol1 110K 11.3T 110K /pool/tvol1</div><div><br></div><div>11. gluster volume quota list</div><div><br></div><div>root@server-1:~# gluster volume quota tvol1 list</div><div> Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?</div><div>-------------------------------------------------------------------------------------------------------------------------------</div><div>/ 100.0GB 80%(80.0GB) 512Bytes 100.0GB No No</div><div><br></div><div>12. Views from the client file</div><div><br></div><div>root@client:~# ls -al /mnt</div><div>total 10246</div><div>drwxr-xr-x 4 root root 9 1월 30 02:23 .</div><div>drwxr-xr-x 22 root root 4096 1월 28 07:48 ..</div><div>-rw-r--r-- 1 root root 10485760 1월 30 02:23 10m</div><div>drwxr-xr-x 3 root root 6 1월 30 02:14 .trashcan</div><div><br></div><div>root@client:~# df -k</div><div>Filesystem 1K-blocks Used Available Use% Mounted on</div><div>38.38.38.101:/tvol1 104857600 10240 104847360 1% /mnt</div><div><br></div><div>root@server-1:~# gluster volume quota tvol1 list</div><div> Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?</div><div>-------------------------------------------------------------------------------------------------------------------------------</div><div>/ 100.0GB 80%(80.0GB) 10.0MB 100.0GB No No</div><div><br></div><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">-- </font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">Sungsik, Park/corazy [박성식, 朴成植]</font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">Software Development Engineer</font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">Email: <a href="mailto:mulgo79@gmail.com" target="_blank">mulgo79@gmail.com</a></font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움"><br></font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><span style="font-family:돋움;font-size:12.8px">----------------------------------------------------------------------------------------</span><br></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">This email may be confidential and protected by legal privilege. </font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">If you are not the intended recipient, disclosure, copying, distribution </font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">and use are prohibited; please notify us immediately and delete this copy </font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">from your system.</font></p><p style="margin:0cm 0cm 0.0001pt;text-align:justify"><font face="돋움">----------------------------------------------------------------------------------------</font></p></div></div></div></div></div></div></div>
</div>