glusterfs.git/extras/group-gluster-block, branch v4.1.6

block-profile: enable cluster.eager-lock in block-profile

2018-05-01T12:22:48+00:00

Eager-lock gave 2.5X perf improvement. On top of that with batching
fix in tcmu-runner and client-io-threads we are seeing close to 3X perf
improvement. But we don't want to include that in the default profile
option but enable it on a case by case basis. So not adding
client-io-threads option.

BUG: 1573119
Fixes: bz#1573119
Change-Id: Ida53c3ef9a041a73b65fdd06158ac082da437206
Signed-off-by: Prasanna Kumar Kalever

extras: Disable choose-local in groups virt and gluster-block

2018-04-13T10:23:03+00:00

Change-Id: Icba68406d86623195d59d6ee668e0850c037c63a
fixes: bz#1566386
Signed-off-by: Krutika Dhananjay

gluster-block: strict-o-direct should be on

2017-08-29T13:36:07+00:00

tcmu-runner is not going to open block with O_SYNC anymore
so writes have a chance of getting cached in write-behind
when that happens, there is a chance that on failover some
data could be stuck in cache and be lost.

BUG: 1485962
Change-Id: If9835d914821dfc4ff432dc96775677a55d2918f
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/18120
CentOS-regression: Gluster Build System 
Reviewed-by: Vijay Bellur 
Smoke: Gluster Build System 
Tested-by: Shyamsundar Ranganathan

extras: Turn eager-lock off for gluster-block

2017-08-02T04:45:53+00:00

With the current implementation of eager-lock FINODELK is taking so much
time that cassandra workload times out and errors out. AFR eager-locking
needs to be changed similar to EC eager-locking to make things work
as expected. In the interim, it is better to turn it off

This is how the profile looks if eager-lock is turned on:
      0.35     628.26 us      64.00 us  129882.00 us          42278    FXATTROP
     17.45   16500.54 us     212.00 us  375829.00 us          79568       WRITE
     81.76  209862.12 us      15.00 us 1992486.00 us          29318    FINODELK

This is how profile looks if eager-lock is turned off:
      1.87     283.71 us      65.00 us  298970.00 us          68346    FXATTROP
      6.33     199.04 us      13.00 us  373428.00 us         330524    FINODELK
     10.37    3151.47 us      53.00 us 1528484.00 us          34172       FSYNC
     81.31    5110.45 us     270.00 us 1519722.00 us         165244       WRITE

BUG: 1477404
Change-Id: I98026b1ecf30002ddac01be76f375c2e8c0b7838
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17954
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Tested-by: Krutika Dhananjay 
Reviewed-by: Krutika Dhananjay

group-gluster-block: Set default shard-block-size to 64MB

2017-07-27T10:39:52+00:00

With 4MB shard size I/O slows down more because of the extra
inodelk/xattrops in replicate. So increasing it to 64MB which
gave better performance than 4MB.

To simulate writes on a preallocated VM-image, fallocate the file and then do
dd with notrunc
do "fallocate -l 1GB" then "dd if=/dev/zero of=file-1GB bs=1MB count=1024 conv=notrunc"

These are the results on my laptop for dd:
With 4MB:
      1.84    1357.37 us      19.00 us   12431.00 us           1188    FINODELK
      2.45     255.08 us      58.00 us    4038.00 us           8428       WRITE
     95.69   78967.76 us      30.00 us 20324240.00 us           1063    FXATTROP

With 64MB:
      0.13      59.36 us      15.00 us     814.00 us            657    FINODELK
      6.02     225.53 us      69.00 us    6556.00 us           8205       WRITE
     93.82  103015.12 us      32.00 us 13046368.00 us            280    FXATTROP

BUG: 1475605
Change-Id: I4ed5441409df639e38c731ba0d140fe92902f25f
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17887
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay

extras: Disable remote-dio in gluster-block profile

2017-07-24T14:17:10+00:00

gluster-block file is opened with O_DIRECT but because block-profile has
remote-dio enabled, it is leading to high latency for fsync which leads to
failures in cassandra. Disabling remote-dio fixed this issue. We need to change
remote-dio to disabled in gluster-block.

BUG: 1474190
Change-Id: Ifd845ea9cbdcc08dd6073faca6082682af376ca3
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17856
CentOS-regression: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
Smoke: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Niels de Vos

features/shard: Handle offset in appending writes

2017-05-27T16:00:50+00:00

When a file is opened with append, all writes are appended at the end of file
irrespective of the offset given in the write syscall. This needs to be
considered in shard size update function and also for choosing which shard to
write to.

At the moment shard piggybacks on queuing from write-behind
xlator for ordering of the operations. So if write-behind is disabled and
two parallel appending-writes come both of which can increase the file size
beyond shard-size the file will be corrupted.

BUG: 1455301
Change-Id: I9007e6a39098ab0b5d5386367bd07eb5f89cb09e
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17387
Smoke: Gluster Build System 
Reviewed-by: Krutika Dhananjay 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

extras: Provide group set for gluster-block workloads

2017-05-12T13:21:28+00:00

For gluster-block workloads I/O is always with o-direct so it doesn't
benefit by any of the perf xlators so disabling all of them to save
on memory.
performance.quick-read=off
performance.read-ahead=off
performance.io-cache=off
performance.stat-prefetch=off
performance.write-behind=off
performance.open-behind=off
performance.readdir-ahead=off

We want the I/O on the file to be with o-direct
network.remote-dio=enable

Options that are proven to give good performance with
VM workloads which is very similar to gluster-block
cluster.eager-lock=enable
cluster.quorum-type=auto
cluster.data-self-heal-algorithm=full
cluster.locking-scheme=granular
cluster.shd-max-threads=8
cluster.shd-wait-qlength=10000
features.shard=on

It is better to turn off things we are not using
user.cifs=off

It is better to have allow-insecure to be on so that
ports that are > 1024 in tcmu-runner are allowed.
server.allow-insecure=on

Change-Id: I9a21c824fa42242f02b57569feedd03d9b6f9439
BUG: 1450010
Signed-off-by: Pranith Kumar K 
Reviewed-on: https://review.gluster.org/17254
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy