glusterfs.git/extras/group-gluster-block, branch v4.0dev features/shard: Handle offset in appending writes 2017-05-27T16:00:50+00:00 Pranith Kumar K pkarampu@redhat.com 2017-05-24T17:00:29+00:00 bea02e26a3967a6e679e30fbb77ecfeff1e71f37 When a file is opened with append, all writes are appended at the end of file irrespective of the offset given in the write syscall. This needs to be considered in shard size update function and also for choosing which shard to write to. At the moment shard piggybacks on queuing from write-behind xlator for ordering of the operations. So if write-behind is disabled and two parallel appending-writes come both of which can increase the file size beyond shard-size the file will be corrupted. BUG: 1455301 Change-Id: I9007e6a39098ab0b5d5386367bd07eb5f89cb09e Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17387 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> 
When a file is opened with append, all writes are appended at the end of file
irrespective of the offset given in the write syscall. This needs to be
considered in shard size update function and also for choosing which shard to
write to.

At the moment shard piggybacks on queuing from write-behind
xlator for ordering of the operations. So if write-behind is disabled and
two parallel appending-writes come both of which can increase the file size
beyond shard-size the file will be corrupted.

BUG: 1455301
Change-Id: I9007e6a39098ab0b5d5386367bd07eb5f89cb09e
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17387
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
extras: Provide group set for gluster-block workloads 2017-05-12T13:21:28+00:00 Pranith Kumar K pkarampu@redhat.com 2017-05-10T10:56:35+00:00 ef61a79f33ca43a9548b9076bb152e6421416f78 For gluster-block workloads I/O is always with o-direct so it doesn't benefit by any of the perf xlators so disabling all of them to save on memory. performance.quick-read=off performance.read-ahead=off performance.io-cache=off performance.stat-prefetch=off performance.write-behind=off performance.open-behind=off performance.readdir-ahead=off We want the I/O on the file to be with o-direct network.remote-dio=enable Options that are proven to give good performance with VM workloads which is very similar to gluster-block cluster.eager-lock=enable cluster.quorum-type=auto cluster.data-self-heal-algorithm=full cluster.locking-scheme=granular cluster.shd-max-threads=8 cluster.shd-wait-qlength=10000 features.shard=on It is better to turn off things we are not using user.cifs=off It is better to have allow-insecure to be on so that ports that are > 1024 in tcmu-runner are allowed. server.allow-insecure=on Change-Id: I9a21c824fa42242f02b57569feedd03d9b6f9439 BUG: 1450010 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17254 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> 
For gluster-block workloads I/O is always with o-direct so it doesn't
benefit by any of the perf xlators so disabling all of them to save
on memory.
performance.quick-read=off
performance.read-ahead=off
performance.io-cache=off
performance.stat-prefetch=off
performance.write-behind=off
performance.open-behind=off
performance.readdir-ahead=off

We want the I/O on the file to be with o-direct
network.remote-dio=enable

Options that are proven to give good performance with
VM workloads which is very similar to gluster-block
cluster.eager-lock=enable
cluster.quorum-type=auto
cluster.data-self-heal-algorithm=full
cluster.locking-scheme=granular
cluster.shd-max-threads=8
cluster.shd-wait-qlength=10000
features.shard=on

It is better to turn off things we are not using
user.cifs=off

It is better to have allow-insecure to be on so that
ports that are > 1024 in tcmu-runner are allowed.
server.allow-insecure=on

Change-Id: I9a21c824fa42242f02b57569feedd03d9b6f9439
BUG: 1450010
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: https://review.gluster.org/17254
Smoke: Gluster Build System <jenkins@build.gluster.org>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>