glusterfs-snapshot.git/xlators/cluster/afr/src/afr-transaction.c, branch gluster-snap

libglusterfs: Add monotonic clocking counter for timer thread

2013-10-15T07:14:57+00:00

gettimeofday() returns the current wall clock time and timezone.
Using these functions in order to measure the passage of time
(how long an operation took) therefore seems like a no-brainer.

This time suffer's from some limitations:

a. They have a low resolution: “High-performance” timing by
definition, requires clock resolutions into the microseconds
or better.

b. They can jump forwards and backwards in time: Computer
clocks all tick at slightly different rates, which causes
the time to drift. Most systems have NTP enabled which
periodically adjusts the system clock to keep them in sync
with “actual” time. The adjustment can cause the clock to
suddenly jump forward (artificially inflating your timing
numbers) or jump backwards (causing your timing calculations
to go negative or hugely positive). In such cases timer
thread could go into an infinite loop.

From 'man gettimeofday':
----------
..
..
The time returned by gettimeofday() is affected by discontinuous
jumps in the system time (e.g., if the system administrator manually
changes the system time).  If you need a monotonically increasing
clock, see clock_gettime(2).
..
..
----------

Rationale:

For calculating interval timing for Timer thread, all that’s
needed should be clock as a simple counter that increments
at a stable rate.

This is necessary to avoid the jumps which are caused by using
"wall time", this counter must be monotonic that can never
“tick” backwards, ever.

Change-Id: I701d31e71a85a73d21a6c5cd15583e7a5a645eeb
BUG: 1017993
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/6070
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/afr: Add special handling for failure postops

2013-08-29T00:42:59+00:00

Idea is to not leave the file in FOOL-FOOL scenario in case on
all the bricks data transaction failed with EDQUOT to avoid
increasing un-necessary load of self-heals in the system.

For directory transactions don't leave pending changelog in case
the failures are seen on all the subvolumes.

Change-Id: I38a5561d1d581a78347a76a4a509514e4a0c3fb7
BUG: 969461
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5709
Reviewed-by: Anand Avati 
Tested-by: Gluster Build System

cluster/afr: Don't delay post op in cases of failures

2013-08-28T22:40:31+00:00

Change-Id: Ib0c3af6babc61dc3ed45252582876e2f243d6446
BUG: 958118
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5635
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Anand Avati

cluster/afr: Disable eager-lock if open-fd-count > 1

2013-08-02T09:25:55+00:00

Lets say mount1 has eager-lock(full-lock) and after the eager-lock
is taken mount2 opened the same file, it won't be able to
perform any data operations until mount1 releases eager-lock.
To avoid such scenario do not enable eager-lock for transaction
if open-fd-count is > 1. Delaying of changelog piggybacking is
avoided in this situation.

Change-Id: I51b45d6a7c216a78860aff0265a0b8dabc6423a5
BUG: 910217
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5432
Tested-by: Gluster Build System 
Reviewed-by: venkatesh somyajulu 
Reviewed-by: Vijay Bellur

storage/posix: implement batched fsync in a single thread

2013-07-23T13:11:12+00:00

Because of the extra fsync()s issued by AFR transaction, they
could potentially "clog" all the io-threads denying unrelated
operations from making progress.

This patch assigns a dedicated thread to issues fsyncs, as
an experimental feature to understand performance characteristics
with the approach.

As a basis, incoming individual fsync requests are grouped into
batches, falling in the same @batch-fsync-delay-usec window of
time. These windows can extend in practice, as processing of
the previous batch can take longer than @batch-fsync-delay-usec
while new requests are getting batched.

The feature support three modes (similar to the -S modes of fs_mark)

- syncfs: In this mode one syncfs() is issued per batch, instead
  of N fsync()s (one per file.)

- syncfs-single-fsync: In this mode one syncfs() is issued per
  batch (which, on Linux, guarantees the completion of write-out
  of dirty pages in the filesystem up to that point) and one single
  fsync() to synchronize or flush the controller/drive cache. This
  corresponds to -S 2 of fsmark.

- syncfs-reverse-fsync: In this mode, one syncfs() is issued per
  batch, and all the open files in that batch are fsync()'ed in
  the reverse order of the queue. This corresponds to -S 4 of
  fsmark.

- reverse-fsync: In this mode, no syncfs() is issued and all the
  files in the batch are fsync()'ed in the reverse order. This
  corresponds to -S 3 of fsmark.

Change-Id: Ia1e170a810c780c8d80e02cf910accc4170c4cd4
BUG: 927146
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/4746
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Refactor inodelk to handle multiple domains

2013-07-04T04:28:11+00:00

- afr_local_copy should not be memduping locked nodes, that would
  mean that lock is taken in self-heal on those nodes even before
  it actually takes the lock. So removed memdup code. Even entry
  lock related copying (lockee info) is also not necessary for
  self-heal functionality, so removing that as well. Since it is
  not local_copy anymore changed its name.

- My editor changed tabs to spaces.

Change-Id: I8dfb92cb8338e9a967c06907a8e29a8404782d61
BUG: 967717
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5099
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Provide an option to disable afr durability

2013-07-03T14:33:13+00:00

Change-Id: I40eec20ca6b3f857245a2438883822e251077ee9
BUG: 979365
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5269
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: post-op should complete before starting flush

2013-07-03T07:40:08+00:00

Problem:
At the moment afr-flush makes sure that a delayed post-op
is woken up but it does not wait for it to complete the
post-op before flush unwinds.
These are the steps that are happening:
1) flush fop comes on an fd which wakes up a delayed post-op
and continues with the flush fop.
2) post-op sends fsync on the wire.
3) flush completes and unwinds to fuse.
4) graph switch happens on the fuse mount disconnecting the
old graph's client connections to bricks.
5) xattrop after fsync fails with ENOTCONN because the
connections from old graph are taken down now.

Fix:
Wait for post-op to complete before starting to flush.
We could make flush act similar to fsync (i.e.) wind
flush as is but wait for post-op to complete before unwinding
flush, but it is better to send flush as the final fop. So
wind of flush will start after post-op is complete. Had to
change fsync to accommodate this change.

Change-Id: I93aa642647751969511718b0e137afbd067b388a
BUG: 980548
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5274
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Handle NULL fdctx in fsync

2013-06-28T06:34:29+00:00

Problem:
If fdctx is NULL in afr_fsync, process crashes because
of NULL dereference.

Fix:
if fdctx is NULL, always say witnessed unstable write so
that fsyncs are done properly. Handled fdctx being null
in afr_delayed_changelog_post_op otherwise fsync stub is
never resumed and the mount was hanging.

Change-Id: Icacc900e9be63c29db3325cb0e19cc250adebaac
BUG: 978794
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5258
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/afr: Fix fd/memory leak on fsync

2013-06-24T16:45:19+00:00

Change-Id: I764883811e30ca9d9c249ad00b6762101083a2fe
BUG: 976800
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/5248
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy