glusterfs.git/libglusterfs/src/syncop.c, branch v3.10.7

cluster/dht: rebalance perf enhancement

2017-08-11T17:40:29+00:00

Problem: Throttle settings "normal" and "aggressive" for rebalance
did not have performance difference.

normal mode spawns $(no. of cores - 4)/2 threads and aggressive
spawns $(no. of cores - 4) threads. Though aggressive mode has twice
the number of threads compared to that of normal mode, there was no
performance gain when switched to aggressive mode from normal mode.

RCA:
During the course of debugging the above problem, we tried assigning
migration job to migration threads spawned by rebalance, rather than
synctasks(as there is more overhead associated to manage the task
queue and threads). This gave us a significant improvement over rebalance
under synctasks. This patch does not really gurantee that there will be a
clear performance difference between normal and aggressive mode, but this
patch certainly maximized the disk utilization for 1GBfiles run.

Results:

Test enviroment:
Gluster Config:
Number of Bricks: 2 (one brick per disk(RAID-6 12 disk))
Bricks:
Brick1: server1:/brick/test1/1
Brick2: server2:/brick/test1/1
Options Reconfigured:
performance.readdir-ahead: on
server.event-threads: 4
client.event-threads: 4

1000 files with 1GB each were created/renamed such that all files will have
server1 as cached and server2 as hashed, so that all files will be migrated.

Test machines had 24 cores each.

Results  with/without synctask based migration:
-----------------------------------------------

mode                    normal(10threads)          aggressive(20threads)

timetaken               0:55:30 (h:m:s)            0:56:3 (h:m:s)
withsynctask

timetaken
with migrator           0:38:3 (h:m:s)             0:23:41 (h:m:s)
threads

From above table it can be seen that, there is a clear 2x perf gain between
rebalance with synctask vs rebalance with migrator threads.

Additionally this patch modifies the code so that caller will have the exact error
number returned by dht_migrate_file(earlier the errno meaning was overloaded). This
will help avoiding scenarios where migration failure due to ENOENT, can result in
rebalance abort/failure.

> Change-Id: I8904e2fb147419d4a51c1267be11a08ffd52168e
> BUG: 1420166
> Signed-off-by: Susant Palai 
> Reviewed-on: https://review.gluster.org/16427
> Smoke: Gluster Build System 
> Reviewed-by: N Balachandran 
> Reviewed-by: Raghavendra G 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Signed-off-by: Susant Palai 

Change-Id: I8904e2fb147419d4a51c1267be11a08ffd52168e
BUG: 1473134
Signed-off-by: Susant Palai 
Reviewed-on: https://review.gluster.org/17834
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan

core: fix synclocks' handling of "woken" flag

2017-04-13T15:39:09+00:00

The "woken" flag wasn't being reset when it should have been, leading
(eventually) to a SEGV when someone tried to folow a synclock's waitq
to a task structure that had been freed while still on the queue.  See
the bug report for (far) more detail.

Backport of:
> Commit 31377765dbbb8d49292c4362837a695adcbc6cb4
> BUG: 1434062
> Reviewed-on: https://review.gluster.org/16926

Change-Id: I5cd9ae1bcb831555274108b292181ec2a29b6d95
BUG: 1441474
Signed-off-by: Jeff Darcy 
Reviewed-on: https://review.gluster.org/17043
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Shyamsundar Ranganathan

core: Honour mandatory lock flags during lock migration

2016-05-20T11:23:19+00:00

lk_flags from posix_lock_t structure is the primary key used to
differentiate locks as either advisory and mandatory type. During
lock migration this field is not read in getactivelk() call path.
So in order to copy the exact lock state from source to destination
it is necessary to include lk_flags within lock_migration_info_t
structure to maintain accurate state. This change also includes
minor modifications to setactivelk() call to consider lk_flags
during lock migration.

Change-Id: I20a7b6b6a0f3bdac5734cce8a2cd2349eceff195
BUG: 1332501
Signed-off-by: Anoop C S 
Reviewed-on: http://review.gluster.org/14189
Smoke: Gluster Build System 
Reviewed-by: Susant Palai 
Reviewed-by: Poornima G 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Pranith Kumar Karampuri

core: add setactivelk () fop

2016-05-02T01:04:42+00:00

Change-Id: Ic2ba77a1fdd27801a6e579e04e6c0dd93cd7127b
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/14011
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

core: add getactivelk () fop

2016-05-02T01:04:21+00:00

Change-Id: Ifd0ff278dcf43da064021f5c25e5dcd34347fcde
BUG: 1326085
Signed-off-by: Susant Palai 
Reviewed-on: http://review.gluster.org/13970
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

Use pthread_equal to compare threads

2016-04-25T16:43:37+00:00

The man pages about pthreads are quite clear on the fact that pthread_t
is supposed to be opaque, and so can't be compared using the equality
operator.

Change-Id: Id69e166ed73a98668d19a71cd6d9ab9a0429ec38
BUG: 1330225
Signed-off-by: Michael Scherer

core: add lease fop

2016-04-21T10:01:38+00:00

Change-Id: Ia27d66b1061b0377857827515590eb89b18515c9
BUG: 1319992
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/11596
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Pranith Kumar Karampuri

syncop: add seek() FOP

2016-01-31T11:34:04+00:00

Add the new seek() FOP to the syncop framework. gfapi will use this in
the future.

Change-Id: I0c15153beb27de73d5844b6f692175750fc28f60
BUG: 1220173
Singed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/11481
Smoke: Gluster Build System 
Tested-by: Niels de Vos 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY

syncop: Include iatt to 'syncop_link' args

2015-07-10T17:42:26+00:00

Include iatt to 'syncop_link' args to fetch proper attributes of
the newly linked inode.
Signed-off-by: Soumya Koduri 

Change-Id: If6b92961bd7a89add3791ed3a9b494087348b492
BUG: 1241788
Reviewed-on: http://review.gluster.org/11611
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Kaleb KEITHLEY

quota/marker: use smaller stacksize in synctask for marker updation

2015-07-10T06:54:23+00:00

Default stacksize that synctask uses is 2M.
For marker we set it to 16k

Also move market xlator close to io-threads
to have smaller stack

Change-Id: I8730132a6365cc9e242a3564a1e615d94ef2c651
BUG: 1207735
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/11499
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Raghavendra G