glusterfs.git/xlators/cluster/dht/src/dht.c, branch v6.4

libglusterfs: Move devel headers under glusterfs directory

2018-12-05T21:47:04+00:00

libglusterfs devel package headers are referenced in code using
include semantics for a program, this while it works can be better
especially when dealing with out of tree xlator builds or in
general out of tree devel package usage.

Towards this, the following changes are done,
- moved all devel headers under a glusterfs directory
- Included these headers using system header notation <> in all
code outside of libglusterfs
- Included these headers using own program notation "" within
libglusterfs

This change although big, is just moving around the headers and
making it correct when including these headers from other sources.

This helps us correctly include libglusterfs includes without
namespace conflicts.

Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b
Updates: bz#1193929
Signed-off-by: ShyamsundarR

dht: volume_options 'options' collision with nfs-ganesha's 'options'

2018-10-04T14:41:17+00:00

When dht was converted to xlator_api, the variable 'options' was not
changed to dht_options, the same as was done to all the other xlators
that were converted to xlator_api.

Thus the reference to 'options' in dht.c is not resolved until runtime,
and the RTlinker's search path starts with symbols in the executable,
i.e. ganesha.nfsd's 'options'. (Which is obviously not the right one.)

The unused extern references to 'options' (now dht_options) in nufa.c
and switch.c is curious.

Change-Id: Idf4a5d5fbd39aadfa5a4b529bceea65a3cbdf8f3
updates: bz#1193929
Signed-off-by: Kaleb S. KEITHLEY

dht: utilize the framework to pass-through xlator tasks

2018-09-19T04:26:55+00:00

Also fixes the issue caused due to not converting back the fn function
to after getting its address. We wanted the value of the field, not
the address of the pt_fop field.

With this patch, DHT will always be started in pass-through
mode if the number of subvols is just 1.

Fixes some tests to make sure DHT is in full config (ie, subvols > 1).
  - increased timeout of brick-mux test as it was bordering on 300 seconds.
  - Also change the volume type to supported 'replica 3' from 'replica 2'.
  - also no DHT tests should assume presence of DHT when there is
    just 1 brick in volume

Credits: Nithya B 

fixes: #405
Change-Id: I8e55239ce58d6ac6ae1901e2e384be1ecbd33d6e
Signed-off-by: Amar Tumballi

Land part 2 of clang-format changes

2018-09-12T12:22:45+00:00

Change-Id: Ia84cc24c8924e6d22d02ac15f611c10e26db99b4
Signed-off-by: Nigel Babu

dht: Implement ipc fop

2016-08-27T12:48:36+00:00

ipc is used by md-cache to communicate the list of xattrs that
it is caching, to the upcall xlator. Hence implement this in
dht, such that it winds to all the bricks if the ipc op is
GF_IPC_MDC_TARGET_UPCALL. The ips should not fail if any of
the bricks is down, as md-cache will replay the ipc late when
the brick comes back up.

Change-Id: Ica551a550c04cbb1240c0d211fe831c2e5eb6017
BUG: 1211863
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/15225
CentOS-regression: Gluster Build System 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Raghavendra G

dht: Add lease() fop

2016-04-25T14:30:16+00:00

Change-Id: I0bbc2c2ef115c78393f6570815a5b80316e7e4be
BUG: 1319992
Signed-off-by: Poornima G 
Reviewed-on: http://review.gluster.org/11720
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: Rajesh Joseph 
Reviewed-by: Jeff Darcy

cluster/dht : Ftruncate on migrating file fails with EINVAL

2015-12-22T19:13:00+00:00

What:
If dht_open is called on a migrating file after the inode_ctx is set,
subsequent FOPs on that fd do not open the fd on the dst subvol.
This is seen when the open-ftruncate-close sequence is repeatedly
called on a migrating file.
A second call to the sequence described above causes dht_truncate_cbk
to call dht_truncate2 as the dht_inode_ctx was already set by the first
call. As dht_rebalance_in_progress_check is not called, the fd is not
opened on the dst subvol.
On a distributed-replicate volume, this causes AFR to
open the fd using afr_fix_open, but with the wrong flags, causing
posix_ftruncate to fail with EINVAL.
The fix: We require fd specific information to make a decision while
handling migrating files.
Set the fd_ctx to indicate the fd has been opened on the dst subvol
and check if it has been set while processing Phase1/Phase2 checks
in the FOP callback functions.

Change-Id: I43cdcd8017b4a11e18afdd210469de7cd9a5ef14
BUG: 1284823
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12985
Reviewed-by: Raghavendra G 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

cluster/tier: fix loading tier.so into glusterd

2015-12-03T18:01:07+00:00

glusterd occasionally loads shared libraries of translators. This
failed for tiering due to a reference to dht_methods which is defined
as a global variable which is not necessary.
The global variable has been removed and this is now a member of
dht_conf and is now initialised in the *_init calls.

Change-Id: Ifa0a21e3962b5cd8d9b927ef1d087d3b25312953
BUG: 1287842
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/12863
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Dan Lambright 
Tested-by: Dan Lambright

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

cluster/dht: Add tier translator.

2015-03-21T16:50:29+00:00

The tier translator shares most of DHT's code. It differs in how
subvolumes are chosen for I/Os, and how file migration (cache promotion
and demotion) is managed. That different functionality is split to either
DHT or tier logic according to the "tier_methods" structure.

A cache promotion and demotion thread is created in a manner
similar to the rebalance daemon. The thread operates a timing
wheel which periodically checks for promotion and demotion candidates
(files). Candidates are queued and then migrated. Candidates must exist on
the same node as the daemon and meet other critera per caching policies.

This patch has two authors (Dan Lambright and Joseph Fernandes). Dan
did the DHT changes and Joe wrote the cache policies. The fix depends on
DHT readidr changes and the database library which have been submitted
separately.  Header files in libglusterfs/src/gfdb should be reviewed in
patch 9683.

For more background and design see the feature page [1].

[1]
http://www.gluster.org/community/documentation/index.php/Features/data-classification

Change-Id: Icc26c517ccecf5c42aef039f5b9c6f7afe83e46c
BUG: 1194753
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9724
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur