glusterfs.git/xlators/cluster/dht/src/dht-layout.c, branch v3.8.8

cluster/distribute: detect stale layouts in entry fops

2016-04-22T17:28:54+00:00

dht_mkdir ()
{
      first-hashed-subvol = hashed-subvol for "bname" in in-memory
                            layout of "parent";
      inodelk (SETLKW, parent, "LAYOUT_HEAL_DOMAIN", "can be any
               subvol, but we choose first-hashed-subvol randomly");
      {
begin:
            hashed-subvol = hashed-subvol for "bname" in in-memory
                            layout of "parent";
            hash-range = extract hashe-range from layout of "parent";

            ret = mkdir (parent/bname, hashed-subvol, hash-range);
            if (ret == "hash-value doesn't fall into layout stored on
                       the brick (this error is returned by posix-mkdir)")
            {
                refresh_parent_layout ();
                goto begin;
            }

      }
      inodelk (UNLCK, parent, "LAYOUT_HEAL_DOMAIN",
               "first-hashed-subvol");

      proceed with other parts of dht_mkdir;
}

posix_mkdir (parent/bname, client-hash-range)
{

       disk-hash-range = getxattr (parent, "dht-layout-key");
       if (disk-hash-range != client-hash-range) {
              fail-with-error ("hash-value doesn't fall into layout
                                stored on the brick");
              return 0;
       }

       continue-with-posix-mkdir;
}

Similar changes need to be done for dentry operations like create,
symlink, link, unlink, rmdir, rename. These will be addressed in
subsequent patches. This patch addresses only mkdir codepath.

This change breaks stripe tests, as on some striped subvols dht layout
xattrs are not set for some reason. This results in failure of
mkdir. Since striped volumes are always created with dht, some tests
associated with stripe also fail. So, I am making following tests
changes (since stripe is out of maintainance):
* modify ./tests/basic/rpc-coverage.t to not to use striped volumes
* mark all (2) tests in tests/bugs/stripe/ as bad tests

Change-Id: Idd1ae879f24a48303dc743c1bb4d91f89a629e25
BUG: 1323040
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/13885
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: N Balachandran

dht : updating return value for layout set function

2015-08-13T05:59:56+00:00

Change-Id: I7fd89e00b418391afe0a13c2033919c979cc8bbb
BUG: 789278
Signed-off-by: Sakshi 
Reviewed-on: http://review.gluster.org/10869
Tested-by: NetBSD Build System 
Reviewed-by: N Balachandran 
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G

dht: Adding log messages to the new logging framework

2015-06-24T05:15:10+00:00

Change-Id: Ib3bb61c5223f409c23c68100f3fe884918d2dc3f
BUG: 1194640
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/10021
Reviewed-by: N Balachandran 
Reviewed-by: Joseph Fernandes
Tested-by: Joseph Fernandes
Reviewed-by: Dan Lambright 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

build: do not #include "config.h" in each file

2015-05-29T12:43:17+00:00

Instead of including config.h in each file, and have the additional
config.h included from the compiler commandline (-include option).

When a .c file tests for a certain #define, and config.h was not
included, incorrect assumtions were made. With this change, it can not
happen again.

BUG: 1222319
Change-Id: I4f9097b8740b81ecfe8b218d52ca50361f74cb64
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10808
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Pranith Kumar Karampuri

dht: make lookup-unhashed=auto do something actually useful

2015-05-10T13:17:22+00:00

The key concept here is to determine whether a directory is "clean" by
comparing its last-known-good topology to the current one for the
volume.  These are stored as "commit hashes" on the directory and the
volume root respectively.  The volume's commit hash changes whenever a
brick is added or removed, and a fix-layout is done.  A directory's
commit hash changes only when a full rebalance (not just fix-layout)
is done on it.  If all bricks are present and have a directory
commit hash that matches the volume commit hash, then we can assume
that every file is in its "proper" place. Therefore, if we look for
a file in that proper place and don't find it, we can assume it's not
on any other subvolume and *safely* skip the global (broadcast to all)
lookup.

Change-Id: Id6ce4593ba1f7daffa74cfab591cb45960629ae3
BUG: 1219637
Signed-off-by: Jeff Darcy 
Signed-off-by: Shyam 
Reviewed-on: http://review.gluster.org/7702
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System
Reviewed-by: Vijay Bellur

dht: Fixing dereference after null check

2015-05-08T08:22:35+00:00

CID: 1223229
The 'loc' ptr is not checked before
dereferencing, which is handled.

Change-Id: Icf668150bde190e6f1b9f58a038099338516efe8
BUG: 789278
Signed-off-by: arao 
Reviewed-on: http://review.gluster.org/9666
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

dht : coverity fixes

2015-04-07T10:52:05+00:00

CID : 1124352,1124365 (unchecked return value),
      1124377 ( logically dead code),
      1124511 (null dereference)

Change-Id: I61e029a078559cfe15d36bf0aa53418f6214e5cb
BUG: 789278
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/9622
Tested-by: Gluster Build System 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: N Balachandran 
Reviewed-by: Vijay Bellur

Avoid conflict between contrib/uuid and system uuid

2015-04-04T17:48:35+00:00

glusterfs relies on Linux uuid implementation, which
API is incompatible with most other systems's uuid. As
a result, libglusterfs has to embed contrib/uuid,
which is the Linux implementation, on non Linux systems.
This implementation is incompatible with systtem's
built in, but the symbols have the same names.

Usually this is not a problem because when we link
with -lglusterfs, libc's symbols are trumped. However
there is a problem when a program not linked with
-lglusterfs will dlopen() glusterfs component. In
such a case, libc's uuid implementation is already
loaded in the calling program, and it will be used
instead of libglusterfs's implementation, causing
crashes.

A possible workaround is to use pre-load libglusterfs
in the calling program (using LD_PRELOAD on NetBSD for
instance), but such a mechanism is not portable, nor
is it flexible. A much better approach is to rename
libglusterfs's uuid_* functions to gf_uuid_* to avoid
any possible conflict. This is what this change attempts.

BUG: 1206587
Change-Id: I9ccd3e13afed1c7fc18508e92c7beb0f5d49f31a
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/10017
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

Xlators : Fixed typos

2015-04-02T10:21:08+00:00

Change-Id: I948f85cb369206ee8ce8b8cd5e48cae9adb971c9
BUG: 1075417
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/9529
Reviewed-by: Niels de Vos 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Humble Devassy Chirammal

cluster/dht: Change the subvolume encoding in d_off to be a "global"

2015-03-18T11:47:41+00:00

position in the graph rather than relative (local) to a particular
translator.

Encoding the volume in this way allows a single translator to manage
which brick is currently being scanned for directory entries. Using a
single translator minimizes allocated bits in the d_off. It also allows
multiple DHT translators in the same graph to have a common frame of
reference (the graph position) for which brick is being read. Multiple
DHT translators are needed for the Tiering feature.

The fix builds off a previous change (9332) which removed subvolume
encoding from AFR. The fix makes an equivalent change to the EC
translator.

More background can be found in fix 9332 and gluster-dev discussions [1].

DHT and AFR/EC are responsibile (as before) for choosing which brick to
enumerate directory entries in over the readdir lifecycle.

The client translator receiving the readdir fop encodes the dht_t. It
is referred to as the "leaf node" in the graph and corresponds to the
brick being scanned.

When DHT decodes the d_off, it translates the leaf node to a local
subvolume, which represents the next node in the graph leading to
the brick.

Tracking of leaf nodes is done in common utility functions. Leaf nodes
counts and positional information are updated on a graph switch.

[1] www.gluster.org/pipermail/gluster-devel/2015-January/043592.html

Change-Id: Iaf0ea86d7046b1ceadbad69d88707b243077ebc8
BUG: 1190734
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9688
Reviewed-by: Xavier Hernandez 
Reviewed-by: Krishnan Parthasarathi 
Reviewed-by: Vijay Bellur 
Tested-by: Vijay Bellur