glusterfs.git/xlators/cluster/dht/src/dht-common.c, branch v3.4.1

cluster/dht: Ignore subvols with error in min-free-disk/inodes

2013-09-10T16:05:18+00:00

Currently when selecting a alternative subvolume when hashed
subvol has exceeded min-free-disk/inodes, we do not check if
layouts have errors (including decommissioning). This leads
to data being written to those subvolumes, and in case of
decommissioning, will lead to data loss.

BUG: 982919

> Original-Author: shishir gowda 
> Reviewed-on: http://review.gluster.org/5299

Change-Id: If301a86cf3ca5fad6529bd2e61382f9901663ba0
Signed-off-by: Amar Tumballi 
Reviewed-on: http://review.gluster.org/5888
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

dht,posix: support for case discovery

2013-06-08T21:51:22+00:00

This is support for discovering a filename in a given directory
which has a case insensitive match of a given name. It is implemented
as a virtual extended attribute on the directory where the required
filename is specified in the key.

E.g:

  sh# getfattr -e "text" -n user.glusterfs.get_real_filename:FiLe-B /mnt/samba/patchy
  getfattr: Removing leading '/' from absolute path names
  # file: mnt/samba/patchy
  user.glusterfs.get_real_filename:FiLe-B="file-b"

In reality, there can be multiple "answers" as the backend filesystem is
case sensitive and there can be multiple files which can strcasecamp()
successfully. In this case we pick the first matched file from the first
responding server.

If a matching file does not exist, we return ENOENT (and NOT ENODATA).
This way the caller can differentiate between "unsupported" glusterfs
API and file not existing.

This API is used by Samba VFS to perform efficient discovery of the real
filename without doing a full scan at the Samba level.

Change-Id: I53054c4067cba69e585fd0bbce004495bc6e39e8
BUG: 953694
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/5163
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/distribute: Prevent spurious multiple defrag crawls

2013-03-04T10:42:10+00:00

In dht_notify, we used to create a thread to start defrag
crawls after we had heard from all child subvols.
This was in-correct, as a later event, could also trigger the
crawl again(due to the fact that all subvols had responded).

The fix is to make sure, the thread is started only once after
all subvols have responded the first time

BUG: 916449
Change-Id: I1619344fbb1cb51d5e1db38d8a29821fa870fa8b
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4610
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

cluster/dht: Create linkfile with file uid/gid

2013-03-04T10:40:01+00:00

Currently, linkfile creation happens as root.

use uid/gid returned from _cbk (link/rename) to set the correct ownership of
the link files.

Also added test/dht.rc to implement common dht functions

BUG: 884597
Change-Id: I6bc0e04f62d4716fc033681e5678e852a1be7a2f
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4607
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

cluster/dht: pathinfo xattr changes for directories

2013-02-09T03:09:46+00:00

Since directories have presence on all subvolumes there is
no definite meaning of ->hashed_subvol or ->cached_subvol.
getxattr() code path chooses ->cached_subvol for pathinfo
extended attribute. While this makes sense of files, it makes
less sense for directories. Further if a hashed or a cached
subvolume is down, and there's a getxattr request for a
directory, we return with an errno.

This patch changes pathinfo extended attribute contents by
aggregating information from all subvolumes that are up.

Change-Id: I58adb741d63ccfd1d0239af75eb65f26f0fb384d
Signed-off-by: Venky Shankar 
BUG: 856455
Reviewed-on: http://review.gluster.org/4047
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: ignore EEXIST error in mkdir to avoid GFID mismatch

2013-02-03T20:14:19+00:00

In dht_mkdir_cbk, EEXIST error is treated like a true error. Because
of this the following sequence of events can happen, eventually
resulting in GFID mismatch and (and possibly leaked locks and hang,
in the presence of replicate.)

The issue exists when many clients concurrently attempt creation of
directory and subdirectory (e.g mkdir -p /mnt/gluster/dir1/subdir)

0. First mkdir happens by one client on the hashed subvolume. Only
   one client wins the race. Others racing mkdirs get EEXIST. Yet
   other "laggers" in the race encounter the just-created directory
   in lookup() on the hash dir.

1. At least one "lagger" lookup() notices that there are missing
   directories on other subvolumes (which the "winner" mkdir is yet
   to create), and starts off self-heal of the directory.

2. At least on some subvolumes, self-heal's mkdir wins the race
   against the "winner" mkdir and creates the directory first. This
   causes the "winner" mkdir to experience EEXIST error on those
   subvolumes.

3. On other subvolumes where "winner" mkdir won the race, self-heal
   experiences EEXIST error, but self-heal is properly translating
   that into a success (but mkdir code path is not -- which is the
   bug.)

4. Both mkdir and self-heal assign hash layouts to the just created
   directory. But self-heal distributes hash range across N (total)
   subvolumes, whereas mkdir distributes hash range across N - M
   (where M is the number of subvolumes where mkdir lost the race).
   Both the clients "cache" their respective layouts in the near
   future for all future creates inside them (evidence in logs)

5. During the creation of the subdirectory, two clients race again.
   Ideally winner performs mkdir() on the hashed subvolume and proceeds
   to create other dirs, loser experiences EEXIST error on the hashed
   subvolume and backs off. But in this case, because the two clients
   have different layout views of the parent directory (because of
   different hash splits and assignements), the hashed subvolumes for
   the new directory can end up being different. Therefore, both clients
   now win the race (they were never fighting against each other on a
   common server), assigning different GFIDs to the directory on their
   respective (different) subvolumes. Some of the remaining subvolumes
   get GFID1, others GFID2.

Conclusion/Fix:
   Making mkdir translate EEXIST error as success (just the way self-heal
   is already rightly doing) will bring back truth to the design claim
   that concurrent mkdir/self-heals perform deterministic + idempotent
   operations. This will prevent the differing "hash views" by different
   clients and thereby also avoid GFID mismatch by forcing all clients
   to have a "fair race", because the hashed subvolume for all will be
   the same (and thereby avoiding leaked locks and hangs.)

Change-Id: I84592fb9b8a3f739a07e2afb23b33758a0a9a157
BUG: 907072
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/4459
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

cluster/dht: stack wind with cookie

2013-02-01T01:18:03+00:00

Default_fops uses stack_wind_tail. It winds without creating the frame leading
into wrong subvol return in the cookie. To avoid the problem caused by the
same, we're getting the subvol by passing the cookie.

Change-Id: I51ee79b22c89e4fb0b89e9a0bc3ac96c5b469f8f
BUG: 893338
Signed-off-by: Varun Shastry 
Reviewed-on: http://review.gluster.org/4388
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati 
Tested-by: Anand Avati

cluster/distribute: If cached_subvol is down, return ENOTCONN in lookup

2013-01-21T20:03:03+00:00

When we follow a linkfile, and the lookup returns a ENOTCONN error, return
the error, as the cached subvol is down, and lookup_everywhere wont succeed,
but actually ends up clearing the linkfile, and clearing the namespace.

Change-Id: I772bf71531bc646e8fb62d3e8549a5fe0f3896da
BUG: 893378
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4383
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: update ctx-time only if we receive the new iatt

2013-01-17T08:26:57+00:00

1. Used local->postparent(contains merged iatt of all succesful calls) instead
of postparent for dht ctx time update.

2. dht_inode_ctx_time_update avoided in case of opret -1.

Change-Id: Ie04a7842a41c241f911b6a3f76267b996d27fb43
BUG: 881013
Signed-off-by: Varun Shastry 
Reviewed-on: http://review.gluster.org/4338
Reviewed-by: Shishir Gowda 
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: fail fix-layout if any of the subvol is down

2012-11-30T06:11:06+00:00

If any subvolume is down, and a layout is re-written and hash
values change, entry names in the downed subvol can be reused
in the other subvol which got the same hash range. when the
downed subvol is brought back up, duplicate entried might appear

Also separated handling of ENOSPC and ENOTCONN error.

Change-Id: I5ed93990425a4cee70df2dab7c7c119fdc87ad56
BUG: 860663
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4000
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati