glusterfs.git/libglusterfs/src, branch v3.3.0qa20

pump: move internal pump xattrs out of trusted domain

2012-01-23T17:46:43+00:00

* the trusted.glusterfs.pump.{start|pause|commit|status|abort} xattrs
  have been moved out of trusted domain. This enables separation of
  xattrs used as gluster-internal commands (handled by pump) for
  replace-brick, which are not set in the back-end, from xattrs set
  on the replace-brick source and destinations bricks.

* macros definitions from pump.h and glusterd.h, #defining these xattrs
  have been merged and put into libglusterfs/src/glusterfs.h

Change-Id: I87b8bfbf045aa140f5d3f0c9baa9b2e79f87b67b
BUG: 783049
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.com/2663
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Vijay Bellur

core: GFID filehandle based backend and anonymous FDs

2012-01-20T13:03:42+00:00

1. What
--------
This change introduces an infrastructure change in the filesystem
which lets filesystem operation address objects (inodes) just by its
GFID. Thus far GFID has been a unique identifier of a user-visible
inode. But in terms of addressability the only mechanism thus far has
been the backend filesystem path, which could be derived from the
GFID only if it was cached in the inode table along with the entire set
of dentry ancestry leading up to the root.

This change essentially decouples addressability from the namespace. It
is no more necessary to be aware of the parent directory to address a
file or directory.

2. Why
-------
The biggest use case for such a feature is NFS for generating
persistent filehandles. So far the technique for generating filehandles
in NFS has been to encode path components so that the appropriate
inode_t can be repopulated into the inode table by means of a recursive
lookup of each component top-down.

Another use case is the ability to perform more intelligent self-healing
and rebalancing of inodes with hardlinks and also to detect renames.

A derived feature from GFID filehandles is anonymous FDs. An anonymous FD
is an internal USABLE "fd_t" which does not map to a user opened file
descriptor or to an internal ->open()'d fd. The ability to address a file
by the GFID eliminates the need to have a persistent ->open()'d fd for the
purpose of avoiding the namespace. This improves NFS read/write performance
significantly eliminating open/close calls and also fixes some of today's
limitations (like keeping an FD open longer than necessary resulting
in disk space leakage)

3. How
-------

At each storage/posix translator level, every file is hardlinked inside
a hidden .glusterfs directory (under the top level export) with the name
as the ascii-encoded standard UUID format string. For reasons of performance
and scalability there is a two-tier classification of those hardlinks
under directories with the initial parts of the UUID string as the directory
names.

For directories (which cannot be hardlinked), the approach is to use a symlink
which dereferences the parent GFID path along with basename of the directory.
The parent GFID dereference will in turn be a dereference of the grandparent
with the parent's basename, and so on recursively up to the root export.

4. Development
---------------

4a. To leverage the ability to address an inode by its GFID, the technique is
to perform a "nameless lookup". This means, to populate a loc_t structure as:

loc_t {
   pargfid: NULL
   parent: NULL
   name: NULL
   path: NULL
   gfid: GFID to be looked up [out parameter]
   inode: inode_new () result [in parameter]
}

and performing such lookup will return in its callback an inode_t
populated with the right contexts and a struct iatt which can be
used to perform an inode_link () on the inode (without a parent and
basename). The inode will now be hashed and linked in the inode table
and findable via inode_find().

A fundamental change moving forward is that the primary fields in a
loc_t structure are now going to be (pargfid, name) and (gfid) depending
on the kind of FOP. So far path had been the primary field for operations.
The remaining fields only serve as hints/helpers.

4b. If read/write is to be performed on an inode_t, the approach so far
has been to: fd_create(), STACK_WIND(open, fd), fd_bind (in callback) and
then perform STACK_WIND(read, fd) etc. With anonymous fds now you can do
fd_anonymous (inode), STACK_WIND (read, fd). This results in great boost
in performance in the inbuilt NFS server.

5. Misc
-------
The inode_ctx_put[2] has been renamed to inode_ctx_set[2] to be consistent
with the rest of the codebase.

Change-Id: Ie4629edf6bd32a595f4d7f01e90c0a01f16fb12f
BUG: 781318
Reviewed-on: http://review.gluster.com/669
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

core/setxattr: prevent users from setting glusterfs xattrs

2012-01-14T12:57:59+00:00

* Each xlator prevents the user from setting glusterfs-internal
xattrs like trusted.gfid by handling it in respective setxattr
functions. The speacial case of trusted.gfid is handled in
fuse (Not in posix because posix_setxattr is used to set gfid).

* For xlators which did not define setxattr and/or fsetxattr,
the functions have been implemented with appropriate checks.

xlator         |     fops-added
_______________|__________________________
               |
1. afr         |     fsetxattr
2. stripe      |     setxatrr and fsetxattr
3. quota       |     setxattr and fsetxattr

Change-Id: Ib62abb7067415b23a708002f884d30e8866fbf48
BUG: 765487
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.com/685
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

cluster/distribute: dht_aggregate() fix a logic error before xattr comparisons

2012-01-13T05:47:06+00:00

Change-Id: I20f015263bed9851225005d5f41a5d518bd22592
BUG: 769691
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.com/2557
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

common-utils: Cleanup old stuff and standardize

2012-01-13T05:40:36+00:00

Change-Id: If15058c3a1cc4070d1ebbabe37e012a70353310e
BUG: 769691
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.com/2635
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Vijay Bellur

cli: volume status enhancement

2012-01-12T09:58:26+00:00

    * Support "gluster volume status (all)" option to display all
      volumes' status.

    * On option "detail" appended to "gluster volume status *",
      amount of storage free, total storage, and backend filesystem
      details like inode size, inode count, free inodes, fs type,
      device name of each brick is displayed.

    * One can also obtain [detailed]status of only one brick.

    * Format of the enhanced volume status command is:
      "gluster volume status [all|] [] [detail]"

    * Some generic functions have been added to common-utils:

           skipword
           get_nth_word
      These functions enable parsing and fetching
      of words in a sentence.

           glusterd_get_brick_root    (in glusterd)
      These are self explanatory.

Change-Id: I6f40c1e19810f8504cd3b1786207364053d82e28
BUG: 765464
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.com/777
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

core/stack: perform locked winds

2012-01-11T07:52:58+00:00

In configurations like pump, where there is a cluster translator on
top of io-threads, there are situations where two concurrent stack-winds
can be performed on the same call stack in multiple threads. This patch
holds locks during the call frame list manipulation

Change-Id: I51539210dc8101f7a80cf9bc103b5eff0c86dc9f
BUG: 765522
Reviewed-on: http://review.gluster.com/774
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

lib/mount-broker: move utility functions to common-utils

2012-01-05T16:51:22+00:00

functions skipwhite and nwstrtail have been moved from
mount-broker to common-utils library for general use.

Change-Id: I9cfefb28bbfcf5d0bd37e35865ff3f3b7923fc53
BUG: 765464
Signed-off-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.com/2560
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur 
Reviewed-by: Csaba Henk

libglusterfs: Add boundary conditions to data_to_int8() function.

2012-01-03T21:29:37+00:00

Change-Id: Iff50d44568895a75fc8743a10346990f764cf8fa
BUG: 769692
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.com/2556
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Vijay Bellur

libglusterfs: set inode path to be NULL in case of errors

2011-12-22T13:14:40+00:00

Some of the functions calling inode_path, __inode_path
are assuming the path to be set to NULL in case of errors.
Instead of fixing it in the calling functions, fixing it in
the __inode_path function.

Change-Id: I77736a2700d3c2c9732a536bcf2a398fe626d54e
BUG: 765430
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.com/810
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy 
Reviewed-by: Vijay Bellur