glusterfs.git/xlators/cluster/dht/src/dht-rename.c, branch v3.5qa2

cluster/dht: instruct marker whenever it shouldn't do accounting

2013-11-26T18:27:21+00:00

This is needed for two reasons:

* since dht-linkfiles are internal, they shouldn't be accounted.
* hardlink handling in marker is broken. link/unlink of hardlinks
  present in same directory can break marker accounting. Hence, if src
  and dst are in same directory in case of rename, dht - if it breaks
  rename into link/unlink operations - should instruct marker to not to
  do accounting.

Change-Id: I9c9f7384569f75a2792f6450ee7a5279bf751ae7
BUG: 1022995
Signed-off-by: Raghavendra G 
Reviewed-on: http://review.gluster.org/6203
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: mark linkfile creation/deletion as internal fop

2013-07-24T11:27:29+00:00

Currently dht creates/deletes linkfiles for various ops like
rename/linking and when layout changes. dht_linkfile_create
already sends a key GLUSTERFS_INTERNAL_FOP_KEY in dict to
identify this as an internal fop and not user based op.

Enhancing rename related links/unlinks to send this key too.
Marker/changelog or other xlators can now identify these as
internal fops and handle them accordingly

Change-Id: Ib1ca789e6dbce48703c55ad3f4f88f7cd2df3d06
BUG: 987428
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/5335
Reviewed-by: Amar Tumballi 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: Linkfiles creation with correct uid/gid

2013-05-31T13:15:15+00:00

If renames are done with different uid/gid (non-owners), then we would
end up with incorrect uid/gid.

The fix is to create linkfiles, and heal the uid/gid as root:root. This
preserves our notion of creation as root:root and heal the uid/gid as
root:root in all paths. Additionally, we need to consider uid/gid from
only src_cached subvol, and not from linkfiles.

rename is also done as root:root if done on linkfile, as setattr of ownership
on linkfile is done after the rename

Change-Id: Icb5d431dc42da9c02dfae81980e3fe769a47a274
BUG: 884597
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4682
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

dht: make DHT xattr names configurable

2013-03-21T22:04:31+00:00

This is necessary to support "DHT over DHT" configurations, so that the
upper and lower instances of DHT don't step all over each other.  Why
would we even consider such a thing?  Because it gives us the ability to
do data tiering and rack-aware placement, either by themselves or as
complements to other functionality such as erasure codes or
deduplication which save space but cost performance.  By setting up the
top-level DHT to place data into one of several lower-level DHT pools
based on policy instead of pure elastic hashing, we get better
performance for 90% of accesses and better storage efficiency for 90% of
data, all for relatively low effort.

Change-Id: I72e65c29edfc80babf39f7a2a00090f4588c4070
BUG: 924265
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.org/4694
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: Create linkfile with file uid/gid

2013-02-14T00:17:44+00:00

Currently, linkfile creation happens as root.

use uid/gid returned from _cbk (link/rename) to set the correct ownership of
the link files.

Also added test/dht.rc to implement common dht functions

Change-Id: Iecdcf52d89fed8286670ce430b9d19deebd27b8c
BUG: 884597
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4304
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

license: dual license under GPLV2 and LGPLV3+

2012-05-11T05:06:16+00:00

Note that the license was not changed in any of the following:
	.../argp-standalone/...
        .../booster/...
        .../cli/...
        .../contrib/...
        .../extras/...
        .../glusterfsd/...
        .../glusterfs-hadoop/...
        .../mod_clusterfs/...
        .../scheduler/...
        .../swift/...

The license was not changed in any of the non-building xlators. The
license was not changed in any of the xlators that seemed — to me — to
be clearly server-side only, e.g. protocol/server

Note too that copyright was changed along with the license; I did
not change the copyright in files where the license did not change.

If you find any errors or ommissions please don't hesitate to let me know.

The complete list of files with the license change is:
	libglusterfs/src/byte-order.h
	libglusterfs/src/call-stub.c
	libglusterfs/src/call-stub.h
	libglusterfs/src/checksum.c
	libglusterfs/src/checksum.h
	libglusterfs/src/circ-buff.c
	libglusterfs/src/circ-buff.h
	libglusterfs/src/common-utils.c
	libglusterfs/src/common-utils.h
	libglusterfs/src/compat-errno.c
	libglusterfs/src/compat-errno.h
	libglusterfs/src/compat.c
	libglusterfs/src/compat.h
	libglusterfs/src/daemon.c
	libglusterfs/src/daemon.h
	libglusterfs/src/defaults.c
	libglusterfs/src/defaults.h
	libglusterfs/src/dict.c
	libglusterfs/src/dict.h
	libglusterfs/src/event-history.c
	libglusterfs/src/event-history.h
	libglusterfs/src/event.c
	libglusterfs/src/event.h
	libglusterfs/src/fd-lk.c
	libglusterfs/src/fd-lk.h
	libglusterfs/src/fd.c
	libglusterfs/src/fd.h
	libglusterfs/src/gf-dirent.c
	libglusterfs/src/gf-dirent.h
	libglusterfs/src/globals.c
	libglusterfs/src/globals.h
	libglusterfs/src/glusterfs.h
	libglusterfs/src/graph-print.c
	libglusterfs/src/graph-utils.h
	libglusterfs/src/graph.c
	libglusterfs/src/hashfn.c
	libglusterfs/src/hashfn.h
	libglusterfs/src/iatt.h
	libglusterfs/src/inode.c
	libglusterfs/src/inode.h
	libglusterfs/src/iobuf.c
	libglusterfs/src/iobuf.h
	libglusterfs/src/latency.c
	libglusterfs/src/latency.h
	libglusterfs/src/list.h
	libglusterfs/src/lkowner.h
	libglusterfs/src/locking.h
	libglusterfs/src/logging.c
	libglusterfs/src/logging.h
	libglusterfs/src/mem-pool.c
	libglusterfs/src/mem-pool.h
	libglusterfs/src/mem-types.h
	libglusterfs/src/options.c
	libglusterfs/src/options.h
	libglusterfs/src/rbthash.c
	libglusterfs/src/rbthash.h
	libglusterfs/src/run.c
	libglusterfs/src/run.h
	libglusterfs/src/scheduler.c
	libglusterfs/src/scheduler.h
	libglusterfs/src/stack.c
	libglusterfs/src/stack.h
	libglusterfs/src/statedump.c
	libglusterfs/src/statedump.h
	libglusterfs/src/syncop.c
	libglusterfs/src/syncop.h
	libglusterfs/src/syscall.c
	libglusterfs/src/syscall.h
	libglusterfs/src/timer.c
	libglusterfs/src/timer.h
	libglusterfs/src/trie.c
	libglusterfs/src/trie.h
	libglusterfs/src/xlator.c
	libglusterfs/src/xlator.h
	libglusterfsclient/src/libglusterfsclient-dentry.c
	libglusterfsclient/src/libglusterfsclient-internals.h
	libglusterfsclient/src/libglusterfsclient.c
	libglusterfsclient/src/libglusterfsclient.h
	rpc/rpc-lib/src/auth-glusterfs.c
	rpc/rpc-lib/src/auth-null.c
	rpc/rpc-lib/src/auth-unix.c
	rpc/rpc-lib/src/protocol-common.h
	rpc/rpc-lib/src/rpc-clnt.c
	rpc/rpc-lib/src/rpc-clnt.h
	rpc/rpc-lib/src/rpc-transport.c
	rpc/rpc-lib/src/rpc-transport.h
	rpc/rpc-lib/src/rpcsvc-auth.c
	rpc/rpc-lib/src/rpcsvc-common.h
	rpc/rpc-lib/src/rpcsvc.c
	rpc/rpc-lib/src/rpcsvc.h
	rpc/rpc-lib/src/xdr-common.h
	rpc/rpc-lib/src/xdr-rpc.c
	rpc/rpc-lib/src/xdr-rpc.h
	rpc/rpc-lib/src/xdr-rpcclnt.c
	rpc/rpc-lib/src/xdr-rpcclnt.h
	rpc/rpc-transport/rdma/src/name.c
	rpc/rpc-transport/rdma/src/name.h
	rpc/rpc-transport/rdma/src/rdma.c
	rpc/rpc-transport/rdma/src/rdma.h
	rpc/rpc-transport/socket/src/name.c
	rpc/rpc-transport/socket/src/name.h
	rpc/rpc-transport/socket/src/socket.c
	rpc/rpc-transport/socket/src/socket.h
	xlators/cluster/afr/src/afr-common.c
	xlators/cluster/afr/src/afr-dir-read.c
	xlators/cluster/afr/src/afr-dir-read.h
	xlators/cluster/afr/src/afr-dir-write.c
	xlators/cluster/afr/src/afr-dir-write.h
	xlators/cluster/afr/src/afr-inode-read.c
	xlators/cluster/afr/src/afr-inode-read.h
	xlators/cluster/afr/src/afr-inode-write.c
	xlators/cluster/afr/src/afr-inode-write.h
	xlators/cluster/afr/src/afr-lk-common.c
	xlators/cluster/afr/src/afr-mem-types.h
	xlators/cluster/afr/src/afr-open.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.h
	xlators/cluster/afr/src/afr-self-heal-common.c
	xlators/cluster/afr/src/afr-self-heal-common.h
	xlators/cluster/afr/src/afr-self-heal-data.c
	xlators/cluster/afr/src/afr-self-heal-entry.c
	xlators/cluster/afr/src/afr-self-heal-metadata.c
	xlators/cluster/afr/src/afr-self-heal.h
	xlators/cluster/afr/src/afr-self-heald.c
	xlators/cluster/afr/src/afr-self-heald.h
	xlators/cluster/afr/src/afr-transaction.c
	xlators/cluster/afr/src/afr-transaction.h
	xlators/cluster/afr/src/afr.c
	xlators/cluster/afr/src/afr.h
	xlators/cluster/afr/src/pump.c
	xlators/cluster/afr/src/pump.h
	xlators/cluster/dht/src/dht-common.c
	xlators/cluster/dht/src/dht-common.h
	xlators/cluster/dht/src/dht-diskusage.c
	xlators/cluster/dht/src/dht-hashfn.c
	xlators/cluster/dht/src/dht-helper.c
	xlators/cluster/dht/src/dht-inode-read.c
	xlators/cluster/dht/src/dht-inode-write.c
	xlators/cluster/dht/src/dht-layout.c
	xlators/cluster/dht/src/dht-linkfile.c
	xlators/cluster/dht/src/dht-mem-types.h
	xlators/cluster/dht/src/dht-rebalance.c
	xlators/cluster/dht/src/dht-rename.c
	xlators/cluster/dht/src/dht-selfheal.c
	xlators/cluster/dht/src/dht.c
	xlators/cluster/dht/src/nufa.c
	xlators/cluster/dht/src/switch.c
	xlators/cluster/stripe/src/stripe-helpers.c
	xlators/cluster/stripe/src/stripe-mem-types.h
	xlators/cluster/stripe/src/stripe.c
	xlators/cluster/stripe/src/stripe.h
	xlators/features/index/src/index-mem-types.h ¹
	xlators/features/index/src/index.c ¹
	xlators/features/index/src/index.h ¹
	xlators/performance/io-cache/src/io-cache.c
	xlators/performance/io-cache/src/io-cache.h
	xlators/performance/io-cache/src/ioc-inode.c
	xlators/performance/io-cache/src/ioc-mem-types.h
	xlators/performance/io-cache/src/page.c
	xlators/performance/io-threads/src/io-threads.c
	xlators/performance/io-threads/src/io-threads.h
	xlators/performance/io-threads/src/iot-mem-types.h
	xlators/performance/md-cache/src/md-cache-mem-types.h
	xlators/performance/md-cache/src/md-cache.c
	xlators/performance/quick-read/src/quick-read-mem-types.h
	xlators/performance/quick-read/src/quick-read.c
	xlators/performance/quick-read/src/quick-read.h
	xlators/performance/read-ahead/src/page.c
	xlators/performance/read-ahead/src/read-ahead-mem-types.h
	xlators/performance/read-ahead/src/read-ahead.c
	xlators/performance/read-ahead/src/read-ahead.h
	xlators/performance/symlink-cache/src/symlink-cache.c
	xlators/performance/write-behind/src/write-behind-mem-types.h
	xlators/performance/write-behind/src/write-behind.c
	xlators/protocol/auth/addr/src/addr.c ¹
	xlators/protocol/auth/login/src/login.c ¹
	xlators/protocol/client/src/client-callback.c
	xlators/protocol/client/src/client-handshake.c
	xlators/protocol/client/src/client-helpers.c
	xlators/protocol/client/src/client-lk.c
	xlators/protocol/client/src/client-mem-types.h
	xlators/protocol/client/src/client.c
	xlators/protocol/client/src/client.h
	xlators/protocol/client/src/client3_1-fops.c

¹ Copyright only, license reverted to original

Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8
BUG: 820551
Signed-off-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.com/3304
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

core: adding extra data for fops

2012-03-22T23:40:27+00:00

with this change, the xlator APIs will have a dictionary as extra
argument, which is passed between all the layers. This can be
utilized for overloading in some of the operations.

Change-Id: I58a8186b3ef647650280e63f3e5e9b9de7827b40
Signed-off-by: Amar Tumballi 
BUG: 782265
Reviewed-on: http://review.gluster.com/2960
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: handle ENOENT err in rename fop

2012-01-25T10:18:04+00:00

A ENOENT should not be a error propogated for rename failures.
As, ENOENT can arise only due to internal unlink call of rename.

Change-Id: I925622da8ef370d0385bc5b30cf8dc9b8e852beb
BUG: 768879
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.com/2583
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi 
Reviewed-by: Anand Avati

cluster/distribute: minor fixes in open file migration

2011-09-20T06:49:10+00:00

* incorporated Avati's comments on the first patch.
* send proper stat information while unwinding

Change-Id: I36982cec610753c241c372272620ab2bd581fd9f
BUG: 3071
Reviewed-on: http://review.gluster.com/408
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

distribute rebalance: handle the open file migration

2011-09-13T06:57:38+00:00

Complexity involved: To migrate a file with open fd, we have to
notify the other client process which has the open fd, and make
sure the write()s happening on that fd is properly synced to the
migrated file. Once the migration is complete, the client
process which has open-fd should get notified and it should
start performing all the operations on the new subvolume,
instead of earlier cached volume.

How to solve the notification part:

We can overload the 'postbuf' attribute in the _cbk() function to
understand if a file is 'under-migration' or 'migration-complete'
state. (This will be something similar to deciding whether a file
is DHT-linkfile by its 'mode').

Overall change includes below mentioned major changes:

1. dht_linkfile is decided by only 2 factors (mode(01000),
   xattr(trusted.glusterfs.dht.linkto)), instead of earlier
   3 factors (size==0)

2. in linkfile self-heal part (in 'dht_lookup_everywhere_cbk()'),
   don't delete a linkfile if there is a open-fd on it. It means,
   there may be a migration in progress.

3. if a file's revalidate fails with ENOENT, it may be due to file
   migration, and hence need a lookup_everywhere()

4. There will be 2 phases of file-migration.

-> Phase 1: Migration in progress
   * The source data file will have SGID and STICKY bit set in its mode.
   * The source data file will have a 'linkto' xattr pointing the
     destination.
   * Destination file will have mode set to '01000', and 'linkto' xattr
     set to itself.

-> Phase 2: File migration Complete
   * The source data file will have mode '01000', and will be 'truncated'
     to size 0.
   * The destination file will have inherited mode from the source. (without
     sgid and sticky bit) and its 'linkto' attribute will be removed.

4. Changes in distribute to work smoothly with a file which is in migration /
   got migrated.

The 'fops' are divided into 3 categories, inode-read, inode-write and others.
inode-read fops need to handle only 'phase 2' notification, where as, the
inode-write fops need to handle both 'phase 1' and phase2. The inode-write
operations will be done on source file, and if any of 'file-migration' procedures
are detected in _cbk(), then the operations should be performed on the destination
too.

when a phase-2 is detected, then the inode-ctx itself should be changed to represent
a new layout.

With these changes, the open file migration will work smoothly with multiple clients.

Change-Id: I512408463814e650f34c62ed009bf2101d016fd6
BUG: 3071
Reviewed-on: http://review.gluster.com/209
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur