summaryrefslogtreecommitdiffstats
path: root/xlators/cluster/dht/src/dht-rebalance.c
Commit message (Collapse)AuthorAgeFilesLines
* cluster/dht: Add "afr.readdir-failover=off" option the rebalance processshishir gowda2012-12-161-7/+27
| | | | | | | | | | | | | | | | | By failing over readdir (default behaviour), rebalance could get duplicate files, as readdir would re-read from offset 0. Rebalance should not attempt to migrate these files again. Additionally, we need to handle these cases as failure in rebalance crawl. No test case provided, as we cannot determine the read child in afr. Change-Id: If07508b4f92dacc17e0f695b48a866c7c66004be BUG: 859387 Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.org/4300 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* cluster/dht: fail fix-layout if any of the subvol is downshishir gowda2012-11-291-0/+3
| | | | | | | | | | | | | | | | If any subvolume is down, and a layout is re-written and hash values change, entry names in the downed subvol can be reused in the other subvol which got the same hash range. when the downed subvol is brought back up, duplicate entried might appear Also separated handling of ENOSPC and ENOTCONN error. Change-Id: I5ed93990425a4cee70df2dab7c7c119fdc87ad56 BUG: 860663 Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.org/4000 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* Fixed some general typing errors.Varun Shastry2012-09-271-1/+1
| | | | | | | | | | | | Eg: changed recieved to received Change-Id: I360fcb99c97c8a0222e373fee20ea2fccfb938db BUG: 860543 Signed-off-by: Varun Shastry <vshastry@redhat.com> Reviewed-on: http://review.gluster.org/3998 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* dht/rebalance: set the correct ownership on the dst file.shishir gowda2012-08-281-0/+8
| | | | | | | | | | | | | | | Currently, the dst file created has root:root ownership, till migration is completed. During this phase, open fails on the dst file if uid/gid is non-root. Setting the dst_file to the correct ownership fixes the issue Change-Id: Icfec89eb10dc866cdee38dab17695fe21174ef99 BUG: 852361 Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.org/3861 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* syncop: handle 'dataonly' flag in syncop_fsync()Amar Tumballi2012-08-201-1/+1
| | | | | | | | | | | | * and also in syncop_readv(), don't look at _cbk args if op_ret is < 0. Change-Id: I3ab2982bc6d186e75b6adb74c8981e4ff7058bbe Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 839950 Reviewed-on: http://review.gluster.org/3828 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* glusterfs_ctx_t: un-globalize the filesystem contextAnand Avati2012-07-171-6/+4
| | | | | | | | | | | | | | | | | | | | | | | | So far there has been a global glusterfs_ctx_t object which represents the running instance of the filesystem (client or server). It contains the various graphs, connection to the management daemon over which new graphs are obtained, calls stacks issued on this filesystem, and a bunch of such things. With the introduction of libgfapi, it is no more true that there will be only one filesystem context in a process. Applications can be written to use libgfapi and obtain serveral instances of different filesystems/volumes in the same process. This involves messy untangling of assumptions inside libglusterfs that there would only be one global glusterfs_ctx_t and offload that assumption to glusterfsd/ and cli/ (where it is true). Change-Id: Ifd7d1259428c26076140a5764a2dc7361694139c BUG: 839950 Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.com/3678 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* remove useless if-before-free (and free-like) functionsJim Meyering2012-07-131-4/+2
| | | | | | | | | | | | See comments in http://bugzilla.redhat.com/839925 for the code to perform this change. Signed-off-by: Jim Meyering <meyering@redhat.com> BUG: 839925 Change-Id: I10e4ecff16c3749fe17c2831c516737e08a3205a Reviewed-on: http://review.gluster.com/3661 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* cluster/dht: Remove dht dependency on glusterfsd-mgmtshishir gowda2012-06-291-1/+7
| | | | | | | | | | | | | | | glusterfs_ctx->notify can be used by any xlator to talk to glusterfsd-mgmt. Note- This is for any rpc communication initiated by the xlator, and not from glusterd. Change-Id: Ic0e4af106fe1e98d797ca621facda8839b87598a BUG: 835757 Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.com/3618 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* cluster/dht: set conf->defrag to NULL after freeing the defrag structureRaghavendra Bhat2012-05-311-2/+3
| | | | | | | | | | | | | Also no need to free the xlator object after rebalance is over, as the process is about to be killed. Change-Id: I6973e43c0353b5de61c0b39e52a22c618be361f4 BUG: 826584 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> Reviewed-on: http://review.gluster.com/3495 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* glusterd/rebalance: Display run-time of rebalance processshishir gowda2012-05-241-2/+41
| | | | | | | | | | | | | | | | | Added run-time value field to cli output of rebalance/remove-brick. A new cluster/distribute boolean option rebalance-stats when set to ON, time taken for migration of each file is logged. With rebalance-stats OFF (default), rebalance logs will only have entries showing time spent in each directory. Change-Id: I02a8918621120068cd71ffaf2999d30b3a2d10a2 BUG: 821987 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3303 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* distribute: use global synctask 'env' instead of localAmar Tumballi2012-05-231-4/+1
| | | | | | | | | | | | | | creating a local synctask_env can lead to creating of many more syncop threads than required. The current syncop logic can handle the scale-up/scale-down of threads depending on the load. Hence, its neater to use global synctask env. Change-Id: Id46f963a0190c0154513317ae03323db155ac15a Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 823774 Reviewed-on: http://review.gluster.com/3412 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* license: dual license under GPLV2 and LGPLV3+Kaleb KEITHLEY2012-05-101-14/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Note that the license was not changed in any of the following: .../argp-standalone/... .../booster/... .../cli/... .../contrib/... .../extras/... .../glusterfsd/... .../glusterfs-hadoop/... .../mod_clusterfs/... .../scheduler/... .../swift/... The license was not changed in any of the non-building xlators. The license was not changed in any of the xlators that seemed — to me — to be clearly server-side only, e.g. protocol/server Note too that copyright was changed along with the license; I did not change the copyright in files where the license did not change. If you find any errors or ommissions please don't hesitate to let me know. The complete list of files with the license change is: libglusterfs/src/byte-order.h libglusterfs/src/call-stub.c libglusterfs/src/call-stub.h libglusterfs/src/checksum.c libglusterfs/src/checksum.h libglusterfs/src/circ-buff.c libglusterfs/src/circ-buff.h libglusterfs/src/common-utils.c libglusterfs/src/common-utils.h libglusterfs/src/compat-errno.c libglusterfs/src/compat-errno.h libglusterfs/src/compat.c libglusterfs/src/compat.h libglusterfs/src/daemon.c libglusterfs/src/daemon.h libglusterfs/src/defaults.c libglusterfs/src/defaults.h libglusterfs/src/dict.c libglusterfs/src/dict.h libglusterfs/src/event-history.c libglusterfs/src/event-history.h libglusterfs/src/event.c libglusterfs/src/event.h libglusterfs/src/fd-lk.c libglusterfs/src/fd-lk.h libglusterfs/src/fd.c libglusterfs/src/fd.h libglusterfs/src/gf-dirent.c libglusterfs/src/gf-dirent.h libglusterfs/src/globals.c libglusterfs/src/globals.h libglusterfs/src/glusterfs.h libglusterfs/src/graph-print.c libglusterfs/src/graph-utils.h libglusterfs/src/graph.c libglusterfs/src/hashfn.c libglusterfs/src/hashfn.h libglusterfs/src/iatt.h libglusterfs/src/inode.c libglusterfs/src/inode.h libglusterfs/src/iobuf.c libglusterfs/src/iobuf.h libglusterfs/src/latency.c libglusterfs/src/latency.h libglusterfs/src/list.h libglusterfs/src/lkowner.h libglusterfs/src/locking.h libglusterfs/src/logging.c libglusterfs/src/logging.h libglusterfs/src/mem-pool.c libglusterfs/src/mem-pool.h libglusterfs/src/mem-types.h libglusterfs/src/options.c libglusterfs/src/options.h libglusterfs/src/rbthash.c libglusterfs/src/rbthash.h libglusterfs/src/run.c libglusterfs/src/run.h libglusterfs/src/scheduler.c libglusterfs/src/scheduler.h libglusterfs/src/stack.c libglusterfs/src/stack.h libglusterfs/src/statedump.c libglusterfs/src/statedump.h libglusterfs/src/syncop.c libglusterfs/src/syncop.h libglusterfs/src/syscall.c libglusterfs/src/syscall.h libglusterfs/src/timer.c libglusterfs/src/timer.h libglusterfs/src/trie.c libglusterfs/src/trie.h libglusterfs/src/xlator.c libglusterfs/src/xlator.h libglusterfsclient/src/libglusterfsclient-dentry.c libglusterfsclient/src/libglusterfsclient-internals.h libglusterfsclient/src/libglusterfsclient.c libglusterfsclient/src/libglusterfsclient.h rpc/rpc-lib/src/auth-glusterfs.c rpc/rpc-lib/src/auth-null.c rpc/rpc-lib/src/auth-unix.c rpc/rpc-lib/src/protocol-common.h rpc/rpc-lib/src/rpc-clnt.c rpc/rpc-lib/src/rpc-clnt.h rpc/rpc-lib/src/rpc-transport.c rpc/rpc-lib/src/rpc-transport.h rpc/rpc-lib/src/rpcsvc-auth.c rpc/rpc-lib/src/rpcsvc-common.h rpc/rpc-lib/src/rpcsvc.c rpc/rpc-lib/src/rpcsvc.h rpc/rpc-lib/src/xdr-common.h rpc/rpc-lib/src/xdr-rpc.c rpc/rpc-lib/src/xdr-rpc.h rpc/rpc-lib/src/xdr-rpcclnt.c rpc/rpc-lib/src/xdr-rpcclnt.h rpc/rpc-transport/rdma/src/name.c rpc/rpc-transport/rdma/src/name.h rpc/rpc-transport/rdma/src/rdma.c rpc/rpc-transport/rdma/src/rdma.h rpc/rpc-transport/socket/src/name.c rpc/rpc-transport/socket/src/name.h rpc/rpc-transport/socket/src/socket.c rpc/rpc-transport/socket/src/socket.h xlators/cluster/afr/src/afr-common.c xlators/cluster/afr/src/afr-dir-read.c xlators/cluster/afr/src/afr-dir-read.h xlators/cluster/afr/src/afr-dir-write.c xlators/cluster/afr/src/afr-dir-write.h xlators/cluster/afr/src/afr-inode-read.c xlators/cluster/afr/src/afr-inode-read.h xlators/cluster/afr/src/afr-inode-write.c xlators/cluster/afr/src/afr-inode-write.h xlators/cluster/afr/src/afr-lk-common.c xlators/cluster/afr/src/afr-mem-types.h xlators/cluster/afr/src/afr-open.c xlators/cluster/afr/src/afr-self-heal-algorithm.c xlators/cluster/afr/src/afr-self-heal-algorithm.h xlators/cluster/afr/src/afr-self-heal-common.c xlators/cluster/afr/src/afr-self-heal-common.h xlators/cluster/afr/src/afr-self-heal-data.c xlators/cluster/afr/src/afr-self-heal-entry.c xlators/cluster/afr/src/afr-self-heal-metadata.c xlators/cluster/afr/src/afr-self-heal.h xlators/cluster/afr/src/afr-self-heald.c xlators/cluster/afr/src/afr-self-heald.h xlators/cluster/afr/src/afr-transaction.c xlators/cluster/afr/src/afr-transaction.h xlators/cluster/afr/src/afr.c xlators/cluster/afr/src/afr.h xlators/cluster/afr/src/pump.c xlators/cluster/afr/src/pump.h xlators/cluster/dht/src/dht-common.c xlators/cluster/dht/src/dht-common.h xlators/cluster/dht/src/dht-diskusage.c xlators/cluster/dht/src/dht-hashfn.c xlators/cluster/dht/src/dht-helper.c xlators/cluster/dht/src/dht-inode-read.c xlators/cluster/dht/src/dht-inode-write.c xlators/cluster/dht/src/dht-layout.c xlators/cluster/dht/src/dht-linkfile.c xlators/cluster/dht/src/dht-mem-types.h xlators/cluster/dht/src/dht-rebalance.c xlators/cluster/dht/src/dht-rename.c xlators/cluster/dht/src/dht-selfheal.c xlators/cluster/dht/src/dht.c xlators/cluster/dht/src/nufa.c xlators/cluster/dht/src/switch.c xlators/cluster/stripe/src/stripe-helpers.c xlators/cluster/stripe/src/stripe-mem-types.h xlators/cluster/stripe/src/stripe.c xlators/cluster/stripe/src/stripe.h xlators/features/index/src/index-mem-types.h ¹ xlators/features/index/src/index.c ¹ xlators/features/index/src/index.h ¹ xlators/performance/io-cache/src/io-cache.c xlators/performance/io-cache/src/io-cache.h xlators/performance/io-cache/src/ioc-inode.c xlators/performance/io-cache/src/ioc-mem-types.h xlators/performance/io-cache/src/page.c xlators/performance/io-threads/src/io-threads.c xlators/performance/io-threads/src/io-threads.h xlators/performance/io-threads/src/iot-mem-types.h xlators/performance/md-cache/src/md-cache-mem-types.h xlators/performance/md-cache/src/md-cache.c xlators/performance/quick-read/src/quick-read-mem-types.h xlators/performance/quick-read/src/quick-read.c xlators/performance/quick-read/src/quick-read.h xlators/performance/read-ahead/src/page.c xlators/performance/read-ahead/src/read-ahead-mem-types.h xlators/performance/read-ahead/src/read-ahead.c xlators/performance/read-ahead/src/read-ahead.h xlators/performance/symlink-cache/src/symlink-cache.c xlators/performance/write-behind/src/write-behind-mem-types.h xlators/performance/write-behind/src/write-behind.c xlators/protocol/auth/addr/src/addr.c ¹ xlators/protocol/auth/login/src/login.c ¹ xlators/protocol/client/src/client-callback.c xlators/protocol/client/src/client-handshake.c xlators/protocol/client/src/client-helpers.c xlators/protocol/client/src/client-lk.c xlators/protocol/client/src/client-mem-types.h xlators/protocol/client/src/client.c xlators/protocol/client/src/client.h xlators/protocol/client/src/client3_1-fops.c ¹ Copyright only, license reverted to original Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8 BUG: 820551 Signed-off-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.com/3304 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* glusterd/rebalance: Remove state dependency on rpc eventshishir gowda2012-04-251-1/+28
| | | | | | | | | | | | | Status is updated by event_notify handshake by the rebalance process. If there is disconnect, before rebalance process, only then the state is moved to failure. Change-Id: I334055397c3d38688dbef653826bc07ac47a1911 BUG: 813167 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3165 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* rebalance: handshake_event_notify to make fsd talk to glusterdshishir gowda2012-04-251-1/+7
| | | | | | | | | | | | | | | | | | | Event_notify can be used by others to communicate with glusterd. A cbk event is also added for future use. req has a op, and dict. rsp has op_ret, op_errno, and dict. With this, rebalance process can update the status before exiting. Signed-off-by: shishir gowda <shishirng@gluster.com> Change-Id: If5c0ec00514eb3a109a790b2ea273317611e4562 BUG: 807126 Reviewed-on: http://review.gluster.com/3013 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* glusterd/rebalance: Start process with xlator option client-pid -3shishir gowda2012-04-181-0/+2
| | | | | | | | | | | | | | This would prevent updation of xtime by the marker. Set frame->root->pid while creating sync-task. Removing the command option, as it is in-effective Change-Id: Ifaa587df67a72ec0571c7d0043de8048e2be12a8 BUG: 812287 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3180 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* cluster/dht: Revert to calling ftruncateshishir gowda2012-04-161-5/+3
| | | | | | | | | | | Additionally, change lookup failure error to DEBUG level. Change-Id: I6eaf8e92a9cd34eba125a1e0e183bc3b6dc71945 BUG: 812769 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3155 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* dht/rebalance: Display migration failure countshishir gowda2012-04-131-2/+9
| | | | | | | | | | Change-Id: Ib2f8be2c98eae1b4df5d2090c1f1eec8ab264a35 BUG: 811923 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3135 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* dht/rebalance: Handle ASSERT_ON_CHILD_DOWN gracefullyshishir gowda2012-04-131-3/+11
| | | | | | | | | | | | | | | | In rebalance process, instead of terminating, send a stop event. The migration in question will either complete or be handled as error. Also, handle few syncop calls as errors, instead of just logging them. Change-Id: If6ed54474cb0f1fe0e28a5765e6d90966740dfde BUG: 811444 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3120 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* dht/rebalance: Correct log messagesv3.3.0beta3shishir gowda2012-04-111-5/+6
| | | | | | | | | | Change-Id: Ica73707922c1bf1a0975a8057d39c1441b987b0a BUG: 808376 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3127 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* rebalance: revert sending PARENT_DOWN event to xlatorsshishir gowda2012-04-091-2/+0
| | | | | | | | | | | reverting patch 1043dedfb54c80bdbb6e050920ced204515b226e Change-Id: I2e919c8a5e43f3cce74a8eaf58554830fa708364 BUG: 808402 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3107 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* dht/rebalance: Send PARENT_DOWN event before cleanup in rebalancev3.3.0qa33shishir gowda2012-04-021-1/+3
| | | | | | | | | | | | To prevent any crashes, due to backend ops (e.g: afr self heal), sending GF_EVENT_PARENT_DOWN before calling cleanup_and_exit. Change-Id: I58c87c114bf259a69a8d081d291ca908b30ec407 BUG: 808402 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/3062 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* core: adding extra data for fopsAmar Tumballi2012-03-221-1/+1
| | | | | | | | | | | | | with this change, the xlator APIs will have a dictionary as extra argument, which is passed between all the layers. This can be utilized for overloading in some of the operations. Change-Id: I58a8186b3ef647650280e63f3e5e9b9de7827b40 Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 782265 Reviewed-on: http://review.gluster.com/2960 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* distribute-rebalance: fix the logic of ENOENT handlingAmar Tumballi2012-03-141-3/+9
| | | | | | | | | | | | due to this, we were missing the entries from the last subvolume of distribute for all the rebalance or remove-brick operations Change-Id: I486f15c1d3d17cbf6f353f6179b2623af61f4868 Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 770346 Reviewed-on: http://review.gluster.com/2933 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shishir Gowda <shishirng@gluster.com>
* cluster/dht: Replace ftruncate call with truncate in rebalanceshishir gowda2012-03-091-2/+4
| | | | | | | | | | | ftruncate call seems to be broken as of now (marker). Signed-off-by: shishir gowda <shishirng@gluster.com> Change-Id: Ib49ced0471e1b8e9ee411eb5a9ff1b83c4d82bec BUG: 801658 Reviewed-on: http://review.gluster.com/2905 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* glusterd/rebalance: Bring in support for parallel rebalanceshishirng2012-03-071-7/+62
| | | | | | | | | | | | | | | | | This patch, enables rebalance processes to be started on all nodes where the volume is spread across (1 process per node) node-uuid xattr identifies which node takes ownership of the task to migrate the file. The model employed is push (src pushes to dst) Change-Id: Ieacd46a6216cf6ded841bbaebd10cfaea51c16d6 BUG: 763844 Signed-off-by: shishirng <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/2873 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* cluster/rebalance: Fix expression in check_free_spaceshishir gowda2012-02-291-1/+1
| | | | | | | | | | | | | | Migration should fail if available space in dst < src The expression was evaluating otherwise Change-Id: Ief64ad8797e405d0cdc787eef49bfbf4051dffac BUG: 798864 Signed-off-by: shishir gowda <shishirng@gluster.com> Reviewed-on: http://review.gluster.com/2847 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* cluster/dht: Support for hardlink rebalance when decommissioningshishir gowda2012-02-191-12/+143
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The support for hardlink rebalance is only available for decommissioning of a node. this can be triggered in two ways 1. remove-brick start 2. if decommission node value is set in vol file, then a normal rebalance command The way we handle it is- if (nlink > 1) do * if src file doesnt have linkto xattr * mark src's linkto to the dst * else * perform a link on the dst * do a look up * if nlinks = dst.nlinks * migrate data * else * continue crawling done Signed-off-by: shishir gowda <shishirng@gluster.com> Change-Id: If43b5524b872fd1413e9f7aa7f436cb244e30d8d BUG: 763844 Reviewed-on: http://review.gluster.com/2737 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* cluster/dht: Rebalance will be a new glusterfs processshishirng2012-02-191-1/+605
| | | | | | | | | | | | | | | | | | | | | | | | rebalance will not use any maintainance clients. It is replaced by syncops, with the volfile. Brickop (communication between glusterd<->glusterfs process) is used for status and stop commands. Dept-first traversal of dir is maintained, but data is migrated as and when encounterd. fix-layout (dir) do Complete migrate-data of dir fix-layout (subdir) done Rebalance state is saved in the vol file, for restart-ability. A disconnect event and pidfile state determine the defrag-status Signed-off-by: shishirng <shishirng@gluster.com> Change-Id: Iec6c80c84bbb2142d840242c28db3d5f5be94d01 BUG: 763844 Reviewed-on: http://review.gluster.com/2540 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* core: add an extra flag to readv()/writev() APIAmar Tumballi2012-02-141-4/+4
| | | | | | | | | | | | needed to implement a proper handling of open flag alterations using fcntl() on fd. Change-Id: Ic280d5db6f1dc0418d5c439abb8db1d3ac21ced0 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 782265 Reviewed-on: http://review.gluster.com/2723 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* distribute: utilize new 'fremovexattr()' fop for rebalancev3.3.0qa21Amar Tumballi2012-02-021-1/+1
| | | | | | | | | | | | | instead of existing 'syncop_removexattr()' which is not rename proof for now. Change-Id: Ib171710645a6ee35c86d851a057b68461ecbab27 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 766571 Reviewed-on: http://review.gluster.com/2691 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* Revert "distribute: utilize new 'fremovexattr()' fop for rebalance"Harshavardhana2012-01-231-1/+1
| | | | | | | | | | | This reverts commit c4e4be31ec2783b984e7dbb9ecbc1eea84044ad0 .. There is a problem here. syncop_fremovexattr doesn't exist in the codebase yet. Have a dependency merge for this. No other patches can be uploaded without a build failure now. Change-Id: Ic2c6194d905ffcfa7cbdc29c9bc8739f628d404e Reviewed-on: http://review.gluster.com/2679 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
* distribute: utilize new 'fremovexattr()' fop for rebalanceAmar Tumballi2012-01-231-1/+1
| | | | | | | | | | | | | instead of existing 'syncop_removexattr()' which is not rename proof for now. Change-Id: I3b2afbe58ce90658100cc56b01e23bed672854e8 Signed-off-by: Amar Tumballi <amar@gluster.com> BUG: 766571 Reviewed-on: http://review.gluster.com/803 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Shishir Gowda <shishirng@gluster.com>
* cluster/dht: Fix a typoHarshavardhana2011-11-161-3/+3
| | | | | | | | | | Change-Id: I6bcdc7d600ebb9ef68c60319f96cd9e28d12c861 BUG: 3809 Signed-off-by: Harshavardhana <fharshav@redhat.com> Reviewed-on: http://review.gluster.com/732 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Amar Tumballi <amar@gluster.com>
* distribute: handle migration of symlink and special filesAmar Tumballi2011-10-201-55/+185
| | | | | | | | | | | | TODO: currently, wrt. rebalance/decommissioning, only pending thing is hardlink migration. Change-Id: I30cd06802e84c95601a5a081198f1f09c6d6bc01 BUG: 3714 Reviewed-on: http://review.gluster.com/578 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Shishir Gowda <shishirng@gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* distribute rebalance: preserve proper mode in destinationv3.3.0qa13Amar Tumballi2011-09-301-3/+11
| | | | | | | | | | | * don't set 'sticky' and 'sgid' bits to 0 without checking if the source had those bits prior to rebalance. Change-Id: Ia826cb3cfb55312cdbf00d3421f2bd06b3103ce6 BUG: 3656 Reviewed-on: http://review.gluster.com/541 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* rebalance process: propagate proper errors to userAmar Tumballi2011-09-301-8/+45
| | | | | | | | | | | | | | | | * cluster/distribute: while rebalance, differentiate between valid errors, validation check failures (may not be migrate failure), and success. * mgmt/glusterd: differentiate the errors based on errno. If a valid error has happened, mark the rebalance status as failure, and stop the rebalnace crawl. Change-Id: I2d7bd7b73d2b79bfaf892ad4364bc89830a0d5bb BUG: 3656 Reviewed-on: http://review.gluster.com/540 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* distribute rebalance: handle the open file migrationAmar Tumballi2011-09-121-105/+221
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Complexity involved: To migrate a file with open fd, we have to notify the other client process which has the open fd, and make sure the write()s happening on that fd is properly synced to the migrated file. Once the migration is complete, the client process which has open-fd should get notified and it should start performing all the operations on the new subvolume, instead of earlier cached volume. How to solve the notification part: We can overload the 'postbuf' attribute in the _cbk() function to understand if a file is 'under-migration' or 'migration-complete' state. (This will be something similar to deciding whether a file is DHT-linkfile by its 'mode'). Overall change includes below mentioned major changes: 1. dht_linkfile is decided by only 2 factors (mode(01000), xattr(trusted.glusterfs.dht.linkto)), instead of earlier 3 factors (size==0) 2. in linkfile self-heal part (in 'dht_lookup_everywhere_cbk()'), don't delete a linkfile if there is a open-fd on it. It means, there may be a migration in progress. 3. if a file's revalidate fails with ENOENT, it may be due to file migration, and hence need a lookup_everywhere() 4. There will be 2 phases of file-migration. -> Phase 1: Migration in progress * The source data file will have SGID and STICKY bit set in its mode. * The source data file will have a 'linkto' xattr pointing the destination. * Destination file will have mode set to '01000', and 'linkto' xattr set to itself. -> Phase 2: File migration Complete * The source data file will have mode '01000', and will be 'truncated' to size 0. * The destination file will have inherited mode from the source. (without sgid and sticky bit) and its 'linkto' attribute will be removed. 4. Changes in distribute to work smoothly with a file which is in migration / got migrated. The 'fops' are divided into 3 categories, inode-read, inode-write and others. inode-read fops need to handle only 'phase 2' notification, where as, the inode-write fops need to handle both 'phase 1' and phase2. The inode-write operations will be done on source file, and if any of 'file-migration' procedures are detected in _cbk(), then the operations should be performed on the destination too. when a phase-2 is detected, then the inode-ctx itself should be changed to represent a new layout. With these changes, the open file migration will work smoothly with multiple clients. Change-Id: I512408463814e650f34c62ed009bf2101d016fd6 BUG: 3071 Reviewed-on: http://review.gluster.com/209 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vijay@gluster.com>
* Eliminate many "var set but not used" warnings with newer gcc.Jeff Darcy2011-09-071-2/+0
| | | | | | | | | | | | | | | | This fixes ~200 such warnings, but leaves three categories untouched. (1) Rpcgen code. (2) Macros which set variables in the outer (calling function) scope. (3) Variables which are set via function calls which may have side effects. Change-Id: I6554555f78ed26134251504b038da7e94adacbcd BUG: 2550 Reviewed-on: http://review.gluster.com/371 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
* LICENSE: s/GNU Affero General Public/GNU General Public/Pranith Kumar K2011-08-061-3/+3
| | | | | | | | Change-Id: I3914467611e573cccee0d22df93920cf1b2eb79f BUG: 3348 Reviewed-on: http://review.gluster.com/182 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@gluster.com>
* distribute: bring in feature to do rebalance from insideAmar Tumballi2011-07-141-0/+518
do the 'rebalance' on a file with 'setxattr()' with 'distribute.migrate-data' key. the valid 'value' is "force" or anything else. current implementation of 'rebalance' is same as 'glusterd-rebalance', and hence comes with limitation of not rebalancing hardlinks, non-regular files, and files with open-fds. Signed-off-by: Amar Tumballi <amar@gluster.com> Signed-off-by: Anand Avati <avati@gluster.com> BUG: 3070 (After 'rebalance', the gfid of the file gets changed) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=3070
generated by cgit (git 2.34.1) at 2024-11-13 01:03:34 +0000