glusterfs.git/xlators/cluster/dht/src/dht-selfheal.c, branch v3.4.0alpha

cluster/distribute: get_layout should account only available subvols

2013-01-24T07:43:39+00:00

The earlier logic used to check if (layout-spread-count <= subvol_cnt -
decommissioned bricks). With this if a subvol was down, and layout-spread was >
upsubvols, a mkdir ended up creating holes in the layout.

The fix is to consider only the combination of subvols which are usable (not
down or not decommissioned).

Change-Id: I61ad3bcaf4589f5a75f7887cfa595c98311ae3bb
BUG: 902610
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4412
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/distribute: re-set layouts to prevent overlaps

2012-12-12T00:05:50+00:00

When subvols-per-directory option is used, with bricks addition/removal
the layouts might get distributed to other subvols, which were not part
of the layout before. We need to clean up layouts on old subvolumes, to
prevent overlaps.

Also, we need to make sure if layout-cnt is never less
than subvolume-cnt.

Change-Id: I00994a092ca0c99aedcc41bd9412d43460f88a04
BUG: 884455
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4281
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: fail fix-layout if any of the subvol is down

2012-11-30T06:11:06+00:00

If any subvolume is down, and a layout is re-written and hash
values change, entry names in the downed subvol can be reused
in the other subvol which got the same hash range. when the
downed subvol is brought back up, duplicate entried might appear

Also separated handling of ENOSPC and ENOTCONN error.

Change-Id: I5ed93990425a4cee70df2dab7c7c119fdc87ad56
BUG: 860663
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/4000
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

cluster/dht: Heal dir uid/gid

2012-11-29T21:56:11+00:00

Identify mismatching uid/gid in lookup, and trigger a syncop
heal. uid/gid of subvol with latest ctime is trusted (local->prebuf).

Change-Id: Ib5c4bc438e7f4b1f33080e73593f40f400e997f0
BUG: 862967
Signed-off-by: shishir gowda 
Reviewed-on: http://review.gluster.org/3964
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

dht: improve dht_fix_layout_of_directory for better re-assignment

2012-09-12T21:29:51+00:00

Jeff Darcy wrote:
> AFAICT, the fix-layout code doesn't do the same rotation that the
> new-directory code does. Therefore, the new bricks always claim
> completely predictable hash ranges for every directory, leading to
> either a 0-1-2-3 pattern or a 1-0-2-3 pattern.  In other words, a
> file whose hash falls into the second quarter of the range will always
> be assigned to brick 2, and a file whose hash falls into the fourth
> quarter will always be assigned to brick 3.  The rest will be split
> according to the original pattern.  Put still another way, instead of
> same-named files in different directories being spread across N bricks,
> they might be spread across only two bricks (bad) or totally
> concentrated on one brick (worse) regardless of N.

The current dht_fix_layout_of_directory() code, in an attempt to
maximize overlap of new layout with existing layout (to minimize
movement of data) fails to do a good job of randomizing new assignment
even when it could do a better job. In an example where we expand
from 2 nodes to 4 nodes, the current possibilities are limited in the
following way -

(theoretical hash range: 00 - 99)

OLD 1
-----
server1: 00 - 49
server2: 50 - 99

NEW 1
-----
server1: 00 - 24
server2: 50 - 74
server3: 25 - 49
server4: 75 - 99

OLD 2
-----
server1: 50 - 99
server2: 00 - 49

NEW 2
------
server1: 50 - 74
server2: 00 - 24
server3: 25 - 49
server4: 75 - 99

The above shows that when add-brick from 2 bricks to 4 bricks, server3
and server4 always get the _same_ hash range no matter what the original
hash range assignment was.

The fix in this patch is first do the standard new directory assignment
to a directory (with rotation etc.) and then do the reassignment to
maximize overlap. This way newly added servers still get random ranges
and existing servers have a probability of getting either of the quarters
which were part of its half previously. The same principles hold for
all add-brick from M to M+N.

Change-Id: I0cbbf3bfa334645728072d66aaaa80120d0b295f
BUG: 853258
Signed-off-by: Anand Avati 
Reviewed-on: http://review.gluster.org/3883
Tested-by: Gluster Build System

remove useless if-before-free (and free-like) functions

2012-07-13T21:03:42+00:00

See comments in http://bugzilla.redhat.com/839925 for
the code to perform this change.

Signed-off-by: Jim Meyering 
BUG: 839925
Change-Id: I10e4ecff16c3749fe17c2831c516737e08a3205a
Reviewed-on: http://review.gluster.com/3661
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

distribute: support user-specified layouts.

2012-06-01T00:13:29+00:00

The new type is DHT_HASH_TYPE_DM_USER=1 (on disk in network byte order) and
we treat it the same as DHT_HASH_TYPE_DM except that we don't stomp on it
during rebalance.

Change-Id: I893571a9b89577acdea2fe868915b18d3663fd77
BUG: 807312
Signed-off-by: Jeff Darcy 
Reviewed-on: http://review.gluster.com/3004
Tested-by: Gluster Build System 
Reviewed-by: Anand Avati

license: dual license under GPLV2 and LGPLV3+

2012-05-11T05:06:16+00:00

Note that the license was not changed in any of the following:
	.../argp-standalone/...
        .../booster/...
        .../cli/...
        .../contrib/...
        .../extras/...
        .../glusterfsd/...
        .../glusterfs-hadoop/...
        .../mod_clusterfs/...
        .../scheduler/...
        .../swift/...

The license was not changed in any of the non-building xlators. The
license was not changed in any of the xlators that seemed — to me — to
be clearly server-side only, e.g. protocol/server

Note too that copyright was changed along with the license; I did
not change the copyright in files where the license did not change.

If you find any errors or ommissions please don't hesitate to let me know.

The complete list of files with the license change is:
	libglusterfs/src/byte-order.h
	libglusterfs/src/call-stub.c
	libglusterfs/src/call-stub.h
	libglusterfs/src/checksum.c
	libglusterfs/src/checksum.h
	libglusterfs/src/circ-buff.c
	libglusterfs/src/circ-buff.h
	libglusterfs/src/common-utils.c
	libglusterfs/src/common-utils.h
	libglusterfs/src/compat-errno.c
	libglusterfs/src/compat-errno.h
	libglusterfs/src/compat.c
	libglusterfs/src/compat.h
	libglusterfs/src/daemon.c
	libglusterfs/src/daemon.h
	libglusterfs/src/defaults.c
	libglusterfs/src/defaults.h
	libglusterfs/src/dict.c
	libglusterfs/src/dict.h
	libglusterfs/src/event-history.c
	libglusterfs/src/event-history.h
	libglusterfs/src/event.c
	libglusterfs/src/event.h
	libglusterfs/src/fd-lk.c
	libglusterfs/src/fd-lk.h
	libglusterfs/src/fd.c
	libglusterfs/src/fd.h
	libglusterfs/src/gf-dirent.c
	libglusterfs/src/gf-dirent.h
	libglusterfs/src/globals.c
	libglusterfs/src/globals.h
	libglusterfs/src/glusterfs.h
	libglusterfs/src/graph-print.c
	libglusterfs/src/graph-utils.h
	libglusterfs/src/graph.c
	libglusterfs/src/hashfn.c
	libglusterfs/src/hashfn.h
	libglusterfs/src/iatt.h
	libglusterfs/src/inode.c
	libglusterfs/src/inode.h
	libglusterfs/src/iobuf.c
	libglusterfs/src/iobuf.h
	libglusterfs/src/latency.c
	libglusterfs/src/latency.h
	libglusterfs/src/list.h
	libglusterfs/src/lkowner.h
	libglusterfs/src/locking.h
	libglusterfs/src/logging.c
	libglusterfs/src/logging.h
	libglusterfs/src/mem-pool.c
	libglusterfs/src/mem-pool.h
	libglusterfs/src/mem-types.h
	libglusterfs/src/options.c
	libglusterfs/src/options.h
	libglusterfs/src/rbthash.c
	libglusterfs/src/rbthash.h
	libglusterfs/src/run.c
	libglusterfs/src/run.h
	libglusterfs/src/scheduler.c
	libglusterfs/src/scheduler.h
	libglusterfs/src/stack.c
	libglusterfs/src/stack.h
	libglusterfs/src/statedump.c
	libglusterfs/src/statedump.h
	libglusterfs/src/syncop.c
	libglusterfs/src/syncop.h
	libglusterfs/src/syscall.c
	libglusterfs/src/syscall.h
	libglusterfs/src/timer.c
	libglusterfs/src/timer.h
	libglusterfs/src/trie.c
	libglusterfs/src/trie.h
	libglusterfs/src/xlator.c
	libglusterfs/src/xlator.h
	libglusterfsclient/src/libglusterfsclient-dentry.c
	libglusterfsclient/src/libglusterfsclient-internals.h
	libglusterfsclient/src/libglusterfsclient.c
	libglusterfsclient/src/libglusterfsclient.h
	rpc/rpc-lib/src/auth-glusterfs.c
	rpc/rpc-lib/src/auth-null.c
	rpc/rpc-lib/src/auth-unix.c
	rpc/rpc-lib/src/protocol-common.h
	rpc/rpc-lib/src/rpc-clnt.c
	rpc/rpc-lib/src/rpc-clnt.h
	rpc/rpc-lib/src/rpc-transport.c
	rpc/rpc-lib/src/rpc-transport.h
	rpc/rpc-lib/src/rpcsvc-auth.c
	rpc/rpc-lib/src/rpcsvc-common.h
	rpc/rpc-lib/src/rpcsvc.c
	rpc/rpc-lib/src/rpcsvc.h
	rpc/rpc-lib/src/xdr-common.h
	rpc/rpc-lib/src/xdr-rpc.c
	rpc/rpc-lib/src/xdr-rpc.h
	rpc/rpc-lib/src/xdr-rpcclnt.c
	rpc/rpc-lib/src/xdr-rpcclnt.h
	rpc/rpc-transport/rdma/src/name.c
	rpc/rpc-transport/rdma/src/name.h
	rpc/rpc-transport/rdma/src/rdma.c
	rpc/rpc-transport/rdma/src/rdma.h
	rpc/rpc-transport/socket/src/name.c
	rpc/rpc-transport/socket/src/name.h
	rpc/rpc-transport/socket/src/socket.c
	rpc/rpc-transport/socket/src/socket.h
	xlators/cluster/afr/src/afr-common.c
	xlators/cluster/afr/src/afr-dir-read.c
	xlators/cluster/afr/src/afr-dir-read.h
	xlators/cluster/afr/src/afr-dir-write.c
	xlators/cluster/afr/src/afr-dir-write.h
	xlators/cluster/afr/src/afr-inode-read.c
	xlators/cluster/afr/src/afr-inode-read.h
	xlators/cluster/afr/src/afr-inode-write.c
	xlators/cluster/afr/src/afr-inode-write.h
	xlators/cluster/afr/src/afr-lk-common.c
	xlators/cluster/afr/src/afr-mem-types.h
	xlators/cluster/afr/src/afr-open.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.c
	xlators/cluster/afr/src/afr-self-heal-algorithm.h
	xlators/cluster/afr/src/afr-self-heal-common.c
	xlators/cluster/afr/src/afr-self-heal-common.h
	xlators/cluster/afr/src/afr-self-heal-data.c
	xlators/cluster/afr/src/afr-self-heal-entry.c
	xlators/cluster/afr/src/afr-self-heal-metadata.c
	xlators/cluster/afr/src/afr-self-heal.h
	xlators/cluster/afr/src/afr-self-heald.c
	xlators/cluster/afr/src/afr-self-heald.h
	xlators/cluster/afr/src/afr-transaction.c
	xlators/cluster/afr/src/afr-transaction.h
	xlators/cluster/afr/src/afr.c
	xlators/cluster/afr/src/afr.h
	xlators/cluster/afr/src/pump.c
	xlators/cluster/afr/src/pump.h
	xlators/cluster/dht/src/dht-common.c
	xlators/cluster/dht/src/dht-common.h
	xlators/cluster/dht/src/dht-diskusage.c
	xlators/cluster/dht/src/dht-hashfn.c
	xlators/cluster/dht/src/dht-helper.c
	xlators/cluster/dht/src/dht-inode-read.c
	xlators/cluster/dht/src/dht-inode-write.c
	xlators/cluster/dht/src/dht-layout.c
	xlators/cluster/dht/src/dht-linkfile.c
	xlators/cluster/dht/src/dht-mem-types.h
	xlators/cluster/dht/src/dht-rebalance.c
	xlators/cluster/dht/src/dht-rename.c
	xlators/cluster/dht/src/dht-selfheal.c
	xlators/cluster/dht/src/dht.c
	xlators/cluster/dht/src/nufa.c
	xlators/cluster/dht/src/switch.c
	xlators/cluster/stripe/src/stripe-helpers.c
	xlators/cluster/stripe/src/stripe-mem-types.h
	xlators/cluster/stripe/src/stripe.c
	xlators/cluster/stripe/src/stripe.h
	xlators/features/index/src/index-mem-types.h ¹
	xlators/features/index/src/index.c ¹
	xlators/features/index/src/index.h ¹
	xlators/performance/io-cache/src/io-cache.c
	xlators/performance/io-cache/src/io-cache.h
	xlators/performance/io-cache/src/ioc-inode.c
	xlators/performance/io-cache/src/ioc-mem-types.h
	xlators/performance/io-cache/src/page.c
	xlators/performance/io-threads/src/io-threads.c
	xlators/performance/io-threads/src/io-threads.h
	xlators/performance/io-threads/src/iot-mem-types.h
	xlators/performance/md-cache/src/md-cache-mem-types.h
	xlators/performance/md-cache/src/md-cache.c
	xlators/performance/quick-read/src/quick-read-mem-types.h
	xlators/performance/quick-read/src/quick-read.c
	xlators/performance/quick-read/src/quick-read.h
	xlators/performance/read-ahead/src/page.c
	xlators/performance/read-ahead/src/read-ahead-mem-types.h
	xlators/performance/read-ahead/src/read-ahead.c
	xlators/performance/read-ahead/src/read-ahead.h
	xlators/performance/symlink-cache/src/symlink-cache.c
	xlators/performance/write-behind/src/write-behind-mem-types.h
	xlators/performance/write-behind/src/write-behind.c
	xlators/protocol/auth/addr/src/addr.c ¹
	xlators/protocol/auth/login/src/login.c ¹
	xlators/protocol/client/src/client-callback.c
	xlators/protocol/client/src/client-handshake.c
	xlators/protocol/client/src/client-helpers.c
	xlators/protocol/client/src/client-lk.c
	xlators/protocol/client/src/client-mem-types.h
	xlators/protocol/client/src/client.c
	xlators/protocol/client/src/client.h
	xlators/protocol/client/src/client3_1-fops.c

¹ Copyright only, license reverted to original

Change-Id: If560e826c61b6b26f8b9af7bed6e4bcbaeba31a8
BUG: 820551
Signed-off-by: Kaleb KEITHLEY 
Reviewed-on: http://review.gluster.com/3304
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

dht: Log message fix gfid's to gfids

2012-04-24T16:41:30+00:00

Change-Id: I9ad67b6ded05992d1b56ca843f685c4fe2b15b71
BUG: 815186
Signed-off-by: shylesh 
Reviewed-on: http://review.gluster.com/3217
Tested-by: Gluster Build System 
Reviewed-by: Amar Tumballi

core: coverity issues fixed

2012-04-23T21:52:57+00:00

this is not a complete set of issues getting fixed. Will
address other issues in another patch.

Change-Id: Ib01c7b11b205078cc4d0b3f11610751e32d14b69
Signed-off-by: Amar Tumballi 
BUG: 789278
Reviewed-on: http://review.gluster.com/3145
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy