glusterfs-afrv1.git/libglusterfs, branch v3.3.0qa12

statedump: dump important details of oldgraph also

2011-09-23T08:22:56+00:00

* with this, it helps to find out most of the leaks in the process

Change-Id: I70b476e4d63d74843d6664cfe407164f2b79b781
BUG: 3567
Reviewed-on: http://review.gluster.com/491
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

libglusterfs: Adding missing event strings for events.

2011-09-23T08:19:38+00:00

Change-Id: Idf55abfa77712d32b27798fedcfddb65599b562b
Reviewed-on: http://review.gluster.com/493
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Implemented cmd to trigger self-heal on a replicate volume.

2011-09-22T16:43:25+00:00

This cmd is used in the context of proactive self-heal for replicated
volumes. User invokes the following cmd when (s)he suspects that self-heal
needs to be done on a particular volume,
        gluster volume heal .

Change-Id: I3954353b53488c28b70406e261808239b44997f3
BUG: 3602
Reviewed-on: http://review.gluster.com/454
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

statedump: add more memory accounting related stats

2011-09-22T11:33:40+00:00

* iobuf: add variable to keep count of total number of allocations
* iobuf: include 'purged' and 'filled' arenas also in dump
* mempool: more details added (with a name to tell why mem-pool
  is created)
* memory-accounting: print number of allocs in each type

this would give us much better understanding of the memory
allocation pattern

Change-Id: I117ac0c1da943a4cc91543a01963ba7940db2b5f
BUG: 3567
Reviewed-on: http://review.gluster.com/376
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

libglusterfs: run: fix closing of redirection fd in child

2011-09-19T05:51:54+00:00

When run was doing a program invocation like "echo foo > foof",
the file descriptor to foof was closed before stdout could be
reopened to it.

To fix this, we change the order of actions: first do redirections,
then do the fd cleanup. This also simplifies code (as there are less
fds which are exempt of the cleanup).

Change-Id: I5f74e41f2999e59916a40c8a89d7fb730d808c27
BUG: 2562
Reported-by: Rajesh Amaravathi 
Reviewed-on: http://review.gluster.com/428
Tested-by: Gluster Build System 
Tested-by: Rajesh Amaravathi 
Reviewed-by: Rajesh Amaravathi 
Reviewed-by: Amar Tumballi

Proactive self heal process implementation

2011-09-14T12:36:24+00:00

Change-Id: I96db0d94566ceabf1649f890318363f738c06553
BUG: 2458
Reviewed-on: http://review.gluster.com/403
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

dict: add reset function which empties the dict

2011-09-14T12:36:06+00:00

Change-Id: I267c81a129197534fb318671eafb76e144a15c8c
BUG: 2458
Reviewed-on: http://review.gluster.com/402
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

libglusterfs/common-utils: resolve_ip6() to take AI_ADDRCONFIG

2011-09-14T08:38:11+00:00

AI_ADDRCONFIG flag is needed for 'getaddrinfo()' call as hint so
that while resolving a hostname, ip list will be taken from proper
configured address family

Change-Id: Iad6067ad64444d3930d5be593ca819a8de5fc0c1
BUG: 3548
Reviewed-on: http://review.gluster.com/414
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

GlusterFS Hadoop specific DSL for mountbroker

2011-09-13T17:14:04+00:00

Change-Id: Ie379992bdea0974c8c5e1a4d7bc3e87cefe0d256
BUG: 3539
Reviewed-on: http://review.gluster.com/404
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

distribute rebalance: handle the open file migration

2011-09-13T06:57:38+00:00

Complexity involved: To migrate a file with open fd, we have to
notify the other client process which has the open fd, and make
sure the write()s happening on that fd is properly synced to the
migrated file. Once the migration is complete, the client
process which has open-fd should get notified and it should
start performing all the operations on the new subvolume,
instead of earlier cached volume.

How to solve the notification part:

We can overload the 'postbuf' attribute in the _cbk() function to
understand if a file is 'under-migration' or 'migration-complete'
state. (This will be something similar to deciding whether a file
is DHT-linkfile by its 'mode').

Overall change includes below mentioned major changes:

1. dht_linkfile is decided by only 2 factors (mode(01000),
   xattr(trusted.glusterfs.dht.linkto)), instead of earlier
   3 factors (size==0)

2. in linkfile self-heal part (in 'dht_lookup_everywhere_cbk()'),
   don't delete a linkfile if there is a open-fd on it. It means,
   there may be a migration in progress.

3. if a file's revalidate fails with ENOENT, it may be due to file
   migration, and hence need a lookup_everywhere()

4. There will be 2 phases of file-migration.

-> Phase 1: Migration in progress
   * The source data file will have SGID and STICKY bit set in its mode.
   * The source data file will have a 'linkto' xattr pointing the
     destination.
   * Destination file will have mode set to '01000', and 'linkto' xattr
     set to itself.

-> Phase 2: File migration Complete
   * The source data file will have mode '01000', and will be 'truncated'
     to size 0.
   * The destination file will have inherited mode from the source. (without
     sgid and sticky bit) and its 'linkto' attribute will be removed.

4. Changes in distribute to work smoothly with a file which is in migration /
   got migrated.

The 'fops' are divided into 3 categories, inode-read, inode-write and others.
inode-read fops need to handle only 'phase 2' notification, where as, the
inode-write fops need to handle both 'phase 1' and phase2. The inode-write
operations will be done on source file, and if any of 'file-migration' procedures
are detected in _cbk(), then the operations should be performed on the destination
too.

when a phase-2 is detected, then the inode-ctx itself should be changed to represent
a new layout.

With these changes, the open file migration will work smoothly with multiple clients.

Change-Id: I512408463814e650f34c62ed009bf2101d016fd6
BUG: 3071
Reviewed-on: http://review.gluster.com/209
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur