storage/posix: Janitor should guard against dir renames.

Problem:
Directory rename while a brick is down can cause gfid handle of that directory
to be deleted until next lookup happens on that directory.

*) Self-heal does not have intelligence to detect renames at the moment. So it
has to delete the directory 'd' using special flags, because it has to perform
'rm -rf' of that directory as it is not empty. Posix xlator implements this by
renaming the directory deleted to 'landfill' directory in '.glusterfs' where
janitor thread will perform actual rm -rf by traversing the directory. Janitor
thread wakes up every 10 minutes to check if there are any directories to be
deleted and deletes them. As part of deleting it also deletes the gfid-handles.

Steps to hit the problem:
1) On a replicate volume create a directory 'd', file in 'd' called 'f' so the
   directory 'd' is not empty.

2) bring one of the bricks down (lets call it brick-a, the other one is brick-b

3) Rename d to d1

4) When brick-a comes online again, self-heal deletes directory 'd' and creates
   directory 'd1' on brick-a for performing self-heal. So on brick-a,
   gfid-handle of 'd' pointing to 'da is deleted and recreated to point to 'd1'.

5) This directory 'b' with all its directory hierarchy (for now just the file
   'f') will be under 'landfill' directory.

6) When janitor thread wakes up and deletes directory 'd' and gfid-handle of
   'd' without realizing that it is now pointing to 'd1'. Thus 'd1' loses its
   gfid-handle

Fix:
Delete gfid-handle for a directory only when the gfid-handle is stale.

Change-Id: I21265b3bd3852f0967d916aaa21108ae5c9e7373
BUG: 1101143
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/7879
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Anand Avati <avati@redhat.com>

1 files changed, 5 insertions, 0 deletions

diff --git a/tests/volume.rc b/tests/volume.rc
index 301b57b8d42..acd9f94b124 100644
--- a/tests/volume.rc
+++ b/tests/volume.rc
@@ -364,3 +364,8 @@ function afr_get_index_count {
         local brick=$1
         ls $1/.glusterfs/indices/xattrop | grep -v xattrop | wc -l
 }
+
+function landfill_entry_count {
+        local brick=$1
+        ls $brick/.glusterfs/landfill | wc -l
+}