features/bitrot: do not remove the quarantine handle in forget

If an object is marked as bad, then an entry is corresponding to the
bad object is created in the .glusterfs/quarantine directory to help
scrub status. The entry name is the gfid of the corrupted object.
The quarantine handle is removed in below 2 cases.

1) When protocol/server revceives the -ve lookup on an entry whose inode
   is there in the inode table (it can happen when the corrupted object
   is deleted directly from the backend for recovery purpose) it sends a
   forget on the inode and bit-rot-stub removes the quarantine handle in
   upon getting the forget.
   refer to the below commit
   f853ed9c61bf65cb39f859470a8ffe8973818868:
   http://review.gluster.org/12743)

2) When bit-rot-stub itself realizes that lookup on a corrupted object
   has failed with ENOENT.

But with step1, there is a problem when the bit-rot-stub receives forget
due to lru limit exceeding in the inode table. In such cases, though the
corrupted object is not deleted (either from the mount point or from the
backend), the handle in the quarantine directory is removed and that object
is not shown in the bad objects list in the scrub status command.

So it is better to follow only 2nd step (i.e. bit-rot-stub removing the handle
from the quarantine directory in -ve lookups). Also the handle has to be removed
when a corrupted object is unlinked from the mount point itself.

Change-Id: Ibc3bbaf4bc8a5f8986085e87b729ab912cbf8cf9
BUG: 1308961
Original author: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/13472
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>

1 files changed, 37 insertions, 6 deletions

diff --git a/tests/bitrot/bug-1294786.t b/tests/bitrot/bug-1294786.t
index 2bfae36fd86..4911c1dd6c2 100644
--- a/tests/bitrot/bug-1294786.t
+++ b/tests/bitrot/bug-1294786.t
@@ -13,7 +13,11 @@ function get_bitd_count_2 {
 }
 
 function get_node_uuid {
-        getfattr -n trusted.glusterfs.node-uuid --only-values $M0/FILE 2>/dev/null
+        getfattr -n trusted.glusterfs.node-uuid --only-values $M0/FILE1 2>/dev/null
+}
+
+function get_quarantine_count {
+        ls -l "$B1/.glusterfs/quanrantine" | wc -l
 }
 
 cleanup;
@@ -41,15 +45,26 @@ EXPECT_WITHIN $PROCESS_UP_TIMEOUT "1" get_bitd_count_1
 EXPECT_WITHIN $PROCESS_UP_TIMEOUT "1" get_bitd_count_2
 
 #Create sample file
-TEST `echo "1234" > $M0/FILE`
+TEST `echo "1234" > $M0/FILE1`
+TEST `echo "5678" > $M0/FILE2`
+gfid1=$(getfattr -n glusterfs.gfid.string --only-values $M0/FILE1)
+gfid2=$(getfattr -n glusterfs.gfid.string --only-values $M0/FILE2)
 
 EXPECT "$uuid1" get_node_uuid;
 
 #Corrupt file from back-end
-TEST stat $B1/FILE
-echo "Corrupted data" >> $B1/FILE
+TEST stat $B1/FILE1
+TEST stat $B1/FILE2
+echo "Corrupted data" >> $B1/FILE1
+echo "Corrupted data" >> $B1/FILE2
 #Manually set bad-file xattr since we can't wait for an hour for scrubber.
-TEST setfattr -n trusted.bit-rot.bad-file -v 0x3100 $B1/FILE
+TEST setfattr -n trusted.bit-rot.bad-file -v 0x3100 $B1/FILE1
+TEST setfattr -n trusted.bit-rot.bad-file -v 0x3100 $B1/FILE2
+TEST touch "$B1/.glusterfs/quanrantine/$gfid1"
+TEST chmod 000 "$B1/.glusterfs/quanrantine/$gfid1"
+TEST touch "$B1/.glusterfs/quanrantine/$gfid2"
+TEST chmod 000 "$B1/.glusterfs/quanrantine/$gfid2"
+EXPECT "4" get_quarantine_count;
 
 TEST $CLI_1 volume stop $V0
 TEST $CLI_1 volume start $V0
@@ -61,8 +76,24 @@ EXPECT_WITHIN $CHILD_UP_TIMEOUT "1" afr_child_up_status $V0 1
 EXPECT_WITHIN $PROCESS_UP_TIMEOUT "1" get_bitd_count_1
 EXPECT_WITHIN $PROCESS_UP_TIMEOUT "1" get_bitd_count_2
 #Trigger lookup so that bitrot xlator marks file as bad in its inode context.
-TEST stat $M0/FILE
+TEST stat $M0/FILE1
+TEST stat $M0/FILE2
 
 EXPECT "$uuid2" get_node_uuid;
 
+#BUG 1308961
+#Remove bad files from  mount, it should be removed from quarantine directory.
+TEST rm -f $M0/FILE1
+TEST ! stat "$B1/.glusterfs/quanrantine/$gfid1"
+
+#BUG 1308961
+#Set network.inode-lru-limit to 5 and exceed the limit by creating 10 other files.
+#The bad entry from quarantine directory should not be removed.
+TEST $CLI_1 volume set $V0 network.inode-lru-limit 5
+for i in {1..10}
+do
+     echo "1234" > $M0/file_$i
+done
+TEST stat "$B1/.glusterfs/quanrantine/$gfid2"
+
 cleanup;