Add support for Object Expiration feature

Preventing access to expired objects
------------------------------------
Re-enabled accepting X-Delete-At and X-Delete-After headers. During a GET on
an expired object, DiskFileExpired is raised by DiskFile class. This will
result in object-server returning HTTPNotFound (404) to the client.

Tracking objects to be deleted
------------------------------
Objects to be deleted are tracked using "tracker objects". These are PUT into
a special account(a volume, for now). These zero size "tracker objects" have
names that contain:
    * Expiration timestamp
    * Path of the actual object to be deleted

Deleting actual objects from GlusterFS volume
---------------------------------------------
The object-expirer daemon runs a pass once every X seconds. For every pass it
makes, it queries the special account for "tracker objects". Based on
(timestamp, path) present in name of "tracker objects", object-expirer then
deletes the actual object and the corresponding tracker object.

To run object-expirer forever:
    swift-init object-expirer start
To run just once:
    swift-object-expirer -o -v /etc/swift/object-expirer.conf

Caveat/Limitation: Object-expirer needs a separate account(volume) that
is not used by other services like gswauth. By default, this volume is
named "gsexpiring" and is configurable.

More info about object expiration:
http://docs.openstack.org/developer/swift/overview_expiring_objects.html

Change-Id: I876995bf4f16ef4bfdff901561e0558ecf1dc38f
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6891
Tested-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-by: pushpesh sharma <psharma@redhat.com>
Tested-by: pushpesh sharma <psharma@redhat.com>
Reviewed-by: Chetan Risbud <crisbud@redhat.com>

1 files changed, 21 insertions, 1 deletions

diff --git a/gluster/swift/obj/diskfile.py b/gluster/swift/obj/diskfile.py
index 852f69f..b3e91bc 100644
--- a/gluster/swift/obj/diskfile.py
+++ b/gluster/swift/obj/diskfile.py
@@ -23,6 +23,7 @@ try:
 except ImportError:
     import random
 import logging
+import time
 from collections import defaultdict
 from socket import gethostname
 from hashlib import md5
@@ -33,7 +34,8 @@ from gluster.swift.common.exceptions import AlreadyExistsAsFile, \
     AlreadyExistsAsDir
 from swift.common.utils import TRUE_VALUES, ThreadPool, config_true_value
 from swift.common.exceptions import DiskFileNotExist, DiskFileError, \
-    DiskFileNoSpace, DiskFileDeviceUnavailable, DiskFileNotOpen
+    DiskFileNoSpace, DiskFileDeviceUnavailable, DiskFileNotOpen, \
+    DiskFileExpired
 from swift.common.swob import multi_range_iterator
 
 from gluster.swift.common.exceptions import GlusterFileSystemOSError
@@ -701,6 +703,7 @@ class DiskFile(object):
             the object representation does not exist.
 
         :raises DiskFileNotExist: if the object does not exist
+        :raises DiskFileExpired: if the object has expired
         :returns: itself for use as a context manager
         """
         # Writes are always performed to a temporary file
@@ -731,11 +734,28 @@ class DiskFile(object):
             obj_size = 0
             self._fd = -1
         else:
+            if self._is_object_expired(self._metadata):
+                raise DiskFileExpired(metadata=self._metadata)
             self._fd = fd
 
         self._obj_size = obj_size
         return self
 
+    def _is_object_expired(self, metadata):
+        try:
+            x_delete_at = int(metadata['X-Delete-At'])
+        except KeyError:
+            pass
+        except ValueError:
+            # x-delete-at key is present but not an integer.
+            # TODO: Openstack Swift "quarrantines" the object.
+            # We just let it pass
+            pass
+        else:
+            if x_delete_at <= time.time():
+                return True
+        return False
+
     def _filter_metadata(self):
         if X_TYPE in self._metadata:
             self._metadata.pop(X_TYPE)