object-storage: remove glusterfs filter requirement

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=870589

Remove the Glusterfs object, transforming it into a module providing module
data fields (like swift.common.constraints) and module methods for
mounting/unmounting and access the gluster volume information. As a result, we
can then remove the glusterfs filter from the pipeline since we no longer need
to provide the Glusterfs object through all the plugin code paths.

This is one more step closer to removing our dependency on modifying the Swift
code directly with these changes. See It is also the first step to acknowledging
that we are not a plugin, but a layering on top of Swift.

The major piece of work here is based on a recognition that the
plugins/Glusterfs.py module provided a Glusterfs class that instantiated
instances of an object that always contained the same data from the
configuration file. The fields of such an object were not being changed and
were treated as read-only in all cases. Since the object's data was the same
for all instantiations there was no need to pass the data from the glusterfs
filter all the way down into the bowels of the Gluster_DiskFile and DiskDir
objects.

Taking advantage of the nature of that data, we now just have those fields
read into module variables, and change the Glusterfs object methods into
module level functions. Much of the changes result from the consequence of
making that switch from object to module.

Here are a few other changes made along the way:

  * Bump the release numbers in the spec files in recognition of these changes

  * Create the plugins/fs_utils.py module so that the methods in the
    plugins/Glusterfs.py module don't have to include plugins/utils.py, which
    would create a circular dependency

    * Note that this dependency comes from methods in plugins/utils.py
      depending on the module level constructs in plugins/Glusterfs.py so that
      we only store those values in one place

  * Changed plugins/DiskDir.py:DiskDir class to not check for, and/or
    optionally create, the /etc/swift/db_file.db at run time, just create it a
    module init time

  * Removed the duplicate strip_obj_storage_path() from plugins/DiskDir.py and
    utils.py and move it to the Glusterfs module

  * Used os.path.join in plugins/DiskDir.py where possible

  * Renamed the .conf files to .conf-gluster so that we don't clobber existing
    config files

    * This is not a complete change, as the spec file also needs to be
      modified to avoid the clobbering

    * See also https://bugzilla.redhat.com/show_bug.cgi?id=865867

  * Removed the redundant DIR_TYPE definition in plugins/utils.py

  * Removed MOUNT_PATH from plugins/utils.py replacing references with that from
    Glusterfs

    * This actually fixes a bug if a user every used a different mount path
      from the default in fs.conf

  * Added ASYNCDIR definition to plugins/utils.py until such time as another
    refactoring can rely on the one from swift.obj.server

  * Renamed plugins/utils.py's plugin_enabled() function to Gluster_enabled()

  * The diffs we carry for Swift are now a bit smaller in that we no longer
    have to add the plugin() method, we don't have to keep a fs_object field
    in these objects, and we can reference the Glusterfs module directly

  * Unit tests were modified appropriately, but now need to be run in the
    context of a Swift tree; this is unfortunate, but further refactoring will
    address this

Change-Id: Id5d2510d56364761c03b3979bc71187dbe2f82fe
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4141
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>

1 files changed, 25 insertions, 174 deletions

diff --git a/swift/1.4.8/plugins/utils.py b/swift/1.4.8/plugins/utils.py
index a1ac64db5..bc16ff6f7 100644
--- a/swift/1.4.8/plugins/utils.py
+++ b/swift/1.4.8/plugins/utils.py
@@ -18,10 +18,11 @@ import os
 import errno
 import xattr
 from hashlib import md5
-from swift.common.utils import normalize_timestamp, TRUE_VALUES
-from swift.obj.server import ASYNCDIR
 import cPickle as pickle
 from ConfigParser import ConfigParser, NoSectionError, NoOptionError
+from swift.common.utils import normalize_timestamp, TRUE_VALUES
+from swift.plugins.fs_utils import *
+from swift.plugins import Glusterfs
 
 X_CONTENT_TYPE = 'Content-Type'
 X_CONTENT_LENGTH = 'Content-Length'
@@ -35,15 +36,14 @@ X_CONTAINER_COUNT = 'X-Container-Count'
 X_OBJECT_TYPE = 'X-Object-Type'
 DIR_TYPE = 'application/directory'
 ACCOUNT = 'Account'
-MOUNT_PATH = '/mnt/gluster-object'
 METADATA_KEY = 'user.swift.metadata'
 MAX_XATTR_SIZE = 65536
 CONTAINER = 'container'
 DIR = 'dir'
 MARKER_DIR = 'marker_dir'
 TEMP_DIR = 'tmp'
+ASYNCDIR = 'async_pending'  # Keep in sync with swift.obj.server.ASYNCDIR
 FILE = 'file'
-DIR_TYPE = 'application/directory'
 FILE_TYPE = 'application/octet-stream'
 OBJECT = 'Object'
 OBJECT_TYPE = 'application/octet-stream'
@@ -56,131 +56,6 @@ MEMCACHE_ACCOUNT_DETAILS_KEY_PREFIX = MEMCACHE_KEY_PREFIX + 'account.details.'
 MEMCACHE_CONTAINER_DETAILS_KEY_PREFIX = MEMCACHE_KEY_PREFIX + 'container.details.'
 
 
-def mkdirs(path):
-    """
-    Ensures the path is a directory or makes it if not. Errors if the path
-    exists but is a file or on permissions failure.
-
-    :param path: path to create
-    """
-    if not os.path.isdir(path):
-        try:
-            do_makedirs(path)
-        except OSError, err:
-            #TODO: check, isdir will fail if mounted and volume stopped.
-            #if err.errno != errno.EEXIST or not os.path.isdir(path)
-            if err.errno != errno.EEXIST:
-                raise
-
-def rmdirs(path):
-    if os.path.isdir(path) and dir_empty(path):
-        do_rmdir(path)
-    else:
-        logging.error("rmdirs failed dir may not be empty or not valid dir")
-        return False
-
-def strip_obj_storage_path(path, string='/mnt/gluster-object'):
-    """
-    strip /mnt/gluster-object
-    """
-    return path.replace(string, '').strip('/')
-
-def do_mkdir(path):
-    try:
-        os.mkdir(path)
-    except Exception, err:
-        logging.exception("Mkdir failed on %s err: %s", path, str(err))
-        if err.errno != errno.EEXIST:
-            raise
-    return True
-
-def do_makedirs(path):
-    try:
-        os.makedirs(path)
-    except Exception, err:
-        logging.exception("Makedirs failed on %s err: %s", path, str(err))
-        if err.errno != errno.EEXIST:
-            raise
-    return True
-
-def do_listdir(path):
-    try:
-        buf = os.listdir(path)
-    except Exception, err:
-        logging.exception("Listdir failed on %s err: %s", path, str(err))
-        raise
-    return buf
-
-def do_chown(path, uid, gid):
-    try:
-        os.chown(path, uid, gid)
-    except Exception, err:
-        logging.exception("Chown failed on %s err: %s", path, str(err))
-        raise
-    return True
-
-def do_stat(path):
-    try:
-        #Check for fd.
-        if isinstance(path, int):
-            buf = os.fstat(path)
-        else:
-            buf = os.stat(path)
-    except Exception, err:
-        logging.exception("Stat failed on %s err: %s", path, str(err))
-        raise
-
-    return buf
-
-def do_open(path, mode):
-    try:
-        fd = open(path, mode)
-    except Exception, err:
-        logging.exception("Open failed on %s err: %s", path, str(err))
-        raise
-    return fd
-
-def do_close(fd):
-    #fd could be file or int type.
-    try:
-        if isinstance(fd, int):
-            os.close(fd)
-        else:
-            fd.close()
-    except Exception, err:
-        logging.exception("Close failed on %s err: %s", fd, str(err))
-        raise
-    return True
-
-def do_unlink(path, log = True):
-    try:
-        os.unlink(path)
-    except Exception, err:
-        if log:
-            logging.exception("Unlink failed on %s err: %s", path, str(err))
-        if err.errno != errno.ENOENT:
-            raise
-    return True
-
-def do_rmdir(path):
-    try:
-        os.rmdir(path)
-    except Exception, err:
-        logging.exception("Rmdir failed on %s err: %s", path, str(err))
-        if err.errno != errno.ENOENT:
-            raise
-    return True
-
-def do_rename(old_path, new_path):
-    try:
-        os.rename(old_path, new_path)
-    except Exception, err:
-        logging.exception("Rename failed on %s to %s  err: %s", old_path, new_path, \
-                          str(err))
-        raise
-    return True
-
-
 def read_metadata(path):
     """
     Helper function to read the pickled metadata from a File/Directory.
@@ -262,26 +137,6 @@ def clean_metadata(path):
             raise
         key += 1
 
-def dir_empty(path):
-    """
-    Return true if directory/container is empty.
-    :param path: Directory path.
-    :returns: True/False.
-    """
-    if os.path.isdir(path):
-        try:
-            files = do_listdir(path)
-        except Exception, err:
-            logging.exception("listdir failed on %s err: %s", path, str(err))
-            raise
-        if not files:
-            return True
-        else:
-            return False
-    else:
-        if not os.path.exists(path):
-            return True
-
 def get_device_from_account(account):
     if account.startswith(RESELLER_PREFIX):
         device = account.replace(RESELLER_PREFIX, '', 1)
@@ -302,27 +157,26 @@ def check_user_xattr(path):
         #Remove xattr may fail in case of concurrent remove.
     return True
 
-def _check_valid_account(account, fs_object):
-    mount_path = getattr(fs_object, 'mount_path', MOUNT_PATH)
+def _check_valid_account(account):
+    full_mount_path = os.path.join(Glusterfs.MOUNT_PATH, account)
 
-    if os.path.ismount(os.path.join(mount_path, account)):
+    if os.path.ismount(full_mount_path):
         return True
 
-    if not check_account_exists(fs_object.get_export_from_account_id(account), fs_object):
+    if not Glusterfs.check_account_exists(Glusterfs.get_export_from_account_id(account)):
         logging.error('Account not present %s', account)
         return False
 
-    if not os.path.isdir(os.path.join(mount_path, account)):
-        mkdirs(os.path.join(mount_path, account))
+    if not os.path.isdir(full_mount_path):
+        mkdirs(full_mount_path)
 
-    if fs_object:
-        if not fs_object.mount(account):
-            return False
+    if not Glusterfs.mount(account):
+        return False
 
     return True
 
-def check_valid_account(account, fs_object):
-    return _check_valid_account(account, fs_object)
+def check_valid_account(account):
+    return _check_valid_account(account)
 
 def validate_container(metadata):
     if not metadata:
@@ -401,7 +255,7 @@ def is_marker(metadata):
 
 def _update_list(path, const_path, src_list, reg_file=True, object_count=0,
                  bytes_used=0, obj_list=[]):
-    obj_path = strip_obj_storage_path(path, const_path)
+    obj_path = Glusterfs.strip_obj_storage_path(path, const_path)
 
     for i in src_list:
         if obj_path:
@@ -460,7 +314,7 @@ def get_container_details(cont_path, memcache=None):
     """
     mkey = ''
     if memcache:
-        mkey = MEMCACHE_CONTAINER_DETAILS_KEY_PREFIX + strip_obj_storage_path(cont_path)
+        mkey = MEMCACHE_CONTAINER_DETAILS_KEY_PREFIX + Glusterfs.strip_obj_storage_path(cont_path)
         cd = memcache.get(mkey)
         if cd:
             if not cd.dir_list:
@@ -517,7 +371,7 @@ def get_account_details(acc_path, memcache=None):
     acc_stats = None
     mkey = ''
     if memcache:
-        mkey = MEMCACHE_ACCOUNT_DETAILS_KEY_PREFIX + strip_obj_storage_path(acc_path)
+        mkey = MEMCACHE_ACCOUNT_DETAILS_KEY_PREFIX + Glusterfs.strip_obj_storage_path(acc_path)
         ad = memcache.get(mkey)
         if ad:
             # FIXME: Do we really need to stat the file? If we are object
@@ -627,18 +481,15 @@ def create_account_metadata(acc_path, memcache=None):
     metadata = get_account_metadata(acc_path, memcache)
     return restore_metadata(acc_path, metadata)
 
-def check_account_exists(account, fs_object):
-    if account not in get_account_list(fs_object):
+def check_account_exists(account):
+    if account not in get_account_list():
         logging.warn('Account %s does not exist' % account)
         return False
     else:
         return True
 
-def get_account_list(fs_object):
-    account_list = []
-    if fs_object:
-        account_list = fs_object.get_export_list()
-    return account_list
+def get_account_list():
+    return Glusterfs.get_export_list()
 
 def get_account_id(account):
     return RESELLER_PREFIX + md5(account + HASH_PATH_SUFFIX).hexdigest()
@@ -647,10 +498,10 @@ def get_account_id(account):
 __swift_conf = ConfigParser()
 __swift_conf.read(os.path.join('/etc/swift', 'swift.conf'))
 try:
-    _plugin_enabled = __swift_conf.get('DEFAULT', 'Enable_plugin', 'no') in TRUE_VALUES
+    _gluster_enabled = __swift_conf.get('DEFAULT', 'Enable_plugin', 'no') in TRUE_VALUES
 except NoOptionError, NoSectionError:
-    _plugin_enabled = False
+    _gluster_enabled = False
 del __swift_conf
 
-def plugin_enabled():
-    return _plugin_enabled
+def Gluster_enabled():
+    return _gluster_enabled