summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--README.md168
-rw-r--r--glustolibs-gluster-gd2/glustolibs/__init__.py1
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/__init__.py1
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py127
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py141
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py319
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py319
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/rest.py128
-rw-r--r--glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py519
-rw-r--r--glustolibs-gluster-gd2/setup.py49
10 files changed, 1770 insertions, 2 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..422cc82
--- /dev/null
+++ b/README.md
@@ -0,0 +1,168 @@
+# Glusto Libs
+
+`glusto-libs` repo contains the Libraries/Modules necessary for automating the gluster tests.
+It mainly provides python bindings for the GlusterD-2.0 APIs.
+Latest Code for this repo is managed on review.gluster.org
+
+
+Refer the [glusto-doc](http://glusto.readthedocs.io/en/latest/) for info
+on `glusto` framework.
+Issues need to be filled against the
+[Github](https://github.com/gluster/glusto-libs/issues) repo.
+
+
+To automate/run glusto-tests on GD2 environment we need to do following steps:
+-----------------------------------------------------------------------------
+- install `glusto`
+- clone `glusto-libs` repo
+- clone `glusto-tests` repo
+- install all packages (i.e, glustolibs-gluster-gd2, glustolibs-io and glustolibs-misc libraries)
+
+
+How to install glusto:
+----------------------
+One can use either of the three methods.
+
+- using pip
+
+ # pip install --upgrade git+git://github.com/loadtheaccumulator/glusto.git
+
+- using git
+
+ # git clone https://github.com/loadtheaccumulator/glusto.git
+ # cd glusto
+ # python setup.py install
+
+- using ansible: install glusto, glusto-tests
+
+ # ansible-playbook -i host.ini glusto-tests/ansible/deploy-glusto.yaml
+
+
+For more info refer the [docs](http://glusto.readthedocs.io/en/latest/userguide/install.html).
+
+
+How to clone glusto-tests and glusto-libs repo:
+------------------------------------------------
+- using git
+
+ # git clone ssh://user-name@review.gluster.org/glusto-libs
+
+- using git
+
+ # git clone ssh://user-name@review.gluster.org/glusto-tests
+
+
+How to install the glustolibs-gluster-gd2, glustolibs-io and glustolibs-misc libraries:
+---------------------------------------------------------------------------------------
+ # git clone ssh://user-name@review.gluster.org/glusto-libs
+ # cd glusto-libs/glustolibs-gluster-gd2
+ # python setup.py install
+ # cd ../../glusto-tests/glustolibs-io
+ # python setup.py install
+ # cd ../../glusto-tests/glustolibs-misc
+ # python setup.py install
+
+
+To install glusto-tests dependencies:
+-------------------------------------
+`python-docx` needs to be installed when we run IO's and validates on client node.
+
+- To install run:
+
+ *# easy_install pip
+ # pip install --pre python-docx*
+
+How to run the test case:
+-------------------------
+- Update the information about the servers, clients,
+ servers_info, client_info, running_on_volumes, running_on_mounts
+ and volume_create_force etc, which is necessary on the config
+ file[config](https://github.com/gluster/glusto-tests/blob/master/tests/gluster_tests_config.yml).
+ Refer the following for more info [link](http://glusto.readthedocs.io/en/latest/userguide/configurable.html).
+
+- glusto-tests are run using the `glusto` command available after installing
+ the glusto framework. The various options to run tests as provided by
+ glusto framework:
+
+ - To run PyTest tests:
+
+ - To run all tests that are marked with tag 'bvt':
+ *# glusto -c config.yml --pytest='-v -x tests -m bvt'*
+
+ - To run all tests that are under bvt folder:
+ *# glusto -c config.yml --pytest='-v -s bvt/'*
+
+ - To run a single test case:
+ *# glusto -c config.yml --pytest='-v -s -k test_demo1'*
+
+ For more info about running these tests, refer the [docs](http://glusto.readthedocs.io/en/latest/userguide/glusto.html#options-for-running-unit-tests).
+
+
+Writing tests/libraries for GD2:
+--------------------------------
+ - `tests` directory in glusto-tests contain testcases. Testcases are written as component wise.
+Testcases name and file name should should start with test_.
+
+ - `glustolibs-gluster-gd2` directory in glusto-libs contains libraries for GD2 api's.
+Libraries for io's and miscellaneous are written on `glustolibs-io` and `glustolibs-misc`
+respectively. These functions or libraries can be used while writing testcases.
+
+ - While writing testcases or libraries follow:
+
+ - Please follow the [PEP008 style-guide](https://www.python.org/dev/peps/pep-0008/).
+ - Makes sure all the pylint and pyflakes error are fixed
+ For example:
+
+ - C0326: Exactly one space required around assignment
+ - C0111: Missing module doc-string (missing-doc string)
+ - W: 50: Too long line
+
+ For more information on [pylint](https://docs.pylint.org/en/1.6.0/tutorial.html) and on [pyflakes](http://flake8.pycqa.org/en/latest/user/error-codes.html)
+
+
+
+ - Optimize the code as much as possible. Eliminate the repetitive steps, write it has separate function.
+ - Use proper python standards on returning values.
+ This style guide is a list of do's and don’ts for [Python programs](http://google.github.io/styleguide/pyguide.html).
+
+ - Add docstring to every function you write
+
+ For example: This is an example of a module level function
+
+ def module(param1, param2):
+ """
+ Explain what the module function does in breif
+
+ Args:
+ param1: The first parameter.
+ param2: The second parameter.
+
+ Returns:
+ The return value of the function.
+ """
+
+ - Make sure the log messages are grammatically correct and have no spelling mistakes.
+ - Comment every step of the test case/libraries, log the test step, test result, failure and success.
+ For example:
+
+ # peer status from mnode
+ g.log.info("Get peer status from node %s", self.mnode)
+ ret, out, err = peer_status(self.mnode)
+ self.assertEqual(ret, 0, "Failed to get peer status from node %s: %s" % (self.mnode, err))
+ g.log.info("Successfully got peer status from node %s:\n%s", self.mnode, out)
+ - Don't not use `print` statements in test-cases/libraries because prints statements are not captured in log files. Use logger functions to dump messages into log file.
+
+Logging:
+--------
+Log file name and Log level can be passed as argument to glusto command while
+running the glusto-tests. For example:
+
+ # glusto -c 'config.yml' -l /tmp/glustotests_bvt.log --log-level DEBUG --pytest='-v -x tests -m bvt'
+
+One can configure log files, log levels in the testcases as well. For details
+on how to use glusto framework for configuring logs in tests Refer the following [docs](http://glusto.readthedocs.io/en/latest/userguide/loggable.html).
+
+Default log location is: `/tmp/glustomain.log`
+
+Note: When using `glusto` via the Python Interactive Interpreter,
+the default log location is `/tmp/glusto.log`.
diff --git a/glustolibs-gluster-gd2/glustolibs/__init__.py b/glustolibs-gluster-gd2/glustolibs/__init__.py
new file mode 100644
index 0000000..de40ea7
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/__init__.py
@@ -0,0 +1 @@
+__import__('pkg_resources').declare_namespace(__name__)
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/__init__.py b/glustolibs-gluster-gd2/glustolibs/gluster/__init__.py
new file mode 100644
index 0000000..b0d6433
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/__init__.py
@@ -0,0 +1 @@
+__import__('pkg_resources').declare_namespace(__name__) \ No newline at end of file
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py b/glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py
new file mode 100644
index 0000000..49db0c2
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/exceptions.py
@@ -0,0 +1,127 @@
+#!/usr/bin/env python
+# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+"""Exceptions for Gluster libraries and tests"""
+
+
+class GlusterError(Exception):
+ """Base Gluster exception class."""
+ def __init__(self, arg):
+ Exception.__init__(self, arg)
+ self.msg = arg
+
+
+class TransportEndpointNotConnectedError(GlusterError):
+ """Exception for transport endpoint not connected error."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class NoSuchFileOrDirectoryError(GlusterError):
+ """Exception for no such file or directory error."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class AttributesDoNotMatchError(GlusterError):
+ """Attributes do not match exception."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutIsNotCompleteError(GlusterError):
+ """Exception raised when the layout of a file is not complete."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutIsNotBalancedError(GlusterError):
+ """Exception raised when the layout of a file is not balanced."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutHasHolesError(GlusterError):
+ """Exception raised when the layout of a file has holes."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class LayoutHasOverlapsError(GlusterError):
+ """Exception raised when the layout of a file has overlaps."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class FileDoesNotExistOnHashedBricksError(GlusterError):
+ """Exception raised when a file/dir does not exist where it is hashed."""
+ def __init__(self, arg):
+ GlusterError.__init__(self, arg)
+ self.msg = arg
+
+
+class ConfigError(Exception):
+ '''
+ Custom exception thrown when there is an unrecoverable configuration error.
+ For example, a required configuration key is not found.
+ '''
+ pass
+
+
+class ExecutionError(Exception):
+ '''
+ Custom exception thrown when a command executed by Glusto results in an
+ unrecoverable error.
+ For example, all hosts are not in peer state or a volume canot be setup.
+ '''
+ pass
+
+
+class ExecutionParseError(Exception):
+ '''
+ Custom exception thrown when parsing a command executed by Glusto
+ results in an unexpected error.
+ For example, the output of a command when has to be parsed, can have three
+ states. First, the output was as expected. Second, didn't get the expected
+ output after the parsing result and Third, didn't get the expected result
+ as the command itself failed.
+ '''
+ pass
+
+
+class GlusterApiInvalidInputs(Exception):
+ """
+ Custom exception thrown when parsing invalid json inputs for the
+ particular operation
+ For example, invalid peer id specified
+ """
+ pass
+
+
+class GlusterApiError(Exception):
+ """
+ Custom exception thrown when executing rest-api's results in an
+ unrecoverable error.
+ """
+ pass
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py b/glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py
new file mode 100644
index 0000000..fad84f5
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/gluster_base_class.py
@@ -0,0 +1,141 @@
+# Copyright (C) 2019 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+"""
+ Description: Module containing GlusterBaseClass which defines all the
+ variables necessary for tests.
+"""
+
+import unittest
+from glusto.core import Glusto as g
+from glustolibs.gluster.lib_utils import (
+ get_ip_from_hostname, configure_volumes,
+ configure_mounts, inject_msg_in_gluster_logs,
+ configure_logs, set_conf_entity)
+
+
+class runs_on(g.CarteTestClass):
+ """Decorator providing runs_on capability for standard unittest script"""
+
+ def __init__(self, value):
+ # the names of the class attributes set by the runs_on decorator
+ self.axis_names = ['volume_type', 'mount_type']
+
+ # the options to replace 'ALL' in selections
+ self.available_options = [['distributed', 'replicated',
+ 'distributed-replicated'],
+ ['glusterfs']]
+
+ # these are the volume and mount options to run and set in config
+ # what do runs_on_volumes and runs_on_mounts need to be named????
+ run_on_volumes = self.available_options[0]
+ run_on_mounts = self.available_options[1]
+ if g.config.get('gluster', "")['running_on_volumes']:
+ run_on_volumes = g.config['gluster']['running_on_volumes']
+ if g.config.get('gluster', "")['running_on_mounts']:
+ run_on_mounts = g.config['gluster']['running_on_mounts']
+
+ # selections is the above info from the run that is intersected with
+ # the limits from the test script
+ self.selections = [run_on_volumes, run_on_mounts]
+
+ # value is the limits that are passed in by the decorator
+ self.limits = value
+
+
+class GlusterBaseClass(unittest.TestCase):
+ """GlusterBaseClass to be subclassed by Gluster Tests.
+ This class reads the config for variable values that will be used in
+ gluster tests. If variable values are not specified in the config file,
+ the variable are defaulted to specific values.
+ """
+
+ volume_type = None
+ mount_type = None
+
+ @classmethod
+ def setUpClass(cls):
+ """Initialize all the variables necessary for testing Gluster
+ """
+ # Set the values of servers, clients, servers_info and clients_info
+ cls.servers = set_conf_entity('servers')
+ cls.clients = set_conf_entity('clients')
+ cls.all_servers_info = set_conf_entity('servers_info')
+ cls.all_clients_info = set_conf_entity('clients_info')
+
+ # Set mnode : Node on which gluster commands are executed
+ cls.mnode = cls.servers[0]
+
+ # Server IP's
+ cls.servers_ips = []
+ cls.servers_ips = get_ip_from_hostname(cls.servers)
+
+ # Get the volume configuration
+ (cls.default_volume_type_config, cls.volume_create_force,
+ cls.volume, cls.voltype, cls.volname, cls.mnode) = configure_volumes(
+ cls.servers, cls.volume_type)
+
+ # Get the mount configuration.
+ cls.clients, cls.mounts_dict_list, cls.mounts = configure_mounts(
+ cls.mnode, cls.volname, cls.mount_type, cls.all_clients_info)
+
+ # Get gluster Logs info
+ (cls.server_gluster_logs_dirs, cls.server_gluster_logs_files,
+ cls.client_gluster_logs_dirs, cls.client_gluster_logs_files,
+ cls.glustotest_run_id) = configure_logs()
+
+ msg = "Setupclass: %s : %s" % (cls.__name__, cls.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, cls.servers, cls.clients,
+ cls.mount_type, cls.server_gluster_logs_dirs,
+ cls.server_gluster_logs_files,
+ cls.client_gluster_logs_dirs,
+ cls.client_gluster_logs_dirs)
+
+ # Log the baseclass variables for debugging purposes
+ g.log.debug("GlusterBaseClass Variables:\n %s", cls.__dict__)
+
+ def setUp(self):
+ msg = "Starting Test : %s : %s" % (self.id(), self.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, self.servers, self.clients,
+ self.mount_type, self.server_gluster_logs_dirs,
+ self.server_gluster_logs_files,
+ self.client_gluster_logs_dirs,
+ self.client_gluster_logs_dirs)
+
+ def tearDown(self):
+ msg = "Ending Test: %s : %s" % (self.id(), self.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, self.servers, self.clients,
+ self.mount_type, self.server_gluster_logs_dirs,
+ self.server_gluster_logs_files,
+ self.client_gluster_logs_dirs,
+ self.client_gluster_logs_dirs)
+
+ @classmethod
+ def tearDownClass(cls):
+ msg = "Teardownclass: %s : %s" % (cls.__name__, cls.glustotest_run_id)
+ g.log.info(msg)
+ inject_msg_in_gluster_logs(
+ msg, cls.servers, cls.clients,
+ cls.mount_type, cls.server_gluster_logs_dirs,
+ cls.server_gluster_logs_files,
+ cls.client_gluster_logs_dirs,
+ cls.client_gluster_logs_dirs)
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py b/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py
index 3eb16e2..ebcdd9d 100644
--- a/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/lib_utils.py
@@ -1,5 +1,4 @@
-#!/usr/bin/env python
-# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+# Copyright (C) 2019 Red Hat, Inc. <http://www.redhat.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -19,7 +18,49 @@
Description: Helper library for gluster modules.
"""
+import os
+import random
+import copy
+import datetime
+import socket
+from uuid import UUID
from glusto.core import Glusto as g
+from glustolibs.gluster.mount_ops import create_mount_objs
+from glustolibs.gluster.expections import (
+ ConfigError, GlusterApiInvalidInputs)
+
+
+def validate_uuid(brick_id, version=4):
+ """
+ Validates the uuid
+ Args:
+ brick_id (str) : Brick_id to be validated
+ version (int)
+ Returns:
+ True (bool) on if the uuid is valid hex code,
+ else false
+ """
+ try:
+ UUID(brick_id, version=version)
+ except ValueError:
+ # If it's a value error, then the string
+ # is not a valid hex code for a UUID.
+ g.log.error("Invalid brick_id %s", brick_id)
+ return False
+ return True
+
+
+def validate_peer_id(peerid):
+ """
+ Validates the peer id
+ Args:
+ peer id (str) : peer id to be validated
+ Returns:
+ Exceptions on failure
+ """
+ if not validate_uuid(peerid):
+ g.log.error("Invalid peer id %s speceified", peerid)
+ raise GlusterApiInvalidInputs("Invalid peer id specified")
def inject_msg_in_logs(nodes, log_msg, list_of_dirs=None, list_of_files=None):
@@ -75,3 +116,277 @@ def inject_msg_in_logs(nodes, log_msg, list_of_dirs=None, list_of_files=None):
log_msg, list_of_dirs, list_of_files, host)
_rc = False
return _rc
+
+
+def inject_msg_in_gluster_logs(msg, servers, clients,
+ mount_type,
+ server_gluster_logs_dirs,
+ server_gluster_logs_files,
+ client_gluster_logs_dirs,
+ client_gluster_logs_files):
+
+ """Inject all the gluster logs on servers, clients with msg
+ Args:
+ msg (str): Message string to be injected
+ Returns:
+ bool: True if injecting msg on the log files/dirs is successful.
+ False Otherwise.
+ """
+ _rc = True
+ # Inject msg on server gluster logs
+ ret = inject_msg_in_logs(servers, log_msg=msg,
+ list_of_dirs=server_gluster_logs_dirs)
+ if not ret:
+ _rc = False
+
+ if mount_type is not None and "glusterfs" in mount_type:
+ ret = inject_msg_in_logs(clients, log_msg=msg,
+ list_of_dirs=client_gluster_logs_dirs,
+ list_of_files=client_gluster_logs_files)
+ if not ret:
+ _rc = False
+ return _rc
+
+
+def get_ip_from_hostname(nodes):
+ """Returns list of IP's for the list of nodes in order.
+ Args:
+ nodes(list|str): List of nodes hostnames
+ Returns:
+ list: List of IP's corresponding to the hostnames of nodes.
+ """
+ nodes_ips = []
+ nodes = to_list(nodes)
+ for node in nodes:
+ try:
+ ip = socket.gethostbyname(node)
+ except socket.gaierror as e:
+ g.log.error("Failed to get the IP of Host: %s : %s", node,
+ e.strerror)
+ ip = None
+ nodes_ips.append(ip)
+ return nodes_ips
+
+
+def set_conf_entity(entity_name):
+ """Set the value of the entity
+ Args:
+ entity_name (str) : Value of entity to be set
+ Returns:
+ Value of the entity
+ """
+ entity = g.config.get(entity_name)
+ if not entity:
+ raise ConfigError("'%s' not defined in the global config" % entity_name)
+ return entity
+
+
+def configure_volumes(servers, volume_type):
+ """Defines the volume configurations.
+ Args:
+ servers(list) : List of servers
+ volume_type(str) : Type of volume which will be created
+ Returns:
+ default_volume_type_config(dict) : Volume type configuration
+ volume_create_force(bool) : Volume with force option
+ volume(dict): Volume configuration
+ volname(str): Volume name
+ voltype(str): Volume type
+ """
+ # Defining default volume_types configuration.
+ default_volume_type_config = {
+ 'distributed': {
+ 'type': 'distributed',
+ 'dist_count': 4,
+ 'transport': 'tcp'
+ },
+ 'replicated': {
+ 'type': 'replicated',
+ 'replica_count': 2,
+ 'arbiter_count': 1,
+ 'transport': 'tcp'
+ },
+ 'distributed-replicated': {
+ 'type': 'distributed-replicated',
+ 'dist_count': 2,
+ 'replica_count': 3,
+ 'transport': 'tcp'
+ }
+ }
+
+ # Check if default volume_type configuration is provided in
+ # config yml
+ if g.config.get('gluster')['volume_types']:
+ default_volume_type_from_config = (g.config['gluster']['volume_types'])
+
+ for vol_type in default_volume_type_from_config.keys():
+ if default_volume_type_from_config[vol_type]:
+ if vol_type in default_volume_type_config:
+ default_volume_type_config[volume_type] = (
+ default_volume_type_from_config[vol_type])
+
+ # Create Volume with force option
+ volume_create_force = False
+ if g.config.get('gluster')['volume_create_force']:
+ volume_create_force = (g.config['gluster']['volume_create_force'])
+
+ # Get the volume configuration.
+ volume = {}
+ if volume_type:
+ found_volume = False
+ if g.config.get('gluster')['volumes']:
+ for volume in g.config['gluster']['volumes']:
+ if volume['voltype']['type'] == volume_type:
+ volume = copy.deepcopy(volume)
+ found_volume = True
+ break
+
+ if found_volume:
+ if 'name' not in volume:
+ volume['name'] = 'testvol_%s' % volume_type
+
+ if 'servers' not in volume:
+ volume['servers'] = servers
+
+ if not found_volume:
+ try:
+ if g.config['gluster']['volume_types'][volume_type]:
+ volume['voltype'] = (g.config['gluster']['volume_types'][volume_type])
+ except KeyError:
+ try:
+ volume['voltype'] = (default_volume_type_config
+ [volume_type])
+ except KeyError:
+ raise ConfigError("Unable to get configs of volume "
+ "type: %s", volume_type)
+ volume['name'] = 'testvol_%s' % volume_type
+ volume['servers'] = servers
+
+ # Define Volume Useful Variables.
+ volname = volume['name']
+ voltype = volume['voltype']['type']
+ servers = volume['servers']
+ mnode = servers[0]
+ return (default_volume_type_config, volume_create_force,
+ volume, voltype, volname, mnode)
+
+
+def configure_mounts(mnode, volname, mount_type, all_clients_info):
+ """Defines the mount configurations.
+ Args:
+ mnode(str): Node on which volume should be mounted
+ volname(str): Name of the volume
+ mount_type(list): Defines the mount type
+ all_clients_info(dict): Dict of clients information
+ Returns:
+ mounts_dict_list(list): List of the mount informations
+ mounts(str) : GlusterMount instance
+ """
+ # Get the mount configuration
+ mounts = []
+ if mount_type:
+ mounts_dict_list = []
+ found_mount = False
+ if g.config.get('gluster')['mounts']:
+ for mount in g.config['gluster']['mounts']:
+ if mount['protocol'] == mount_type:
+ temp_mount = {}
+ temp_mount['protocol'] = mount_type
+ if 'volname' in mount and mount['volname']:
+ if mount['volname'] == volname:
+ temp_mount = copy.deepcopy(mount)
+ else:
+ continue
+ else:
+ temp_mount['volname'] = volname
+ if ('server' not in mount or
+ (not mount['server'])):
+ temp_mount['server'] = mnode
+ else:
+ temp_mount['server'] = mount['server']
+ if ('mountpoint' not in mount or
+ (not mount['mountpoint'])):
+ temp_mount['mountpoint'] = (os.path.join(
+ "/mnt", '_'.join([volname,
+ mount_type])))
+ else:
+ temp_mount['mountpoint'] = mount['mountpoint']
+ if ('client' not in mount or
+ (not mount['client'])):
+ temp_mount['client'] = (
+ all_clients_info[
+ random.choice(
+ all_clients_info.keys())]
+ )
+ else:
+ temp_mount['client'] = mount['client']
+ if 'options' in mount and mount['options']:
+ temp_mount['options'] = mount['options']
+ else:
+ temp_mount['options'] = ''
+ mounts_dict_list.append(temp_mount)
+ found_mount = True
+
+ if not found_mount:
+ for client in all_clients_info.keys():
+ mount = {
+ 'protocol': mount_type,
+ 'server': mnode,
+ 'volname': volname,
+ 'client': all_clients_info[client],
+ 'mountpoint': (os.path.join(
+ "/mnt", '_'.join([volname, mount_type]))),
+ 'options': ''
+ }
+ mounts_dict_list.append(mount)
+ mounts = create_mount_objs(mounts_dict_list)
+
+ # Defining clients from mounts.
+ clients = []
+ for mount in mounts_dict_list:
+ clients.append(mount['client']['host'])
+ clients = list(set(clients))
+
+ return clients, mounts_dict_list, mounts
+
+
+def configure_logs():
+ """Defines the gluster log information.
+ Returns:
+ server_gluster_logs_dirs(list) : List of server logs dirs
+ server_gluster_logs_files(list) : List of server logs files
+ client_gluster_logs_dirs(list) : List of client logs dirs
+ client_gluster_logs_files(list) : List of client logs files
+ glustotest_run_id(str) : Time the test run
+ """
+ # Gluster Logs info
+ server_gluster_logs_dirs = ["/var/log/glusterd2/glusterd2.log"]
+ server_gluster_logs_files = []
+ if g.config.get("gluster")['server_gluster_logs_info']['dirs']:
+ server_gluster_logs_dirs = (
+ g.config['gluster']['server_gluster_logs_info']['dirs'])
+
+ if g.config.get("gluster")['server_gluster_logs_info']['files']:
+ server_gluster_logs_files = (
+ g.config['gluster']['server_gluster_logs_info']['files'])
+
+ client_gluster_logs_dirs = ["/var/log/glusterd2/glusterd2.log"]
+ client_gluster_logs_files = ["/var/log/glusterd2/glusterd2.log"]
+ if g.config.get("gluster")['client_gluster_logs_info']['dirs']:
+ client_gluster_logs_dirs = (
+ g.config['gluster']['client_gluster_logs_info']['dirs'])
+
+ if g.config.get("gluster")['client_gluster_logs_info']['files']:
+ client_gluster_logs_files = (
+ g.config['gluster']['client_gluster_logs_info']['files'])
+
+ # Have a unique string to recognize the test run for logging in
+ # gluster logs
+ if 'glustotest_run_id' not in g.config:
+ g.config['glustotest_run_id'] = (
+ datetime.datetime.now().strftime('%H_%M_%d_%m_%Y'))
+ glustotest_run_id = g.config['glustotest_run_id']
+ g.log.info("Glusto test run id %s", glustotest_run_id)
+ return (server_gluster_logs_dirs, server_gluster_logs_files,
+ client_gluster_logs_dirs, client_gluster_logs_files,
+ glustotest_run_id)
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py b/glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py
new file mode 100644
index 0000000..f24251b
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/peer_ops.py
@@ -0,0 +1,319 @@
+# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+"""
+ Description: Library for gluster peer operations.
+"""
+
+import json
+import httplib
+from glustolibs.gluster.rest import RestClient
+from glusto.core import Glusto as g
+
+def peer_probe(mnode, server):
+ """Probe the specified server.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ server (str): Server to be peer probed.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the error message
+ and error code of the the command execution.
+ """
+
+ data = {"addresses": [server]}
+ return RestClient(mnode).handle_request('POST', "/v1/peers", httplib.CREATED, data)
+
+
+def pool_list(mnode):
+ """Runs 'gluster pool list' command on the specified node.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+ return RestClient(mnode).handle_request('GET', "/v1/peers", httplib.OK, None)
+
+
+def peer_detach(mnode, server):
+ """ Detach the specified server.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ server (str): Server to be peer detached.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+
+ server_id = get_peer_id(mnode, server)
+ ret, out, err = RestClient(mnode).handle_request('DELETE', "/v1/peers/%s"
+ % server_id, httplib.NO_CONTENT, None)
+ if ret != httplib.NO_CONTENT:
+ returncode = 1
+ g.log.error("Failed to peer detach the node '%s'.", server)
+ else:
+ returncode = 0
+
+ return (returncode, out, err)
+
+
+def peer_status(mnode, peer=None):
+ """ Fetches the peer status
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+
+ path = "/v1/peers"
+ if peer:
+ peerid = get_peer_id(mnode, peer)
+ path = "%s/%s" % (path, peerid)
+ return RestClient(mnode).handle_request('GET', path, httplib.OK, None)
+
+
+def peer_edit(mnode, peerid, zone):
+ """ Edits the peer zone
+_
+ Args:
+ mnode (str): Node on which command has to be executed.
+ peerid (str): The peerid of the peer.
+ Zone (str): The zone details that has to be edited.
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ of command execution.
+
+ The second element 'out' is of type 'str' and is the stdout value
+ of the command execution.
+
+ The third element 'err' is of type 'str' and is the stderr value
+ of the command execution.
+ """
+
+ data = {"metadata": {"zone": zone}}
+ return RestClient(mnode).handle_request("POST", "/v1/peers/%s" % peerid,
+ httplib.CREATED, data)
+
+
+def get_peer_id(mnode, server):
+ """
+ Returns the peer_id of the given server
+
+ Args:
+ server (str) : A server to fetch peer-ids
+
+ Returns:
+ server_id (str) : Peer-id of the given server/peer
+ """
+ from glustolibs.gluster.lib_utils import get_ip_from_hostname
+
+ _ip = node = ids = []
+ _ip = get_ip_from_hostname([server])
+ server = ''.join(_ip)
+ _, out, _ = pool_list(mnode)
+ output = json.loads(out)
+ for elem in output:
+ item = elem['client-addresses'][1].split(":")
+ node.append(item[0])
+ item = elem['id']
+ ids.append(item)
+ if server in node:
+ return ids[-1]
+
+def is_peer_connected(mnode, servers):
+ """Checks whether specified peer is in cluster and 'Connected' state.
+
+ Args:
+ mnode (str): Node from which peer probe has to be executed.
+ servers (str): A server| list of servers to be validated.
+
+ Returns
+ bool : True on success (peer in cluster and connected), False on
+ failure.
+ """
+ from glustolibs.gluster.lib_utils import to_list
+
+ servers = to_list(servers)
+
+ for server in servers:
+ _, out, _ = peer_status(mnode, server)
+ out = json.loads(out)
+ if not out['online']:
+ g.log.error("The peer %s is not connected", server)
+ return False
+ return True
+
+
+def nodes_from_pool_list(mnode):
+ """Return list of nodes from the 'gluster pool list'.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+
+ Returns:
+ NoneType: None if command execution fails.
+ list: List of nodes in pool on Success, Empty list on failure.
+ """
+ _, pool_list_data, _ = pool_list(mnode)
+ server_list = json.loads(pool_list_data)
+ if server_list is None:
+ g.log.error("Unable to get Nodes from the pool list command.")
+ return None
+
+ nodes = []
+ for server in server_list:
+ nodes.append(server['name'])
+ return nodes
+
+
+def peer_probe_servers(mnode, servers, validate=True):
+ """Probe specified servers and validate whether probed servers
+ are in cluster and connected state if validate is set to True.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ servers (str|list): A server|List of servers to be peer probed.
+
+ Kwargs:
+ validate (bool): True to validate if probed peer is in cluster and
+ connected state. False otherwise. Defaults to True.
+
+ Returns:
+ bool: True on success and False on failure.
+ """
+ from glustolibs.gluster.lib_utils import to_list
+
+ servers = to_list(servers)
+
+ if mnode in servers:
+ servers.remove(mnode)
+
+ # Get list of nodes from 'gluster pool list'
+ nodes_in_pool_list = nodes_from_pool_list(mnode)
+ if not nodes_in_pool_list:
+ g.log.error("Unable to get nodes from gluster pool list. "
+ "Failing peer probe.")
+ return False
+
+ for server in servers:
+ if server not in nodes_in_pool_list:
+ ret, _, _ = peer_probe(mnode, server)
+ if ret != 0:
+ g.log.error("Failed to peer probe the node '%s'.", server)
+ return False
+ g.log.info("Successfully peer probed the node '%s'.", server)
+
+ # Validating whether peer is in connected state after peer probe
+ if validate:
+ _rc = False
+ i = 0
+ while i < 200:
+ if is_peer_connected(mnode, servers):
+ _rc = True
+ break
+
+ if not _rc:
+ g.log.error("Peers are in not connected state")
+ g.log.info("All peers are in connected state")
+ return _rc
+
+
+def peer_detach_servers(mnode, servers, validate=True):
+ """Detach peers and validate status of peer if validate is set to True.
+
+ Args:
+ mnode (str): Node on which command has to be executed.
+ servers (str|list): A server|List of servers to be detached.
+
+ Kwargs:
+ validate (bool): True if status of the peer needs to be validated,
+ False otherwise. Defaults to True.
+
+ Returns:
+ bool: True on success and False on failure.
+ """
+
+ from glustolibs.gluster.lib_utils import to_list
+
+ servers = to_list(servers)
+
+ if mnode in servers:
+ servers.remove(mnode)
+
+ for server in servers:
+ ret, _, _ = peer_detach(mnode, server)
+ if ret:
+ g.log.error("Failed to peer detach the node '%s'.", server)
+ return False
+
+ # Validating whether peer detach is successful
+ if validate:
+ i = 0
+ while i < 200:
+ count = 0
+ nodes_in_pool = nodes_from_pool_list(mnode)
+ _rc = True
+ for server in servers:
+ if server in nodes_in_pool:
+ g.log.error("Peer '%s' still in pool", server)
+ _rc = False
+ count += 1
+ if not count:
+ break
+
+ if not _rc:
+ g.log.error("Validation after peer detach failed.")
+ g.log.info("Validation after peer detach is successful")
+ return _rc
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/rest.py b/glustolibs-gluster-gd2/glustolibs/gluster/rest.py
new file mode 100644
index 0000000..f687ea3
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/rest.py
@@ -0,0 +1,128 @@
+# Copyright (C) 2018 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+"""
+ Description: Library to handle all the common REST methods like GET, POST
+ & DELETE
+"""
+
+import json
+import datetime
+import hashlib
+import jwt
+import requests
+from glusto.core import Glusto as g
+
+
+class RestClient(object):
+ '''
+ Class contains common methods for POST, GET, DELETE and
+ verifies authentication
+ '''
+ def __init__(self, mnode, port='24007', user='glustercli', secret=None,
+ verify=False):
+ """
+ Function to form base url and to get secret key
+
+ Args:
+ mnode (str): The server on which the command has to be executed
+ port (str) : The port which API calls use
+ user (str) : By default the user is glustercli
+ """
+
+ self.user = user
+ self.mnode = mnode
+ self.secret = secret
+ self.verify = verify
+ self.port = port
+ self.base_url = ('http://{mnode}:{port}'.format(mnode=mnode,
+ port=port))
+ if self.secret is None:
+ _, self.secret, _ = g.run(mnode, "cat /var/lib/glusterd2/auth")
+
+ def _set_token_in_header(self, method, url, headers=None):
+ """
+ Function to set token in header
+
+ Args:
+ method (str): It can be GET, POST or DELETE
+ url (str): The url for operation
+
+ For Example:
+ token = _set_token_in_header('GET', '/v1/peers')
+
+ """
+
+ if headers is None:
+ headers = dict()
+ claims = dict()
+ claims['iss'] = self.user
+
+ # Issued at time
+ claims['iat'] = datetime.datetime.utcnow()
+
+ # Expiration time
+ claims['exp'] = datetime.datetime.utcnow() + datetime.timedelta(
+ seconds=1)
+
+ # URI tampering protection
+ val = b'%s&%s' % (method.encode('utf8'), url.encode('utf8'))
+ claims['qsh'] = hashlib.sha256(val).hexdigest()
+
+ token = jwt.encode(claims, self.secret, algorithm='HS256')
+ headers['Authorization'] = b'bearer ' + token
+
+ return headers
+
+ def handle_request(self, method, url, expected_status_code, data=None):
+ """ Function that handles all the methods(GET, POST, DELETE)
+
+ Args:
+ method (str): It can be GET, POST, DELETE
+ url (str): The url of the operation
+ expected_status_code (str) : The status_code expected after
+ the API call
+ data (str): The json input that needs to be passed
+
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and returns the status
+ code of command execution.
+
+ The second element 'out' is of type 'str' and is the
+ stdout value
+
+ The third element 'err' is of type 'str' and is the
+ stderr message|value of the command execution.
+
+ Example:
+ handle_request('GET', "/v1/volumes", '200')
+ handle_request('POST', "/vi/volumes", '201', data)
+ """
+
+ headers = self._set_token_in_header(method, url)
+ resp = requests.request(method, self.base_url + url,
+ data=json.dumps(data),
+ headers=headers, verify=self.verify)
+
+ if resp.status_code != expected_status_code:
+ return (1, None, json.dumps(resp.json()))
+
+ if resp.status_code == 204:
+ return (resp.status_code, None, None)
+
+ return (0, json.dumps(resp.json()), None)
+
diff --git a/glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py b/glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py
new file mode 100644
index 0000000..e65043b
--- /dev/null
+++ b/glustolibs-gluster-gd2/glustolibs/gluster/volume_ops.py
@@ -0,0 +1,519 @@
+# Copyright (C) 2019 Red Hat, Inc. <http://www.redhat.com>
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+import json
+import httplib
+from glusto.core import Glusto as g
+from glustolibs.gluster.rest import RestClient
+from glustolibs.gluster.lib_utils import validate_uuid
+from glustolibs.gluster.exceptions import GlusterApiInvalidInputs
+
+
+"""This module contains the python glusterd2 volume api's implementation."""
+
+
+def validate_brick(bricks_list):
+ """Validate brick pattern.
+ Args:
+ bricks(list): in the form of ["nodeid:brickpath"]
+ Returns:
+ brick_req(list): list of bricks
+ """
+ brick_req = []
+ result = True
+ if bricks_list:
+ for brick in bricks_list:
+ brk = brick.split(":")
+ if len(brk) != 2 or not validate_uuid(brk[0]):
+ result = None
+ break
+ req = {}
+ req['peerid'] = brk[0]
+ req['path'] = brk[1]
+ brick_req.append(req)
+ else:
+ result = None
+
+ if result:
+ return brick_req
+ else:
+ return result
+
+
+def volume_create(mnode, volname, bricks_list, force=False, replica_count=0,
+ arbiter_count=0, transport_type="tcp",
+ options=None, metadata=None):
+ """Create the gluster volume with specified configuration
+ Args:
+ mnode(str): server on which command has to be executed
+ volname(str): volume name that has to be created
+ bricks_list (list): List of bricks to use for creating volume.
+ Example:
+ from glustolibs.gluster.lib_utils import form_bricks_list
+ bricks_list = form_bricks_list(mnode, volname, num_of_bricks,
+ servers, servers_info)
+ Kwargs:
+ force (bool): If this option is set to True, then create volume
+ will get executed with force option. If it is set to False,
+ then create volume will get executed without force option
+ replica_count (int): if volume is replicated type
+ arbiter_count (int):if volume is arbiter type
+ transport_type : tcp, rdma
+ options (dict): volume options
+ metadata (dict): volume metadata
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ (-1, '', ''): If not enough bricks are available to create volume.
+ (ret, out, err): As returned by volume create command execution.
+ Example:
+ volume_create(mnode, volname, bricks_list)
+ """
+
+ if len(bricks_list) <= 0:
+ raise GlusterApiInvalidInputs("Bricks cannot be empty")
+
+ req_bricks = validate_brick(bricks_list)
+ if not req_bricks:
+ raise GlusterApiInvalidInputs("Invalid Brick details, bricks "
+ "should be in form of "
+ "<peerid>:<path>")
+
+ if transport_type not in ("tcp", "rdma", "tcp,rdma"):
+ raise GlusterApiInvalidInputs("Transport type %s not "
+ "supported" % transport_type)
+
+ if not options:
+ options = {}
+
+ if not metadata:
+ metadata = {}
+
+ num_bricks = len(bricks_list)
+ sub_volume = []
+
+ if replica_count > 0:
+ replica = arbiter_count + replica_count
+
+ if num_bricks % replica != 0:
+ raise GlusterApiInvalidInputs(
+ "Invalid number of bricks specified")
+
+ num_subvol = num_bricks / replica
+ for i in range(0, num_subvol):
+ idx = i * replica
+ ida = i * replica + 2
+ # If Arbiter is set, set it as Brick Type for 3rd th brick
+ if arbiter_count > 0:
+ req_bricks[ida]['type'] = 'arbiter'
+ subvol_req = {}
+ subvol_req['type'] = 'replicate'
+ subvol_req['bricks'] = req_bricks[idx:idx + replica]
+ subvol_req['replica'] = replica_count
+ subvol_req['arbiter'] = arbiter_count
+ sub_volume.append(subvol_req)
+ else:
+ subvol_req = {}
+ subvol_req['type'] = 'distrubute'
+ subvol_req['bricks'] = req_bricks
+ sub_volume.append(subvol_req)
+
+ # To create a brick dir
+ create_brick_dir = {"create-brick-dir": True}
+
+ data = {
+ "name": volname,
+ "subvols": sub_volume,
+ "transport": transport_type,
+ "options": options,
+ "force": force,
+ "metadata": metadata,
+ "Flags": create_brick_dir
+ }
+
+ return RestClient(mnode).handle_request(
+ "POST", "/v1/volumes", httplib.CREATED, data)
+
+
+def volume_start(mnode, volname, force=False):
+ """Starts the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ force (bool): If this option is set to True, then start volume
+ will get executed with force option. If it is set to False,
+ then start volume will get executed without force option
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_start("w.x.y.z", "testvol")
+ """
+ data = {
+ "force-start-bricks": force
+ }
+ return RestClient(mnode).handle_request(
+ "POST", "/v1/volumes/%s/start" % volname,
+ httplib.OK, data)
+
+
+def volume_stop(mnode, volname, force=False):
+ """Stops the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ force (bool): If this option is set to True, then stop volume
+ will get executed with force option. If it is set to False,
+ then stop volume will get executed without force option
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_stop(w.x.y.z, "testvol")
+ """
+ return RestClient(mnode).handle_request(
+ "POST", "/v1/volumes/%s/stop" % volname,
+ httplib.OK, None)
+
+
+def volume_delete(mnode, volname, xfail=False):
+ """Deletes the gluster volume if given volume exists in
+ gluster and deletes the directories in the bricks
+ associated with the given volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ xfail (bool): expect to fail (non existent volume, etc.)
+ Returns:
+ bool: True, if volume is deleted
+ False, otherwise
+ Example:
+ volume_delete("w.x.y.z", "testvol")
+ """
+ hosts = []
+ paths = []
+ volinfo = get_volume_info(mnode, volname, xfail)
+ if not volinfo:
+ if xfail:
+ g.log.info(
+ "Volume {} does not exist in {}"
+ .format(volname, mnode)
+ )
+ return True
+ else:
+ g.log.error(
+ "Unexpected: volume {} does not exist in {}"
+ .format(volname, mnode))
+ return False
+
+ _, _, err = RestClient(mnode).handle_request(
+ "DELETE", "/v1/volumes/%s" % volname,
+ httplib.NO_CONTENT, None)
+ if err:
+ if xfail:
+ g.log.info("Volume delete is expected to fail")
+ return True
+
+ g.log.error("Volume delete failed")
+ return False
+
+ # remove all brick directories
+ for j in volinfo['subvols']:
+ for i in j['bricks']:
+ g.run(i['host'], "rm -rf %s" % i['path'])
+
+ return True
+
+
+def volume_reset(mnode, volname, force=False,
+ options=None, all_volumes=False):
+ """Resets the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ force (bool): If this option is set to True, then reset volume
+ will get executed with force option. If it is set to False,
+ then reset volume will get executed without force option.
+ options (dict): volume options
+ all_volumes (bool)
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_reset("w.x.y.z", "testvol")`
+ """
+ if not 'options':
+ options = {}
+ data = {
+ "options": options,
+ "force": force,
+ "all": all_volumes,
+ }
+ return RestClient(mnode).handle_request(
+ "DELETE", "/v1/volumes/%s/options" % volname,
+ httplib.OK, data)
+
+
+def volume_info(mnode, volname):
+ """Get gluster volume info
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ Kwargs:
+ volname (str): volume name.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_info("w.x.y.z")
+ """
+ return RestClient(mnode).handle_request("GET",
+ "/v1/volumes/%s" % volname,
+ httplib.OK, None)
+
+
+def get_volume_info(mnode, volname, xfail=False):
+ """Fetches the volume information as displayed in the volume info.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name.
+ Kwargs:
+ xfail (bool): Expect failure to get volume info
+ Returns:
+ NoneType: If there are errors
+ dict: volume info in dict of dicts
+ Example:
+ get_volume_info("abc.com", volname="testvol")
+ """
+ ret, vol_info, err = volume_info(mnode, volname)
+ if ret:
+ if xfail:
+ g.log.error(
+ "Unexpected: volume info {} returned err ({} : {})"
+ .format(volname, vol_info, err)
+ )
+ return None
+ vol_info = json.loads(vol_info)
+ g.log.info("Volume info: %s", vol_info)
+ return vol_info
+
+
+def volume_status(mnode, volname):
+ """Get gluster volume status
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_status("w.x.y.z", "testvol")
+ """
+ return RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/status" % volname,
+ httplib.OK, None)
+
+
+def get_volume_status(mnode, volname, service=''):
+ """This module gets the status of all or specified volume(s)/brick
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name.
+ Kwargs:
+ service (str): name of the service to get status
+ can be bricks
+ Returns:
+ dict: volume status in dict of dictionary format, on success
+ NoneType: on failure
+ Example:
+ get_volume_status("10.70.47.89", volname="testvol")
+ """
+ if service:
+ _, status, err = volume_brick_status(mnode, volname)
+ else:
+ _, status, err = volume_status(mnode, volname)
+ if not err:
+ status = json.loads(status)
+ return status
+ return None
+
+
+def volume_brick_status(mnode, volname):
+ """Get gluster volume brick status
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_status("w.x.y.z","testvol")
+ """
+ return RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/bricks" % volname,
+ httplib.OK, None)
+
+
+def volume_list(mnode):
+ """List the gluster volume
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ Returns:
+ tuple: Tuple containing three elements (ret, out, err).
+ tuple: Tuple containing three elements (ret, out, err).
+ The first element 'ret' is of type 'int' and is the return value
+ The second element 'out' is of type 'str' and is the output of
+ the operation
+ The third element 'err|status' code on failure.
+ Otherwise None.
+ Example:
+ volume_list("w.x.y.z")
+ """
+ return RestClient(mnode).handle_request(
+ "GET", "/v1/volumes", httplib.OK, None)
+
+
+def get_volume_list(mnode, xfail=False):
+ """Fetches the volume names in the gluster.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ Kwargs:
+ xfail (bool): Expect failure to get volume info
+ Returns:
+ NoneType: If there are errors
+ list: List of volume names
+ Example:
+ get_volume_list("w.x.y.z")
+ """
+ vol_list = []
+ ret, volumelist, err = volume_list(mnode)
+ if ret:
+ if xfail:
+ g.log.error(
+ "Unexpected: volume list returned err ({} : {})"
+ .format(volumelist, err)
+ )
+ return None
+ volumelist = json.loads(volumelist)
+ for i in volumelist:
+ vol_list.append(i["name"])
+ g.log.info("Volume list: %s", vol_list)
+ return vol_list
+
+
+def get_volume_options(mnode, volname, option=None):
+ """Gets the option values for the given volume.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ Kwargs:
+ option (str): volume option to get status.
+ If not given, the function returns all the options for
+ the given volume
+ Returns:
+ dict: value for the given volume option in dict format, on success
+ NoneType: on failure
+ Example:
+ get_volume_options(mnode, "testvol")
+ """
+ if not option:
+ _, get_vol_options, err = RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/options" % volname, httplib.OK, None)
+ else:
+ _, get_vol_options, err = RestClient(mnode).handle_request(
+ "GET", "/v1/volumes/%s/options/%s" % (volname, option),
+ httplib.OK, None)
+ if not err:
+ get_vol_options = json.loads(get_vol_options)
+ return get_vol_options
+ return None
+
+
+def set_volume_options(mnode, volname, options,
+ advance=True, experimental=False,
+ deprecated=False):
+ """Sets the option values for the given volume.
+ Args:
+ mnode (str): Node on which cmd has to be executed.
+ volname (str): volume name
+ options (dict): volume options in key
+ value format
+ Kwargs:
+ advance (bool): advance flag to set options. Default set True
+ experimental (bool): experimental flag to set options.
+ Default set False.
+ deprecated (bool): deprecated flag to set options.
+ Default set False
+ Returns:
+ bool: True, if the volume option is set
+ False, on failure
+ Example:
+ set_volume_option("w.x.y.z", "testvol", options)
+ """
+ if not options:
+ raise GlusterApiInvalidInputs("cannot set empty options")
+
+ vol_options = {}
+ req = {}
+ for key in options:
+ vol_options[key] = options[key]
+ req['options'] = vol_options
+ req['allow-advanced-options'] = advance
+ req['allow-experimental-options'] = experimental
+ req['allow-deprecated-options'] = deprecated
+ _, _, err = RestClient(mnode).handle_request(
+ "POST", "/v1/volumes/%s/options" % volname,
+ httplib.CREATED, req)
+ if err:
+ return True
+ return False
diff --git a/glustolibs-gluster-gd2/setup.py b/glustolibs-gluster-gd2/setup.py
new file mode 100644
index 0000000..92d3e36
--- /dev/null
+++ b/glustolibs-gluster-gd2/setup.py
@@ -0,0 +1,49 @@
+#!/usr/bin/python
+# Copyright (c) 2018 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+
+from setuptools import setup, find_packages
+from distutils import dir_util
+
+version = '0.23'
+name = 'glustolibs-gluster'
+
+setup(
+ name=name,
+ version=version,
+ description='Glusto - Red Hat Gluster Libraries',
+ license='GPLv2+',
+ author='Red Hat, Inc.',
+ author_email='gluster-devel@gluster.org',
+ url='http://www.gluster.org',
+ packages=find_packages(),
+ classifiers=[
+ 'Development Status :: 4 - Beta'
+ 'Environment :: Console'
+ 'Intended Audience :: Developers'
+ 'License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)'
+ 'Operating System :: POSIX :: Linux'
+ 'Programming Language :: Python'
+ 'Programming Language :: Python :: 2'
+ 'Programming Language :: Python :: 2.6'
+ 'Programming Language :: Python :: 2.7'
+ 'Topic :: Software Development :: Testing'
+ ],
+ install_requires=['glusto'],
+ dependency_links=['http://github.com/loadtheaccumulator/glusto/tarball/master#egg=glusto'],
+ namespace_packages = ['glustolibs']
+)