| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Final set of changes to remove the diffs carried to make UFO work with
OpenStack Swift. The code is now a complete layering on top of OpenStack Swift
where we either "monkey patch" or subclass as necessary.
See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).
There are a lot of changes here due for the most part to rearranging the
directory hierarchy to have create a proper python module hierarchy under the
"gluster" namespace. Plugin references have been removed. The differences that
used to be in the swift.diff file are now replaced with server implementations
for account, container, object, and proxy that subclass the swift versions.
Additionally, the plugins/conf directory has been moved to the "etc"
directory, and the plugins/bin directory promoted a level.
Unit tests pass.
A new setup.py file is provided so that the install process can use it for
creating all the necessary python install infrastructure (eggs and paste
support).
A new RPM spec file is provided which to properly install the new code, and
the sample configuration files have been modified to reference the new python
egg.
Change-Id: I4316c1b66dca80f847fe9b0d583174689c175599
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4180
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactor code to use the devices configuration file setting for account,
container, and object servers, dropping mount_path from the fs.conf file, and
constructing new account, container and object server rings.
This removes the next to last set of diffs in the swift diff file. See BZ
870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).
The key to the change is the dropping of the pre-built account, container and
object rings, instead providing a script that will generate them for the user
given the gluster volume name in use. In addition, we override the Swift
check_mount() method and replace it with Gluster's which attempts to
"auto-mount" if it is not already mounted.
The following is an enumeration of the changes contained in this refactoring:
* The refactoring to override the Swift check_mount() involved condensing
a lot of code, removing redundancies and simplifying methods across a
number of modules
* Drop checking the mount point in the low level DiskDir, DiskAccount and
DiskFile objects now that Swift's normal mount checking is used, and
enable it by default in the template configuration file
* Add missing get_container_timestamp() method for DiskAccount objects
* Fix the plugin RPM spec file to provide the new ring builders, and while
we were at it, finally fix the over-writing of the configuration files
on install
* Bug fixes
* The monkey patched version of check_object_creation was not working
correctly due to a missing import of HTTPBadRequest and a bad
reference to validate_obj_name_component()
* Only have the utils module import the plugins constraints module for
the side effect of monkey patching if gluster is enabled
* Removed the db_file.db file in the tree since it is created on the
fly
* Updated 2011 copyright notices to 2012
Change-Id: I8f4454576b1423021c9bbf3c36176f8db51e62c0
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4179
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Further reduce the diffs we have to carry by moving the Gluster specific
metadata stored when we create an object to the DiskFile class's put() method.
Change-Id: I8cef5412627d3ddfdadbf78ef947e067db5bf620
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4177
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Towards swift-diff removal: hide object_only behavior inside DiskDir class
objects. See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589)
One of the last few reasons for having the swift diff is the need to update
the account and container information from the file system when the
configuration is not for object-only use (meaning, other updates to the file
system can come from clients other than UFO, which means UFO currently has to
generate the necessary metadata for each request instead of caching it).
This change moves that logic into the DiskDir class methods so that the
container and account server code does not have to track that.
Change-Id: If0970c77c72245facd09c2035083e681d6d80303
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4176
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
| |
|
|
|
|
|
|
|
| |
Change-Id: I60b4b1b976ad8359ba072d5d0ed6a8dd2087a738
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4174
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Changes to make DiskAccount subclass DiskDir complete. This change helps
reduce the diffs ahead of further refactoring. Note that DiskDir initializes
all the same fields as DiskAccount, plus a few more, and ends up reading the
metadata properly, while only adding a couple of fields not used for account
purposes.
This change also passes the account controller's logger for use by the
DiskAccount class.
Along the way we:
* Remove the redundant update_account() method in the DiskCommon base
class, since the update_container() method in DiskDir performs the exact
same functionality
* Remove the initialization of variables ahead of assignments in two
cases, and also move the initialization of the list variables closer to
where they are used
* Collapse the imports from swift.plugins.utils into one statement
* Use a temporary directory for the db_file.db when we find ourselves in a
unit test environment
Change-Id: I4dc59b416a66393adfe0ed32c7a62c79731d70c0
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4172
Reviewed-by: Pete Zaitcev <zaitcev@redhat.com>
Tested-by: Pete Zaitcev <zaitcev@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=870589
Remove the Glusterfs object, transforming it into a module providing module
data fields (like swift.common.constraints) and module methods for
mounting/unmounting and access the gluster volume information. As a result, we
can then remove the glusterfs filter from the pipeline since we no longer need
to provide the Glusterfs object through all the plugin code paths.
This is one more step closer to removing our dependency on modifying the Swift
code directly with these changes. See It is also the first step to acknowledging
that we are not a plugin, but a layering on top of Swift.
The major piece of work here is based on a recognition that the
plugins/Glusterfs.py module provided a Glusterfs class that instantiated
instances of an object that always contained the same data from the
configuration file. The fields of such an object were not being changed and
were treated as read-only in all cases. Since the object's data was the same
for all instantiations there was no need to pass the data from the glusterfs
filter all the way down into the bowels of the Gluster_DiskFile and DiskDir
objects.
Taking advantage of the nature of that data, we now just have those fields
read into module variables, and change the Glusterfs object methods into
module level functions. Much of the changes result from the consequence of
making that switch from object to module.
Here are a few other changes made along the way:
* Bump the release numbers in the spec files in recognition of these changes
* Create the plugins/fs_utils.py module so that the methods in the
plugins/Glusterfs.py module don't have to include plugins/utils.py, which
would create a circular dependency
* Note that this dependency comes from methods in plugins/utils.py
depending on the module level constructs in plugins/Glusterfs.py so that
we only store those values in one place
* Changed plugins/DiskDir.py:DiskDir class to not check for, and/or
optionally create, the /etc/swift/db_file.db at run time, just create it a
module init time
* Removed the duplicate strip_obj_storage_path() from plugins/DiskDir.py and
utils.py and move it to the Glusterfs module
* Used os.path.join in plugins/DiskDir.py where possible
* Renamed the .conf files to .conf-gluster so that we don't clobber existing
config files
* This is not a complete change, as the spec file also needs to be
modified to avoid the clobbering
* See also https://bugzilla.redhat.com/show_bug.cgi?id=865867
* Removed the redundant DIR_TYPE definition in plugins/utils.py
* Removed MOUNT_PATH from plugins/utils.py replacing references with that from
Glusterfs
* This actually fixes a bug if a user every used a different mount path
from the default in fs.conf
* Added ASYNCDIR definition to plugins/utils.py until such time as another
refactoring can rely on the one from swift.obj.server
* Renamed plugins/utils.py's plugin_enabled() function to Gluster_enabled()
* The diffs we carry for Swift are now a bit smaller in that we no longer
have to add the plugin() method, we don't have to keep a fs_object field
in these objects, and we can reference the Glusterfs module directly
* Unit tests were modified appropriately, but now need to be run in the
context of a Swift tree; this is unfortunate, but further refactoring will
address this
Change-Id: Id5d2510d56364761c03b3979bc71187dbe2f82fe
BUG: 870589
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4141
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Mohammed Junaid <junaid@redhat.com>
Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The upstream Swift code base contains the following commit which adds the
ability to modify constraint values using the swift.conf file:
https://github.com/openstack/swift/commit/a2ac5efaa64f57fbbe059066c6c4636dfd0715c2
These changes rely on the above commit being back-ported to Swift 1.4.8 (see
https://github.com/portante/swift/commit/fc2421b04022ac6bbe9d5014362ec5f99f94c5e0).
As a result, a good number of differences we carry can be removed, since the
provided swift.conf file now contains the values we need.
Along with these changes, we add middleware to get constraints loaded properly
(via monkey patching) until subclassing is implemented in a future commit. We
further simplify the diffs to a minimal set by storing the timestamp in file
system metadata and moving new files to the plugins/middleware directory.
Note that the original "gluster" middleware was in the swift.diff file, and
has been renamed to "glusterfs" and moved to the new plugins/middleware
directory. The "gluster" middleware is now a temporary way to get the
constraints properly loaded (both of these middleware pieces should disappear
in future commits when we refactor further to subclass the Swift objects
instead of patching them).
Change-Id: I9dc00d6b6cdd64e277896d75c2fb06431c3e69cb
BUG: 862052
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4093
Tested-by: Peter Portante <pportant@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A subsequent commit will refactor the changes required of Swift to be much
more minimal in nature, but does so by relying on an upstream Swift change to
allow constraint values specified in the swift.conf file (see
https://github.com/openstack/swift/commit/a2ac5efaa64f57fbbe059066c6c4636dfd0715c2).
As a result, we can remove the test subdirectory diffs and assume that they
will be present as a result of the upstream patch above.
Change-Id: Ic450aca235d4c9f345e480f9b761bcb2857bd689
BUG: 862052
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/4082
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Reviewed-by: Anand Avati <avati@redhat.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When swift server receives multiple requests on a volume and if the
volume is not mounted before hand, for each request swift tries to
mount the GlusterFS volume on the mount point. This process is racy
and may mount multiple times on the same mount point.
Also added a new option object_only in fs.conf which should be enabled
only if the user requires ReST interface and will have better performance.
By default it is set "off".
Change-Id: Ie1718554c5aaf577e823bbd84da8e78d803e954d
BUG: 821310
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.com/3477
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch also contains fixes to bugs
* 811501
* 812498
* 821310
Also, removed the default set of users in the proxy-server.conf file.
Change-Id: Ief83905d10ff7bf7c43685ada4d7f05959cee9d1
BUG: 821310
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.com/3390
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
|
|
Change-Id: I5e07339064b1f3bb2aa5b04740ed870e114db4f9
BUG: 811430
Signed-off-by: Mohammed Junaid <junaid@redhat.com>
Reviewed-on: http://review.gluster.com/3118
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijay Bellur <vijay@gluster.com>
|
