glusterfs.git/swift/1.4.8/plugins/DiskDir.py, branch v3.6.5

object-storage: final changes to remove swift diff

2012-11-16T12:52:26+00:00

Final set of changes to remove the diffs carried to make UFO work with
OpenStack Swift. The code is now a complete layering on top of OpenStack Swift
where we either "monkey patch" or subclass as necessary.

See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).

There are a lot of changes here due for the most part to rearranging the
directory hierarchy to have create a proper python module hierarchy under the
"gluster" namespace. Plugin references have been removed. The differences that
used to be in the swift.diff file are now replaced with server implementations
for account, container, object, and proxy that subclass the swift versions.

Additionally, the plugins/conf directory has been moved to the "etc"
directory, and the plugins/bin directory promoted a level.

Unit tests pass.

A new setup.py file is provided so that the install process can use it for
creating all the necessary python install infrastructure (eggs and paste
support).

A new RPM spec file is provided which to properly install the new code, and
the sample configuration files have been modified to reference the new python
egg.

Change-Id: I4316c1b66dca80f847fe9b0d583174689c175599
BUG: 870589
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4180
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Mohammed Junaid 
Tested-by: Kaleb KEITHLEY

object-storage: refactor to use swift devices

2012-11-16T12:52:06+00:00

Refactor code to use the devices configuration file setting for account,
container, and object servers, dropping mount_path from the fs.conf file, and
constructing new account, container and object server rings.

This removes the next to last set of diffs in the swift diff file. See BZ
870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589).

The key to the change is the dropping of the pre-built account, container and
object rings, instead providing a script that will generate them for the user
given the gluster volume name in use. In addition, we override the Swift
check_mount() method and replace it with Gluster's which attempts to
"auto-mount" if it is not already mounted.

The following is an enumeration of the changes contained in this refactoring:

    * The refactoring to override the Swift check_mount() involved condensing
      a lot of code, removing redundancies and simplifying methods across a
      number of modules

    * Drop checking the mount point in the low level DiskDir, DiskAccount and
      DiskFile objects now that Swift's normal mount checking is used, and
      enable it by default in the template configuration file

    * Add missing get_container_timestamp() method for DiskAccount objects

    * Fix the plugin RPM spec file to provide the new ring builders, and while
      we were at it, finally fix the over-writing of the configuration files
      on install

    * Bug fixes

        * The monkey patched version of check_object_creation was not working
          correctly due to a missing import of HTTPBadRequest and a bad
          reference to validate_obj_name_component()

        * Only have the utils module import the plugins constraints module for
          the side effect of monkey patching if gluster is enabled

        * Removed the db_file.db file in the tree since it is created on the
          fly

        * Updated 2011 copyright notices to 2012

Change-Id: I8f4454576b1423021c9bbf3c36176f8db51e62c0
BUG: 870589
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4179
Reviewed-by: Pete Zaitcev 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Kaleb KEITHLEY 
Reviewed-by: Mohammed Junaid

object-storage: swift-diff removal: hide obj only

2012-11-16T12:49:37+00:00

Towards swift-diff removal: hide object_only behavior inside DiskDir class
objects. See BZ 870589 (https://bugzilla.redhat.com/show_bug.cgi?id=870589)

One of the last few reasons for having the swift diff is the need to update
the account and container information from the file system when the
configuration is not for object-only use (meaning, other updates to the file
system can come from clients other than UFO, which means UFO currently has to
generate the necessary metadata for each request instead of caching it).

This change moves that logic into the DiskDir class methods so that the
container and account server code does not have to track that.

Change-Id: If0970c77c72245facd09c2035083e681d6d80303
BUG: 870589
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4176
Reviewed-by: Pete Zaitcev 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Kaleb KEITHLEY 
Reviewed-by: Mohammed Junaid

object-storage: fix trailing whitespace

2012-11-16T12:48:02+00:00

Change-Id: Ib3c671e693c2c332af98a593ca14c42c36f5ca76
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4175
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Mohammed Junaid 
Tested-by: Kaleb KEITHLEY

object-storage: remove the device and part params

2012-11-16T12:47:09+00:00

Change-Id: I60b4b1b976ad8359ba072d5d0ed6a8dd2087a738
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4174
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Mohammed Junaid 
Tested-by: Kaleb KEITHLEY

object-storage: only update metadata that changes

2012-11-16T12:46:19+00:00

See BZ 868120 (https://bugzilla.redhat.com/show_bug.cgi?id=868120)

Change-Id: Ia7f5562cff32eee30c5ca18652d5d9a5a2b18bf4
BUG: 868120
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4173
Reviewed-by: Pete Zaitcev 
Reviewed-by: Mohammed Junaid 
Tested-by: Kaleb KEITHLEY

object-storage: DiskAccount subclassing of DiskDir

2012-11-16T12:45:51+00:00

Changes to make DiskAccount subclass DiskDir complete. This change helps
reduce the diffs ahead of further refactoring. Note that DiskDir initializes
all the same fields as DiskAccount, plus a few more, and ends up reading the
metadata properly, while only adding a couple of fields not used for account
purposes.

This change also passes the account controller's logger for use by the
DiskAccount class.

Along the way we:

    * Remove the redundant update_account() method in the DiskCommon base
      class, since the update_container() method in DiskDir performs the exact
      same functionality

    * Remove the initialization of variables ahead of assignments in two
      cases, and also move the initialization of the list variables closer to
      where they are used

    * Collapse the imports from swift.plugins.utils into one statement

    * Use a temporary directory for the db_file.db when we find ourselves in a
      unit test environment

Change-Id: I4dc59b416a66393adfe0ed32c7a62c79731d70c0
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4172
Reviewed-by: Pete Zaitcev 
Tested-by: Pete Zaitcev 
Reviewed-by: Mohammed Junaid

object-storage: rename self.name to self.container

2012-11-16T12:41:46+00:00

Minor refactoring ahead of further diffs to add clarity to the code.

Rename the DiskDir field from "name" to "container", which is what it is.

Along the way, add a note regarding the one-to-one mapping of account name to
gluster volume name, and fix the get_info() container method documentation to
list all the dictionary keys returned.

Change-Id: I659e1a9a070c16671287f17a170f9d77198534e7
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4169
Reviewed-by: Pete Zaitcev 
Reviewed-by: Mohammed Junaid 
Tested-by: Kaleb KEITHLEY

object-storage: unused parameter cleanup & handl'n

2012-11-16T12:41:25+00:00

Unsed parameter cleanup and handling changes. The device and partition
parameters are not used. The account parameter is used in place of the device
parameter. Rather than just making the device parameter reference the value of
the account parameter, we just don't use the device parameter at all and make
references to account. We also remove the device_path field from the class
since it is unused.

Additionally, we assert that we have a logger since the rest of the relies on
that fact.

Change-Id: Iea556319744ab4f729b1ce23d987650a164749db
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4168
Reviewed-by: Pete Zaitcev 
Tested-by: Pete Zaitcev 
Reviewed-by: Mohammed Junaid

object-storage: add missing docs for params

2012-11-16T12:39:42+00:00

Add missing documentation for constructor parameters.

Change-Id: I3a656cd22f6a2a61e0b4d563c75c2f7fe68d3c54
Signed-off-by: Peter Portante 
Reviewed-on: http://review.gluster.org/4167
Reviewed-by: Pete Zaitcev 
Reviewed-by: Mohammed Junaid 
Tested-by: Kaleb KEITHLEY