summaryrefslogtreecommitdiffstats
path: root/ufo/test/unit
Commit message (Collapse)AuthorAgeFilesLines
* object-storage: final removal of ufo codePeter Portante2013-05-1020-2477/+0
| | | | | | | | | | | | | | | | | | | See https://git.gluster.org/gluster-swift.git for the new location of the Gluster-Swift code. With this patch, no OpenStack Swift related RPMs are constructed. This patch also removes the unused code that references the user.ufo-test xattr key in the DHT translator. Change-Id: I2da32642cbd777737a41c5f9f6d33f059c85a2c1 BUG: 961902 (https://bugzilla.redhat.com/show_bug.cgi?id=961902) Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4970 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Luis Pabon <lpabon@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* object-storage: turn off stat() for container listMohammed Junaid2013-04-121-3/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Turn of stat() system calls used to fetch the file size during a container listing operation since these system calls can swamp Gluster and the result is most often not used. When a GET or HEAD request is made on a container, stat() system calls are made during the Python standard library method, os.walk, to determine if a given directory entry is another directory to recurse into, and then utils._update_list() will stat() each file to get it size, and finally utils.get_container_details_from_fs() will stat() each directory encountered. For most installations we have seen so far, we don't need the container listing to accurately return the size of all the objects in the container, so we can reduce the number of stat() system calls by not fetching the size of the object. For now, turn it off by default, and provide an /etc/swift/fs.conf configuration parameter to turn it back on: accurate_size_in_listing = yes The default for the above is "no". Change-Id: I7dde11e14bb32ecafa3eabb08852f1ffc4366b35 BUG: 903396 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4787 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* object-storage: use tox for unit tests; fix em tooPeter Portante2013-04-0513-62/+39
| | | | | | | | | | | | | | | | | | | | | | | | Add the ability to use tox for unit tests, since it helps us solve the problem of supporting multiple branches that require different versions of dependencies, and allows us to possibly support multiple versions of python in the future. Also fix the code to work with pre-grizzly environments, by not requiring the constraints backport. Also fixed the xattr support to work with both pyxattr and xattr modules. And fixed the ring tests to also work without a live /etc/swift directory. BUG: 948657 (https://bugzilla.redhat.com/show_bug.cgi?id=948657) Change-Id: I2be79c8ef8916bb6552ef957094f9186a963a068 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4781 Reviewed-by: Alex Wheeler <wheelear@gmail.com> Tested-by: Alex Wheeler <wheelear@gmail.com> Reviewed-by: Anand Avati <avati@redhat.com>
* object-storage: Import missing sys and errno modules.Mohammed Junaid2013-04-041-0/+95
| | | | | | | | | | | | | Import the missing modules and implemented unit test case for Glusterfs module. Thanks to Paul Smith for pointing it out. Change-Id: Ib04202aa0ae05c4da2ebbf11f87d6accc778f827 BUG: 905946 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4758 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* Adds missing functions to ring.py, and more thorough tests.Alex Wheeler2013-04-021-2/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Situation: The function get_part_nodes is being called by Openstack-Swift's proxy/controllers/base.py: https://github.com/openstack/swift/blob/1.7.4/swift/proxy/controllers/base.py#L410 https://github.com/openstack/swift/blob/1.7.6/swift/proxy/controllers/base.py#L447 As this was not implemented in the current GlusterFS version of ring.py, it was calling swift's original get_part_nodes, which would often return the incorrect node, resulting in the incorrect drive being associated with a request. There is another function that the original ring.py implements -- get_other_nodes, which has to do with replication. Since GlusterFS is handling replication, this function should never be called. However, in the interest of completeness, that function is also being replaced. Code changes: The two functions, get_part_nodes, and get_other_nodes have been implemented to override the default functions, and get_nodes has been updated to store information in self vars, about the account being operated on, as the two new functions are not called with that info, and get_nodes appears to always be called first. The code has be refactored to all call _get_part_nodes, much like swift has refactored their code. Reason for implementation this way: I didn't see a better way to do it, but am open to suggestions. Test cases: A mock ring is created with two different devices, test and iops test_first_device: Ensure that the first device, test, is returned for both get_nodes, and get_part_node, and get_more_nodes returns volume_not_in_ring. test_invalid_device: Ensure that a request for a non-existant device returns volume_not_in_ring. test_second_device: Same as test_first_device, but for the second device, iops instead of test. test_second_device_with_reseller_prefix: Test that calling with the reseller prefix, AUTH_ will still return the correct data. Change-Id: I2f3d526934a47b01e9c065d0edf0fbf06f300369 BUG: 924792 Signed-off-by: Alex Wheeler <wheelear@gmail.com> Reviewed-on: http://review.gluster.org/4748 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* object-storage: Removed the redundant REMOTE_CLUSTER option.Mohammed Junaid2013-03-211-1/+1
| | | | | | | | | | | | | | Gluster cli uses the remote-host option to connect to the glusterd and by default it uses localhost to connect to glusterd. So, UFO code will use the remote-host option everytime to connect to the glusterd. Change-Id: I5a684d3c43fe9bdc9cc0b7c472a9d8145f9e1fd4 BUG: 878663 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4690 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* object-storage: Fixing the errors and warnings in unittest.Mohammed Junaid2013-03-071-2/+34
| | | | | | | | | | Change-Id: Id22c968aefd82c4b62445b3ecc93cbabc2b35ffc BUG: 887301 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4394 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: Restoring multi volume support in UFO.Mohammed Junaid2013-03-071-0/+43
| | | | | | | | | | | | | | | | | | | | | | | | * Currently, the users of UFO are restricted to use just one volume at any given point of time. This patch removes this limitation. * The usage of gluster-swift-gen-builders has also changed. With this commit the users should mention the list of volumes that they want to expose through UFO. So, only the volumes mentioned during the ring file generation can be accessed. Usage: gluster-swift-gen-builders <vol-name1> [<vol-name2>]... This is an intermediate fix until we remove the account, container and object server processes. Once we have this frame work running, it will completely eliminate the ring files. Change-Id: I9ad3808519fec9c7c60ad846c4f8b653117a8337 BUG: 909053 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4485 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Peter Portante <pportant@redhat.com>
* object-storage: Use the wrapper functions provided by fs_utils.py to makeMohammed Junaid2013-02-111-0/+277
| | | | | | | | | | | | | | | | | | | | | system calls. The set of changes: * Unit test cases for fs_utils.py * Replaced os.path with os_path * Implemented wrapper functions do_write, do_chmod, etc in fs_utils.py * Replaced os.<sys-call> with the wrapper functions. Change-Id: I770da878e83eda6b98e49d70193990406a2642a7 BUG: 887301 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4360 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* object-storage: use temp file optimizationMohammed Junaid2013-01-221-11/+12
| | | | | | | | | | | | | | | A file name '.<FILENAME>.<RANDOM>' will hash to the same GlusterFS node as a file named '<FILENAME>', thus avoiding creation/deletion of linkfiles on a rename. This is part of the work needed to address BZ 876660 (https://bugzilla.redhat.com/show_bug.cgi?id=876660). Change-Id: I6f18c14b8eaa7a35c96f7e455ef3a19bee7dbde5 BUG: 876660 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4325 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Peter Portante <pportant@redhat.com>
* object-storage: apply upstream DiskFile refactorPeter Portante2012-12-171-31/+34
| | | | | | | | | | | | | Apply the upstream DiskFile refactoring in ahead of its use to easiliy apply the temp file optimization. Change-Id: I2708733eed3d87759c70eb3d9e6cd74ef91d0c7b BUG: 876660 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4288 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* object-storage: a set of cleanups to DiskFilePeter Portante2012-12-171-64/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Here are a set of cleanups for things noticed while working on the unit test coverage: * Remove unused constants, adding a constant for default disk chunk size * Document missing logger parameter from constructor * Add iter_hook paramater to constructor in anticipation of 1.7.4 support * Add back meta_file field even though it is not used in our version of the constructor for paranoid compatibility * Remove is_valid field as it is not referenced anywhere, and it is not a field of the super class * Rename fields only used internally by DiskFile with leading underscores * Convert to using os.path.join() instead of hard coded '/' references * Use data_file field where possible * Assert that put_metadata() will only work when the file exists * Remove update_object() method since it is identical to put_metadata() and a direct write will suffice * Create the directory object only if it does not exist when a marker directory is requested Change-Id: If207fed4c0f423e6bd3232e4507f87de877057c4 BUG: 876660 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4287 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* object-storage: Initial unittest of DiskFile classPeter Portante2012-12-172-0/+979
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we had this ahead of time, we could have avoided the errors that were encountered leading to the fix-account-mapping fix (see http://review.gluster.org/4222). This represents 100% coverage of the DiskFile module, but the coverage report says otherwise, unfortunately. That is because the put() method invokes eventlets during the test run, and coverage is not able to accurately track the coverage lines properly. If one comments out the "tpool.execute()" line in DiskFile.put() the coverage report then reports 100% for the DiskFile module. Additionally, we changed DiskFile.put() in four ways that should not change its behavior: 1. Comments were added to explain various code paths and mark potential issues / fixes 2. It no longer returns a boolean value, matching the behavior of swift.obj.server.DiskFile.put() 3. It no longer logs a message when it detects a directory that already exists, instead is raises an exception We believe this is okay because we cannot find a code path that would lead to his condition. As a result, it makes it easier to test all the code paths in that routine. 4. It no longer logs a message when create_dir_object() fails, since create_dir_object() raises an exception on failure only We also modified create_dir_object() to not return a boolean as a result of the above behavior. Note that by implementing these tests up to this point we found three code paths that would have failed if encountered due to missing imports. We also made changes to the DiskFile module to make it a bit easier to test, also eliminating an extra stat system call when deleting directory objects. Change-Id: I3286de83c1ec7c5e8d6cab9354e1c4397cee7497 BUG: 870589 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4284 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com>
* object-storage: unit tests to 100% cover utilsPeter Portante2012-12-111-12/+146
| | | | | | | | | | Change-Id: I2d540e16cc3d420b9bb88e6857e9795d56e34529 BUG: 884327 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4225 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: use temp file optimization for pklPeter Portante2012-12-111-10/+41
| | | | | | | | | | | | | | | | | | Override OpenStack Swift's swift.common.utils.write_pickle with our own implementation that uses the GlusterFS temporary file operation. A file name '.<FILENAME>.<RANDOM>' will hash to the same GlusterFS node as a file named '<FILENAME>', those avoiding a move of the file on a rename. This is part of the work needed to address BZ 876660 (https://bugzilla.redhat.com/show_bug.cgi?id=876660). Change-Id: I1cb9f97f289ab2ca76ec9221366df74de08268bb BUG: 876660 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4224 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: Add a sorted list comparison to testPete Zaitcev2012-11-211-2/+2
| | | | | | | | | | | | | | | | | | After the commit b0cb7aaf for bz#870589, which adds 2 tarballs with layout patterns, Swift tests sometimes fail (depending on the kind of filesystem they are running at and the phase of the moon). As pattern tarball is unpacked, the underlying filesystem is free to return directory listings in any order, and straightforward use of '==' operator fails. As it turns out, one of the two comparisons had set() applied to the list already. So, add it to the other one. BUG: 874390 Change-Id: I02de99593b9567a13076113d58e242b079fde002 Signed-off-by: Pete Zaitcev <zaitcev@redhat.com> Reviewed-on: http://review.gluster.org/4165 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* object-storage: del 1.4.8 ref from dir hierarchyPeter Portante2012-11-215-0/+818
Remove the "swift/1.4.8" directory from the hierarchy, replacing it with "ufo". Change-Id: I60ba171182d7402a2193616c1c95c90cd2ead5bc BUG: 870589 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4200 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pete Zaitcev <zaitcev@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-19 11:43:41 +0000