summaryrefslogtreecommitdiffstats
path: root/ufo
Commit message (Collapse)AuthorAgeFilesLines
* object-storage: Store the lock file in /var/run/swift.Mohammed Junaid2013-02-071-8/+31
| | | | | | | | | | | | | | | * Openstack swift uses the /var/run/swift directory to store the pid files for all the servers. * Also, added a script that would unmount the gluster client on a volume stop. Change-Id: Ib5b9a2964987ca7696d9a2570f1f7af8490b2168 BUG: 861497 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4417 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* object-storage: yield during directory tree walksPeter Portante2013-02-041-10/+15
| | | | | | | | | | | | | | | | | | | Have the co-routine (greenlet) yield during long running directory tree walks to avoid starving out other greenlets that might be available to run. A directory walk involves statting every file in the tree to get it size. For large numbers of files, this can get expensive. So we yield after every stat call, and after processing every directory. Change-Id: I07f1dfeef6a09b5817e0c237ecc748c491d52a31 BUG: 894674 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4380 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Mohammed Junaid <junaid@redhat.com>
* object-storage: use temp file optimizationMohammed Junaid2013-01-222-16/+32
| | | | | | | | | | | | | | | A file name '.<FILENAME>.<RANDOM>' will hash to the same GlusterFS node as a file named '<FILENAME>', thus avoiding creation/deletion of linkfiles on a rename. This is part of the work needed to address BZ 876660 (https://bugzilla.redhat.com/show_bug.cgi?id=876660). Change-Id: I6f18c14b8eaa7a35c96f7e455ef3a19bee7dbde5 BUG: 876660 Signed-off-by: Mohammed Junaid <junaid@redhat.com> Reviewed-on: http://review.gluster.org/4325 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Peter Portante <pportant@redhat.com>
* object-storage: add UFO unit testsPeter Portante2013-01-171-2/+3
| | | | | | | | | | | | | | | | | | Run the UFO unit tests following the main GlusterFS unit tests. The UFO unit tests run under the Nose unit test running harness, requiring the python-nose package to be installed. These unit tests run fast enough that we could even consider running them on a commit hook run by rfc.sh. Note that they are not functional tests, as they don't require a real Gluster file system to run. Change-Id: I55972890c24be16fc588a6950cb6bfd539287bbe Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4333 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com> Tested-by: Anand Avati <avati@redhat.com>
* object-storage: only open file when requestedPeter Portante2012-12-172-4/+8
| | | | | | | | | | | | | | | | | | | Only open a file when the caller of the constructor indicates that it expects to have a file pointer to access in the "fp" field. Found by Junaid during review http://review.gluster.org/4284, Patch set 6. We also bump the version number to mark a line in the set of changes we want to perform a mini release with to double check for performance regressions. Change-Id: I13cf336bb519088890192ee111f4ece85a5982c4 BUG: 886730 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4303 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: don't sort object list, not req'dPeter Portante2012-12-171-3/+0
| | | | | | | | | | | | | | The REST API for listing objects in a container does not require that the list of objects be sorted (the API for listing containers in an account does require it). Since we can have thousands and thousands of objects in a container, don't sort them when it is not required. Change-Id: I6939ef3fec3ea3814a49e3a3046273304889831c BUG: 887301 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4312 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: do not list directory for deletePeter Portante2012-12-171-4/+5
| | | | | | | | | | | | | | | | Do not list the entire directory contents of the directory containing the file before deleting it. We have already verified the files existence constructing the Gluster_DiskFile object. This fixes a customer issue. See 885281 (https://bugzilla.redhat.com/show_bug.cgi?id=885281). Change-Id: I4425c42af3e03624370c779d74b7f073e6eef024 BUG: 885281 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4302 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: apply upstream DiskFile refactorPeter Portante2012-12-172-73/+81
| | | | | | | | | | | | | Apply the upstream DiskFile refactoring in ahead of its use to easiliy apply the temp file optimization. Change-Id: I2708733eed3d87759c70eb3d9e6cd74ef91d0c7b BUG: 876660 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4288 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* object-storage: a set of cleanups to DiskFilePeter Portante2012-12-172-132/+138
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Here are a set of cleanups for things noticed while working on the unit test coverage: * Remove unused constants, adding a constant for default disk chunk size * Document missing logger parameter from constructor * Add iter_hook paramater to constructor in anticipation of 1.7.4 support * Add back meta_file field even though it is not used in our version of the constructor for paranoid compatibility * Remove is_valid field as it is not referenced anywhere, and it is not a field of the super class * Rename fields only used internally by DiskFile with leading underscores * Convert to using os.path.join() instead of hard coded '/' references * Use data_file field where possible * Assert that put_metadata() will only work when the file exists * Remove update_object() method since it is identical to put_metadata() and a direct write will suffice * Create the directory object only if it does not exist when a marker directory is requested Change-Id: If207fed4c0f423e6bd3232e4507f87de877057c4 BUG: 876660 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4287 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* object-storage: Initial unittest of DiskFile classPeter Portante2012-12-175-49/+1080
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we had this ahead of time, we could have avoided the errors that were encountered leading to the fix-account-mapping fix (see http://review.gluster.org/4222). This represents 100% coverage of the DiskFile module, but the coverage report says otherwise, unfortunately. That is because the put() method invokes eventlets during the test run, and coverage is not able to accurately track the coverage lines properly. If one comments out the "tpool.execute()" line in DiskFile.put() the coverage report then reports 100% for the DiskFile module. Additionally, we changed DiskFile.put() in four ways that should not change its behavior: 1. Comments were added to explain various code paths and mark potential issues / fixes 2. It no longer returns a boolean value, matching the behavior of swift.obj.server.DiskFile.put() 3. It no longer logs a message when it detects a directory that already exists, instead is raises an exception We believe this is okay because we cannot find a code path that would lead to his condition. As a result, it makes it easier to test all the code paths in that routine. 4. It no longer logs a message when create_dir_object() fails, since create_dir_object() raises an exception on failure only We also modified create_dir_object() to not return a boolean as a result of the above behavior. Note that by implementing these tests up to this point we found three code paths that would have failed if encountered due to missing imports. We also made changes to the DiskFile module to make it a bit easier to test, also eliminating an extra stat system call when deleting directory objects. Change-Id: I3286de83c1ec7c5e8d6cab9354e1c4397cee7497 BUG: 870589 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4284 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com>
* object-storage: unit tests to 100% cover utilsPeter Portante2012-12-111-12/+146
| | | | | | | | | | Change-Id: I2d540e16cc3d420b9bb88e6857e9795d56e34529 BUG: 884327 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4225 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: use temp file optimization for pklPeter Portante2012-12-113-10/+73
| | | | | | | | | | | | | | | | | | Override OpenStack Swift's swift.common.utils.write_pickle with our own implementation that uses the GlusterFS temporary file operation. A file name '.<FILENAME>.<RANDOM>' will hash to the same GlusterFS node as a file named '<FILENAME>', those avoiding a move of the file on a rename. This is part of the work needed to address BZ 876660 (https://bugzilla.redhat.com/show_bug.cgi?id=876660). Change-Id: I1cb9f97f289ab2ca76ec9221366df74de08268bb BUG: 876660 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4224 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: Use "-dev" in the version stringPeter Portante2012-12-102-2/+2
| | | | | | | | | | | | | | | | | | | Currently, we have not solved the account mapping story entirely (we don't have a way to support more than one account to gluster volume in the mapping). As such, do not declare this as the final version. Also update the default RHS 2.0 spec file to "-3" for the release number to refer to the fact that we now have a working account mapping (see http://review.gluster.org/4222). Change-Id: Ia1cd0d18c1d7a7b20f732cc0ff867fecda30a7af BUG: 870589 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4283 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* object-storage: enforce RPM package dependenciesPeter Portante2012-12-071-18/+10
| | | | | | | | | | | | | We now have access to openstack-swift-* packages that work with these sources so we can enforce the dependencies and what gets obsoleted. Change-Id: I83f442f3eb2424d176a2257f3498ed39eed32251 BUG: 870589 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4223 Reviewed-by: Mohammed Junaid <junaid@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* object-storage: fix account mappingPeter Portante2012-12-064-10/+8
| | | | | | | | | | | | | | | | | | | | | Fix the account mapping so that we always use what is provided by the request. With this fix, "/mnt/gluster-object/AUTH_ufo0" directory hierachies won't be created. But this fix does not restore the one-to-one account to gluster volume name mapping. When the user runs the /usr/bin/gluster-swift-gen-builders script, it still only allows one cluster volume, and that is the only volume that is used. So the account names are effectively ignored. A future set of changes will address that problem. Change-Id: I2df608de2f00fd356a09c500d49fe12cc1a0a574 BUG: 870589 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4222 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Mohammed Junaid <junaid@redhat.com>
* Replace "ssh foo gluster" with "gluster --remote-host=foo"Pete Zaitcev2012-11-291-4/+4
| | | | | | | | | | | | | | | | | | | With the current lack of authentication, there is no security improvement with passwordless ssh replaced. However, it is more convenient to install UFO if you do not have to generate ssh keys every time. There is a downside the gluster-server RPM becomes required. BUG: 880050 Change-Id: I04f4ad666ac96713055723bd432131a4db325b4f Signed-off-by: Pete Zaitcev <zaitcev@redhat.com> Reviewed-on: http://review.gluster.org/4231 Reviewed-by: Peter Portante <pportant@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Anand Avati <avati@redhat.com>
* object-storage: Add a sorted list comparison to testPete Zaitcev2012-11-211-2/+2
| | | | | | | | | | | | | | | | | | After the commit b0cb7aaf for bz#870589, which adds 2 tarballs with layout patterns, Swift tests sometimes fail (depending on the kind of filesystem they are running at and the phase of the moon). As pattern tarball is unpacked, the underlying filesystem is free to return directory listings in any order, and straightforward use of '==' operator fails. As it turns out, one of the two comparisons had set() applied to the list already. So, add it to the other one. BUG: 874390 Change-Id: I02de99593b9567a13076113d58e242b079fde002 Signed-off-by: Pete Zaitcev <zaitcev@redhat.com> Reviewed-on: http://review.gluster.org/4165 Reviewed-by: Peter Portante <pportant@redhat.com> Tested-by: Peter Portante <pportant@redhat.com>
* object-storage: del 1.4.8 ref from dir hierarchyPeter Portante2012-11-2136-0/+3052
Remove the "swift/1.4.8" directory from the hierarchy, replacing it with "ufo". Change-Id: I60ba171182d7402a2193616c1c95c90cd2ead5bc BUG: 870589 Signed-off-by: Peter Portante <peter.portante@redhat.com> Reviewed-on: http://review.gluster.org/4200 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Pete Zaitcev <zaitcev@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-01 01:31:52 +0000