| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change ports most of swiftonfile object server fixes and changes
into gluster-swift. Storage policy as a feature is not usable here
(it doesn't make sense).
The hacky way of creating zero byte tracker objects for object
expiration has not been ported to this release due to scalability
issues and the need to have a separate volume.
Change-Id: I17ba27dacea9ac000bdb8934700996e4d17f4251
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/13269
Reviewed-by: Thiago da Silva <thiago@redhat.com>
Tested-by: Thiago da Silva <thiago@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a squashed commit imported from this repo:
https://github.com/openstack/swiftonfile/tree/icehouse
Contains the follwing commits from above mentioned repo:
eb50236 Merge "Backport: Fix metadata overall limits bug" into icehouse
79ea52a Backport: Fix metadata overall limits bug
bc43f0b Fix inconsistent data being returned on GET
ad0bb79 Import HTTPBadRequest from swift's module
74d02e6 Exclude .trashcan dir from container listing
b2dbc15 Catch ESTALE in addition to ENOENT
8d60b48 Properly handle read_metadata() exceptions
6762fc6 Fix object server leaking file descriptors
2842e82 Fix API incompatibility in update_metadata()
2beeef6 Merge "Remove swiftkerbauth code" into icehouse
93dbcb5 Update object-expirer.conf with explanations
c9d2f09 Merge "Check if /etc/swift exists in ring builder" into icehouse
d66c14c Remove swiftkerbauth code
3142ed2 Add object expiration functests
97153d1 Merge "Cleanup functest and undo old patch" into icehouse
bc234d0 Remove old travis config file and fix typo
260c8ef Check if /etc/swift exists in ring builder
637dac9 Cleanup functest and undo old patch
051e068 Merge pull request #35 from prashanthpai/backport-1
be104a3 Merge pull request #36 from prashanthpai/backport-2
ff76f42 fix issue with GET on large object (icehouse-backport)
04d0a99 Fix unlink call after successful rename
4c6ca1d updating README file with project name change
10b2680 Merge pull request #18 from thiagol11/icehouse
5bcab8f Updating version on __init__ file
5c2cba2 Merge pull request #15 from thiagol11/update_spec
52b00a8 updating spec file to add dependency on swift icehouse
ae7c93b Merge pull request #6 from prashanthpai/rebase
191e55b Revert: allow non-root user to run functests
cb7e968 Modify unit tests and func tests
d23fd1b Sync with OpenStack Swift v1.13.1
b6d1671 Merge pull request #12 from pushpesh/functionalnosetestremove
962622b Merge pull request #8 from thiagol11/update_readme
4560857 Merge pull request #9 from prashanthpai/spec-expirer
be0ae7e Minor update
65000f1 Removing functionalnosetests
8ab1069 Fix object-expirer.conf-gluster RPM build error
afee30f added new support filesystem section
527b01f updated README.md to Swift-On-File
9a240c7 Merge pull request #3 from thiagol11/add_jenkins_to_travis
34b5a8b removing blank lines
3568b64 fixing missing fi
d8f5b0f adding support to run jenkins triggered by travis
6f4a88c Removing functionalnosetests
8041944 Update README.md
c015148 Merge pull request #2 from thiagol11/master
3ddd952 fixing travis file to run correct unit test
c582669 adding travis status badge to README
8093096 adding py26 unit testing to travis
37835fd trigger travis build
cb6332a adding travis ci testing
All tests have been run sucessfully against this.
tox -e p2p8,py27,functest
Change-Id: I096b611da852d3eb3913844034b443b8272c2ac4
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/13188
|
|
|
|
|
|
|
|
|
|
|
| |
The file descriptor is not being closed because
it is self._fd is None
Change-Id: I7edc8a78b09bdd76d59ac8f3dbc809af652f9b0e
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/7315
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Preventing access to expired objects
------------------------------------
Re-enabled accepting X-Delete-At and X-Delete-After headers. During a GET on
an expired object, DiskFileExpired is raised by DiskFile class. This will
result in object-server returning HTTPNotFound (404) to the client.
Tracking objects to be deleted
------------------------------
Objects to be deleted are tracked using "tracker objects". These are PUT into
a special account(a volume, for now). These zero size "tracker objects" have
names that contain:
* Expiration timestamp
* Path of the actual object to be deleted
Deleting actual objects from GlusterFS volume
---------------------------------------------
The object-expirer daemon runs a pass once every X seconds. For every pass it
makes, it queries the special account for "tracker objects". Based on
(timestamp, path) present in name of "tracker objects", object-expirer then
deletes the actual object and the corresponding tracker object.
To run object-expirer forever:
swift-init object-expirer start
To run just once:
swift-object-expirer -o -v /etc/swift/object-expirer.conf
Caveat/Limitation: Object-expirer needs a separate account(volume) that
is not used by other services like gswauth. By default, this volume is
named "gsexpiring" and is configurable.
More info about object expiration:
http://docs.openstack.org/developer/swift/overview_expiring_objects.html
Change-Id: I876995bf4f16ef4bfdff901561e0558ecf1dc38f
Signed-off-by: Prashanth Pai <ppai@redhat.com>
Reviewed-on: http://review.gluster.org/6891
Tested-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-by: pushpesh sharma <psharma@redhat.com>
Tested-by: pushpesh sharma <psharma@redhat.com>
Reviewed-by: Chetan Risbud <crisbud@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
PUT of a directory fails with gluster-swift when there exists
object of a same name. This is because the objects and directories are placed on
the glusterfs. And hence the filesystem semantics are applicable there.
Exceptions raised in such situation are needed to be handled and reported
correctly back to proxy-server with HTTPConflict as a error code. Although
swift still continues to choose 503 as the best respose in such cases. No
tracebacks reported.
Fix covers the case when there exists a directory and object of the same
name is PUT.
Code changes fixes both the failure cases mentioned in the bug.
Examples of failing PUT requests:
1) curl -v -X PUT http://127.0.0.1:8080/v1/AUTH_test/c1/dir1/obj2/anotherobject
-d'asdasdsadA'
-- obj2 was already an object.
2)curl -v -H 'Content-Length: 0' -H 'Content-Type: application/directory' -X PUT
http://127.0.0.1:8080/v1/AUTH_test/c1/obj1
-- obj1 was already and object
Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1071021
Change-Id: Id3042d920e3f99e740d4042ef5907ac8c59e04db
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/7181
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Tested-by: Prashanth Pai <ppai@redhat.com>
|
|
|
|
|
|
|
| |
Change-Id: I994b0e45d4fdb898378de7cd103f7e429a7fd9cc
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6662
Reviewed-by: Thiago Da Silva <thiago@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The DiskWriter was closing the file descriptor when it finished
writing but initializing it to None. Therefore, the DiskFile
context manager would then try to close it also.
Change-Id: I188ec814d025e28c7b89532f0502ebf1d4a20a09
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6317
Reviewed-by: Peter Portante <pportant@redhat.com>
Reviewed-by: pushpesh sharma <psharma@redhat.com>
Tested-by: pushpesh sharma <psharma@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/6487
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During a post request, the object-server is ovewriting
the existing object metadata. This fix prevents the overwrite
of the system metadata while it allows for the user to
add/remove user metadata
Change-Id: Ic62cd064589b625ee425a9934be8766650622c13
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6254
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Reviewed-on: http://review.gluster.org/6315
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As of October 28th, 2013, we rebase to OpenStack Swift master (commit
4bfe674) to pick up the lastest officially supported DiskFile API
changes. We use a snapshot of OpenStack Swift stored in the
gluster-swift launchpad downloads area so that we can deliberately
rebase at our own pace.
With this refactoring, all the code for handling I/O is wholly
contained in the swift tree for object operations. This will allow us
to use a different fs_utils implementation in the future (for example,
one based on a yet-to-be-implemented python bindings over libgfapi).
This also means the "Fake_file" class has been removed entirely.
Change-Id: I767983f88c59786e30b6c64da16d1cb6ab3c3e7f
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5993
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I90821230a1a7100c74d97cccc9c445251d0f65e7
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/6157
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The modified test case provides the complete coverage over
Fake_file class. These interfaces are to be used for directory
type of the objects.
BUG:987841
https://bugzilla.redhat.com/show_bug.cgi?id=987841
Change-Id: I2401423d2013ce7d90c454c72fbb60f4bc05ceb1
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/6037
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I5e7af267bcbb510108f62a9c66b9baa6771cc8fe
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5985
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I1add2e7e74b59bf24ef46d9053723f0243ee08c5
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5984
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
| |
Change-Id: I18cdb6c9e8aa56014f49d2fcf4458c3499710ddf
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5983
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added a fake_file class that implements minimal set of functions that
are invoked by the code in GET.
BUG: 987841
https://bugzilla.redhat.com/show_bug.cgi?id=987841
Change-Id: I5bdf5be1c0c4c8231f34c9be529e6edc83774f2e
Signed-off-by: Chetan Risbud <crisbud@redhat.com>
Reviewed-on: http://review.gluster.org/5511
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For whatever reason, it appears that GlusterFS, or perhaps FUSE can
continuously return ENOENT on a rename system call even when we have
double checked that there is no reason to do so. That is a bug for
that sub system. However, our response to that bug can result in an
infinite loop, which is bad.
This code reduces that to 10 attempts.
In addition, we restructed the open retry loop to match, providing
module constants for the upper bounds of both retry loops.
BUG: XXXXXX (https://bugzilla.redhat.com/show_bug.cgi?id=XXXXXX)
Change-Id: Ia2d6dd427daba3ea0461863c5ffe3aef27c88f9b
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5670
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The code changes are basically:
* Apply refactoring in the DiskFile class to use the new DiskWriter
abstraction
* Move and rename our diskfile module to match upstream
* ThreadPools allow us to remove the tpool usage around fsync
* Update the Ring subclass to support the get_part() method
* Update to use the 1.9.1 proxy server unit tests
* Move the DebugLogger class to test.unit
* Rebuild the Rings to use the new layout
* Remove backup ring builder files
* Update spec files to 1.9.1, and tox to use swift 1.9.1
* Updated version to 1.9.0-0
Change-Id: Ica12cac8b351627d67500723f1dbd8a54d45f7c8
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Signed-off-by: Luis Pabon <lpabon@redhat.com>
Reviewed-on: http://review.gluster.org/5331
|
|
Change-Id: I75373c63b0b4d80e317208446c3addca07d1f110
Signed-off-by: Peter Portante <peter.portante@redhat.com>
Reviewed-on: http://review.gluster.org/5083
Reviewed-by: Luis Pabon <lpabon@redhat.com>
Tested-by: Luis Pabon <lpabon@redhat.com>
|