gluster-swift.git/tools, branch master

Enhance object expiration

2016-04-15T12:27:54+00:00

This change re-introduces object expiration feature with some internal
differences from earlier implementation such as:

* Creation of zero-byte tracker object is performed directly on the
  mount point. Earlier HTTP request was sent to object server
  for the same. This incurred overhead of metadata creation for
  these zero-byte files which isn't necessarry as all required
  information is encoded in the path itself.
* Crawling of zero-byte tracker objects is done by the object
  expirer daemon itself and not container server.
* Deletion of tracker object is performed by the object expiration daemon
  directly on mount point.

Deletion of actual data object is not carried out by object expiration
daemon directly. The object expirer sends a DELETE request to object
server which deletes the actual object. This behaviour is not changed.

There is no change in behaviour in comparison with older implementation.
This is asserted by re-enabling existing functional tests without
any changes.

Change-Id: I01dc77cc4db3be3147d54e3aa2a19ed182498900
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/13913
Reviewed-by: Thiago da Silva 
Tested-by: Thiago da Silva

Refactor read_metadata() method

2016-01-12T04:47:05+00:00

This change:
* Simplifies read_metadata() method.
* Validates pickle header before attempting to unpickle.

This change does NOT fix the security vulnerability itself.
That would be sent as a separate change.

Change-Id: Id95bd584f3ad00fb075456544495f17f7038f991
Signed-off-by: Prashanth Pai 
Reviewed-by: Thiago da Silva 
Tested-by: Thiago da Silva 
Reviewed-on: http://review.gluster.org/13220

Update repo

2016-01-06T15:53:12+00:00

This is a squashed commit imported from this repo:
https://github.com/openstack/swiftonfile/tree/icehouse

Contains the follwing commits from above mentioned repo:

eb50236 Merge "Backport: Fix metadata overall limits bug" into icehouse
79ea52a Backport: Fix metadata overall limits bug
bc43f0b Fix inconsistent data being returned on GET
ad0bb79 Import HTTPBadRequest from swift's module
74d02e6 Exclude .trashcan dir from container listing
b2dbc15 Catch ESTALE in addition to ENOENT
8d60b48 Properly handle read_metadata() exceptions
6762fc6 Fix object server leaking file descriptors
2842e82 Fix API incompatibility in update_metadata()
2beeef6 Merge "Remove swiftkerbauth code" into icehouse
93dbcb5 Update object-expirer.conf with explanations
c9d2f09 Merge "Check if /etc/swift exists in ring builder" into icehouse
d66c14c Remove swiftkerbauth code
3142ed2 Add object expiration functests
97153d1 Merge "Cleanup functest and undo old patch" into icehouse
bc234d0 Remove old travis config file and fix typo
260c8ef Check if /etc/swift exists in ring builder
637dac9 Cleanup functest and undo old patch
051e068 Merge pull request #35 from prashanthpai/backport-1
be104a3 Merge pull request #36 from prashanthpai/backport-2
ff76f42 fix issue with GET on large object (icehouse-backport)
04d0a99 Fix unlink call after successful rename
4c6ca1d updating README file with project name change
10b2680 Merge pull request #18 from thiagol11/icehouse
5bcab8f Updating version on __init__ file
5c2cba2 Merge pull request #15 from thiagol11/update_spec
52b00a8 updating spec file to add dependency on swift icehouse
ae7c93b Merge pull request #6 from prashanthpai/rebase
191e55b Revert: allow non-root user to run functests
cb7e968 Modify unit tests and func tests
d23fd1b Sync with OpenStack Swift v1.13.1
b6d1671 Merge pull request #12 from pushpesh/functionalnosetestremove
962622b Merge pull request #8 from thiagol11/update_readme
4560857 Merge pull request #9 from prashanthpai/spec-expirer
be0ae7e Minor update
65000f1 Removing functionalnosetests
8ab1069 Fix object-expirer.conf-gluster RPM build error
afee30f added new support filesystem section
527b01f updated README.md to Swift-On-File
9a240c7 Merge pull request #3 from thiagol11/add_jenkins_to_travis
34b5a8b removing blank lines
3568b64 fixing missing fi
d8f5b0f adding support to run jenkins triggered by travis
6f4a88c Removing functionalnosetests
8041944 Update README.md
c015148 Merge pull request #2 from thiagol11/master
3ddd952 fixing travis file to run correct unit test
c582669 adding travis status badge to README
8093096 adding py26 unit testing to travis
37835fd trigger travis build
cb6332a adding travis ci testing

All tests have been run sucessfully against this.
tox -e p2p8,py27,functest

Change-Id: I096b611da852d3eb3913844034b443b8272c2ac4
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/13188

Removing functionalnosetests

2014-04-28T05:09:18+00:00

With the Icehouse release of openstack-swift functional
case classified as fucntionalnose tests have been moved
to functional cases.Although we copied test suites from
test/fucntionalnose/ to test/functional/ but we still
have same cases under test/functionalnose.This might
cause duplicate tests run.

Change-Id: I025206467aad364debd9050b3186e1379d89ffaf
Signed-off-by: Pushpesh Sharma 
Reviewed-on: http://review.gluster.org/7516
Reviewed-by: Prashanth Pai 
Tested-by: Prashanth Pai 
Reviewed-by: Chetan Risbud 
Reviewed-by: Thiago da Silva

Functional tests for SwiftKerbAuth filter.

2014-03-14T18:56:15+00:00

This provides an infrastructure for swiftkerbauth
related functional test cases.
More test cases will be added later.
Added a section in swiftkerbauth guide about how to run
functional tests.

test/functional_auth/swiftkerbauth
----------------------------------

A new authentication filter related functional
tests and configuration to reside here.
The configuration would help setup the
environment. All the generic functional tests
should run fine with PASSIVE mode of swiftkerbatuh.
Please refere to swiftkerbatuh documentation for
ACTIVE/PASSIVE mode of working.

swiftkerbauth/test_swkrbath_active.py
-------------------------------------
This file has all the testcases of active mode of
swiftkerbauth. More test cases to be added later.

SwiftKerbAuth related test cases are meant to run
on the setup where SwiftKerbAuth is setup and
installed.

Change-Id: Ibc2a3945f5c9b6714475fcec0ee9d153debb48e3
Signed-off-by: Chetan Risbud 
Reviewed-on: http://review.gluster.org/6925
Reviewed-by: Luis Pabon 
Tested-by: Luis Pabon

Sync with OpenStack Swift v1.13.0

2014-03-13T12:09:19+00:00

Also, bumped version of gluster-swift to v1.13.0

Change-Id: I797dc704c9523540cba847b1e8ff3da97b79630c
Signed-off-by: Prashanth Pai 
Reviewed-on: http://review.gluster.org/7229
Reviewed-by: Chetan Risbud 
Reviewed-by: Luis Pabon 
Tested-by: Luis Pabon

Sync with OpenStack v1.11.0 Jan 10 2014

2014-01-15T19:28:08+00:00

Updated tox.ini, functional tests, and proxy
unit tests.

BUG: https://bugs.launchpad.net/bugs/1268017

Change-Id: I5ff8359b8abdb8fe5ae82492c12f57c395992735
Signed-off-by: Luis Pabon 
Reviewed-on: http://review.gluster.org/6682
Reviewed-by: Thiago da Silva 
Tested-by: Thiago da Silva

renaming swauth tools to gswauth

2013-12-06T15:44:08+00:00

cli tools for account and user mgmt have been renamed
from swauth-* to gswauth-*

Updated other configuration and test files accordingly

Change-Id: Iced3bb27fbd09da45754ddb264f8fb4528ab423c
Signed-off-by: Thiago da Silva 
Reviewed-on: http://review.gluster.org/6417
Reviewed-by: pushpesh sharma 
Tested-by: pushpesh sharma 
Reviewed-by: Shilpa MJ 
Tested-by: Shilpa MJ 
Reviewed-by: Luis Pabon

adding tabular output to swauth-list

2013-11-27T12:40:06+00:00

swauth-list now shows lists of accounts, users and groups
in a tabular format (default). JSON and plain-text options
are still available

Change-Id: I897009f5f6c05aa4c14e306bb6ad35234e993fcd
Signed-off-by: Thiago da Silva 
Reviewed-on: http://review.gluster.org/6347
Reviewed-by: Luis Pabon 
Tested-by: Luis Pabon

Setup and run in a virtual environment

2013-11-27T12:37:11+00:00

Using tox, developers can down drop into an environment
which sets up and runs gluster-swift with gswauth authentication.
The developer is then dropped into a shell which is running inside
the tox environment.  OpenStack Swift and Gluster-Swift are
available to the developer.  Once the developer exists the shell,
the environment will be cleaned up and terminated.

Usage:
tox -e run

Example:
$ tox -e run
GLOB sdist-make: /home/lpabon/gluster-swift/setup.py
run inst-nodeps: /home/lpabon/gluster-swift/.tox/dist/gluster_swift-1.10.1-0.zip
run runtests: commands[0] | bash tools/tox_run.sh
Ring files are prepared in /etc/swift. Please restart object store services
Redirecting to /bin/systemctl start  memcached.service
Starting proxy-server...(/etc/swift/proxy-server.conf)
Starting container-server...(/etc/swift/container-server.conf)
Starting account-server...(/etc/swift/account-server.conf)
Starting object-server...(/etc/swift/object-server.conf)
bash-4.2$ swauth-list -K gswauthkey
{"accounts": [{"name": "test"}, {"name": "test2"}]}
bash-4.2$ swauth-list -K gswauthkey test
{"services": {"storage": {"default": "local", "local":
"http://127.0.0.1:8080/v1/AUTH_test"}}, "account_id":
"AUTH_test", "users": [{"name": "tester"}, {"name": "tester3"}]}
bash-4.2$ swauth-list -K gswauthkey test tester
{"groups": [{"name": "test:tester"}, {"name": "test"},
{"name": ".admin"}], "auth": "plaintext:testing"}
bash-4.2$ exit
exit
Redirecting to /bin/systemctl stop  memcached.service
Signal proxy-server  pid: 22862  signal: 15
Signal container-server  pid: 22863  signal: 15
Signal account-server  pid: 22864  signal: 15
Signal object-server  pid: 22865  signal: 15
proxy-server (22862) appears to have stopped
container-server (22863) appears to have stopped
account-server (22864) appears to have stopped
object-server (22865) appears to have stopped
  run: commands succeeded
    congratulations :)

Change-Id: I98de5d1b1698b4cd355888f2c15a46df021a9e87
Signed-off-by: Luis Pabon 
Reviewed-on: http://review.gluster.org/6365
Reviewed-by: pushpesh sharma 
Tested-by: pushpesh sharma