summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--doc/markdown/quick_start_guide.md1
-rw-r--r--doc/markdown/s3.md153
2 files changed, 154 insertions, 0 deletions
diff --git a/doc/markdown/quick_start_guide.md b/doc/markdown/quick_start_guide.md
index 4ebabe6..0b0990f 100644
--- a/doc/markdown/quick_start_guide.md
+++ b/doc/markdown/quick_start_guide.md
@@ -5,6 +5,7 @@
* [System Setup](#system_setup)
* [Gluster For Swift Setup](#swift_setup)
* [Using Gluster for Swift](#using_swift)
+* [Accessing over Amazon S3 API](s3.md)
* [What now?](#what_now)
<a name="overview" />
diff --git a/doc/markdown/s3.md b/doc/markdown/s3.md
new file mode 100644
index 0000000..75a9954
--- /dev/null
+++ b/doc/markdown/s3.md
@@ -0,0 +1,153 @@
+# Accessing GlusterFS over Amazon S3 API
+This guide assumes that you have a GlusterFS cluster with volumes exported over Swift API using gluster-swift.
+
+### Install and configure swift3 middleware
+Install swift3 middleware from source or via whatever packaging system you may be using. Here is how you can install from source:
+
+```sh
+git clone https://github.com/openstack/swift3
+cd swift3
+python setup.py install
+```
+
+Modify `/etc/swift/proxy-server.conf` to add swift3 middleware in the pipeline to the left of auth middleware (tempauth/gswauth) and add swift3 filter section as illustrated below:
+
+```ini
+[pipeline:main]
+pipeline = catch_errors cache swift3 tempauth proxy-server
+
+[filter:swift3]
+use = egg:swift3#swift3
+```
+If you are using gswauth, insert swift3 to the left of gswauth similar to the tempauth example above. You can also configure swift3 middleware to [work with keystone](https://github.com/openstack/swift3). You can also [configure tunables](https://github.com/openstack/swift3/blob/master/etc/proxy-server.conf-sample) in the swift3 middleware filter section.
+
+Reload the proxy server process for the change in pipeline to take effect:
+
+```sh
+swift-init proxy reload
+```
+
+If you are not using any authentication filters, refer to [this section](#no_auth) for example.
+
+### S3 API examples
+
+The examples below use the [s3curl](https://aws.amazon.com/code/128) perl script which is a S3 authentication wrapper around curl command. Modify `s3curl.pl` to point to node running proxy server in your cluster.
+
+```perl
+# begin customizing here
+my @endpoints = ( 'localhost');
+```
+
+These examples assume you have an account (GlusterFS volume) named `test` and an admin user named `tester` with password `testing`:
+
+**Create a bucket**
+```sh
+./s3curl.pl --id 'test:tester' --key 'testing' --put /dev/null -- -k -v -s http://localhost:8080/bucket1
+```
+
+**Upload object to the bucket**
+```sh
+./s3curl.pl --id 'test:tester' --key 'testing' --put ./README -- -k -v -s http://localhost:8080/bucket1/a/b/c
+```
+
+**Download an object from the bucket**
+```sh
+./s3curl.pl --id 'test:tester' --key 'testing' --get -- -k -v -s http://localhost:8080/bucket1/a/b/c
+```
+
+**List buckets**
+```sh
+./s3curl.pl --id 'test:tester' --key 'testing' --get -- -k -v -s http://localhost:8080
+```
+
+**List objects in bucket**
+```sh
+./s3curl.pl --id 'test:tester' --key 'testing' --get -- -k -v -s http://localhost:8080/bucket1
+```
+
+**Delete object**
+```sh
+./s3curl.pl --id 'test:tester' --key 'testing' --del -- -k -v -s http://localhost:8080/bucket1/a/b/c
+```
+
+**Delete bucket (should be empty)**
+```sh
+./s3curl.pl --id 'test:tester' --key 'testing' --del -- -k -v -s http://localhost:8080/bucket1
+```
+
+**Using boto module in python to access GlusterFS cluster over S3 API**
+```python
+#!/usr/bin/env python
+
+from boto.s3.connection import S3Connection
+from boto.s3.connection import OrdinaryCallingFormat
+conn = S3Connection(aws_access_key_id='test:tester',
+ aws_secret_access_key='testing',
+ host='localhost', port=8080, is_secure=False,
+ calling_format=OrdinaryCallingFormat())
+
+# List buckets example
+buckets_list = conn.get_all_buckets()
+for bucket in buckets_list:
+ print(bucket.name)
+```
+
+### swift3 middleware and gswauth compatibility
+
+Swift3 middleware can be easily used with gswauth when `auth_type` in gswauth is configured to be `Plaintext`.
+The AWS S3 client uses password in plaintext to compute HMAC signature.
+
+However, when `auth_type` in gswauth is configured to be `Sha1` or `Sha512`, gswauth can only use the stored hashed password to compute HMAC signature. This results in signature mismatch although the user credentials are correct. The only way for S3 clients to authenticate is by providing SHA1/SHA512 of password as input to it’s HMAC function. To generate hash of password, the S3 clients will have to know `auth_type` and `auth_type_salt` beforehand. Here is an example of gswauth configuration:
+
+```ini
+[filter:gswauth]
+use = egg:gluster_swift#gswauth
+set log_name = gswauth
+super_admin_key = gswauthkey
+metadata_volume = gsmetadata
+token_life = 86400
+max_token_life = 86400
+auth_type = sha1
+auth_type_salt = gswauthsalt
+s3_support = on
+```
+
+Note that you have to explicitly set `s3_support` to on.
+
+**Example: Create a bucket**
+
+Generate key
+
+```python
+#!/usr/bin/env python
+import hashlib
+
+key = 'testing'
+salt = 'gswauthsalt'
+
+enc_key = '%s%s' % ('gswauthsalt', 'testing')
+enc_val = hashlib.sha1(enc_key).hexdigest()
+print(enc_val) # f636f4ecc148efdce89ac471f805d7f51cec348f
+```
+This is not required when `auth_type` is set to `Plaintext`
+
+```sh
+./s3curl.pl --id 'test:tester' --key 'f636f4ecc148efdce89ac471f805d7f51cec348f' --put /dev/null -- -k -v -s http://localhost:8080/bucket1
+```
+
+<a name="no_auth" />
+### No auth middleware in pipeline
+In local and insecure deployments such as within an organization/department where no authentication middleware is in the proxy pipeline, you should just use glusterfs volume name as `id` or `AWSAccessKeyId`.
+
+**Example: Create a bucket**
+
+```sh
+./s3curl.pl --id 'test' --key 'blah' --put /dev/null -- -k -v -s http://localhost:8080/bucket1
+```
+In this example, `test` is the name of glusterfs volume and key can be any string and is ignored.
+
+#### References:
+* [Configure Object Storage with the S3 API](http://docs.openstack.org/juno/config-reference/content/configuring-openstack-object-storage-with-s3_api.html)
+* [S3 APIs on OpenStack Swift](http://www.buildcloudstorage.com/2011/11/s3-apis-on-openstack-swift.html)
+* [swift3 middleware](https://github.com/openstack/swift3)
+* [swauth and swift3 compatibility](https://github.com/openstack/swauth/blob/master/doc/source/index.rst#swift3-middleware-compatibility)