Kerberos Authentication Filter for Red Hat Storage and OpenStack Swift ---------------------------------------------------------------------- Red Hat Storage not only provides file system access to its data, but also object-level access. The latter is implemented with OpenStack Swift, and allows containers and objects to be stored and retrieved with an HTTP-based API. Red Hat Storage 2.0 comes with a simple authentication filter that defines user accounts as a static list in the Swift configuration file. For this project, we implemented a new authentication filter that uses Kerberos tickets for single sign on authentication, and grants administrator permissions based on the user's group membership in a directory service like Red Hat Enterprise Linux Identity Management or Microsoft Active Directory. * Building To build the swiftkerbauth and apachekerbauth RPM packages, change into the respective directory and run ./build.sh * Installation ** Swift Server Install the swiftkerbauth RPM on all Red Hat Storage nodes that will provide object-level access via Swift. To active the Kerberos authentication filter, add "kerbauth" in the /etc/swift/proxy-server.conf pipeline parameter: [pipeline:main] pipeline = healthcheck cache kerbauth proxy-server Set the URL of the Apache server that will be used for authentication with the ext_authentication_url parameter in the same file: [filter:kerbauth] paste.filter_factory = swiftkerbauth:filter_factory ext_authentication_url = http://AUTHENTICATION_SERVER/cgi-bin/swift-auth If the Swift server is not one of your Gluster nodes, edit /etc/swift/fs.conf and change the following lines in the DEFAULT section: mount_ip = RHS_NODE_HOSTNAME remote_cluster = yes Activate the changes by running swift-init main restart For troubleshooting, check /var/log/messages. ** Authentication Server On the authentication server, install the apachekerbauth package. Edit /etc/httpd/conf.d/swift-auth.conf and set the KrbAuthRealms and Krb5KeyTab parameters. The keytab must contain a HTTP/$HOSTNAME principal. Usually, you will have to create the Kerberos principal on the KDC, export it, and copy it to a keytab file on the Apache server. If SELinux is enabled, allow Apache to connect to memcache and activate the changes by running setsebool -P httpd_can_network_connect 1 setsebool -P httpd_can_network_memcache 1 service httpd reload For troubleshooting, see /var/log/httpd/error_log. * Testing The tests were done with curl on a machine set up as an IDM client, using the Gluster volume rhs_ufo1. In IDM, we created the following user groups: - auth_reseller_admin Users in this group get full access to all Swift accounts. - auth_rhs_ufo1 Users in this group get full access to the rhs_ufo1 Swift account. Next, we created the following users in IDM: - auth_admin Member of the auth_reseller_admin group - rhs_ufo1_admin Member of the auth_rhs_ufo1 group - jsmith No relevant group membership The authentication tokens were then retrieved with the following commands: kinit auth_admin curl -v -u : --negotiate --location-trusted \ http://rhs1.example.com:8080/auth/v1.0 kinit rhs_ufo1_admin curl -v -u : --negotiate --location-trusted \ http://rhs1.example.com:8080/auth/v1.0 kinit jsmith curl -v -u : --negotiate --location-trusted \ http://rhs1.example.com:8080/auth/v1.0 Each of these commands should output the following two lines: < X-Auth-Token: AUTH_tk4097146ed3814e026209556eeb121fe0 ... 
1365195860 / auth_admin,auth_reseller_admin
The first line contains the authentication token that is used in subsequent requests. The second line is printed by the swift-auth CGI script for debugging - it lists the token expiration (in seconds since January 1, 1970) and the user's groups. Next, we try to get information about the Swift account, replacing the AUTH_tk* with one of the tokens we got with the commands above. This should display statistics, and the list of container names when used with the the admin users. For jsmith, you should get a 403 Forbidden error. curl -v -X GET \ -H 'X-Auth-Token: AUTH_tk4097146ed3814e026209556eeb121fe0' \ http://rhs1.example.com:8080/v1/AUTH_rhs_ufo1 With one of the admin accounts, create a new container and a new object in that container: curl -v -X PUT \ -H 'X-Auth-Token: AUTH_tk4097146ed3814e026209556eeb121fe0' \ http://rhs1.example.com:8080/v1/AUTH_rhs_ufo1/pictures curl -v -X PUT \ -H 'X-Auth-Token: AUTH_tk4097146ed3814e026209556eeb121fe0' \ -H 'Content-Length: 0' \ http://rhs1.example.com:8080/v1/AUTH_rhs_ufo1/pictures/pic1.png Grant permission for jsmith to list and download objects from the pictures container: curl -v -X POST \ -H 'X-Auth-Token: AUTH_tkdbf7725c1e4ad1ebe9ab0d7098d425f2' \ -H 'X-Container-Read: jsmith' \ http://rhs1.example.com:8080/v1/AUTH_rhs_ufo1/pictures List the container contents using the authentication token for jsmith: curl -v -X GET \ -H 'X-Auth-Token: AUTH_tkef8b417ac0c2a73a80ab3b8db85254e2' \ http://rhs1.example.com:8080/v1/AUTH_rhs_ufo1/pictures Try to access a resource without an authentication token. This will return a 303 redirect: curl -v -X GET \ http://rhs1.example.com:8080/v1/AUTH_rhs_ufo1/pictures/pic1.png For curl to follow the redirect, you need to specify additional options. With these, and with a current Kerberos ticket, you should get the Kerberos user's cached authentication token, or a new one if the previous token has expired. curl -v -u : --negotiate --location-trusted -X GET \ http://rhs1.example.com:8080/v1/AUTH_rhs_ufo1/pictures/pic1.png * Implementation Details ** Architecture The Swift API is HTTP-based. As described in the Swift documentation [1], clients first make a request to an authentication URL, providing a username and password. The reply contains a token which is used in all subsequent requests. Swift has a chain of filters through which all client requests go. The filters to use are configured with the pipeline parameter in /etc/swift/proxy-server.conf: [pipeline:main] pipeline = healthcheck cache tempauth proxy-server For the single sign authentication, we added a new filter called "kerbauth" and put it into the filter pipeline in place of tempauth. The filter checks the URL for each client request. If it matches the authentication URL, the client is redirected to a URL on a different server. The URL is handled by a CGI script, which is set up to authenticate the client with Kerberos negotiation, retrieve the user's system groups [2], store them in a memcache ring shared with the Swift server, and return the authentication token to the client. When the client provides the token as part of a resource request, the kerbauth filter checks it against its memcache, grants administrator rights based on the group membership retrieved from memcache, and either grants or denies the resource access. [1] http://docs.openstack.org/api/openstack-object-storage/1.0/content/authentication-object-dev-guide.html [2] The user data and system groups are usually provided by Red Hat Enterprise Linux identity Management or Microsoft Active Directory. The script relies on the system configuration to be set accordingly (/etc/nsswitch.conf). ** swiftkerbauth.py The script /usr/lib/python2.6/site-packages/swiftkerbauth.py began as a copy of the tempauth.py script from /usr/lib/python2.6/site-packages/swift/common/middleware. It contains the following modifications, among others: In the __init__ method, we read the ext_authentication_url parameter from /etc/swift/proxy-server.conf. This is the URL that clients are redirected to when they access either the Swift authentication URL, or when they request a resource without a valid authentication token. The configuration in proxy-server.conf looks like this: [filter:kerbauth] paste.filter_factory = swiftkerbauth:filter_factory ext_authentication_url = http://rhel6-4.localdomain/cgi-bin/swift-auth The authorize method was changed so that global administrator rights are granted if the user is a member of the auth_reseller_admin group. Administrator rights for a specific account like vol1 are granted if the user is a member of the auth_vol1 group. [3] The denied_response method was changed to return a HTTP redirect to the external authentication URL if no valid token was provided by the client. Most of the handle_get_token method was moved to the external authentication script. This method now returns a HTTP redirect. In the __call__ and get_groups method, we removed support for the HTTP_AUTHORIZATION header, which is only needed when Amazon S3 is used. Like tempauth.py, swiftkerbauth.py uses a Swift wrapper to access memcache. This wrapper converts the key to an MD5 hash and uses the hash value to determine on which of a pre-defined list of servers to store the data. [3] "auth" is the default reseller prefix, and would be different if the reseller_prefix parameter in proxy-server.conf was set. ** swift-auth CGI Script swift-auth resides on an Apache server and assumes that Apache is configured to authenticate the user before this script is executed. The script retrieves the username from the REMOTE_USER environment variable, and checks if there already is a token for this user in the memcache ring. If not, it generates a new one, retrieves the user's system groups with "id -Gn USERNAME", stores this information in the memcache ring, and returns the token to the client. For the Swift filter to be able to find the information, it was important to use the Swift memcached module. Because we don't want to require a full Swift installation on the authentication server, /usr/lib/python2.6/site-packages/swift/common/memcached.py from the Swift server was copied to /var/www/cgi-bin on the Apache server. To allow the CGI script to connect to memcache, the SELinux booleans httpd_can_network_connect and httpd_can_network_memcache had to be set. The tempauth filter uses the uuid module to generate token strings. This module creates and runs temporary files, which leads to AVC denial messages in /var/log/audit/audit.log when used from an Apache CGI script. While the module still works, the audit log would grow quickly. Instead of writing an SELinux policy module to allow or to silently ignore these accesses, the swift-auth script uses the "random" module for generating token strings. Red Hat Enterprise Linux 6 comes with Python 2.6 which only provides method to list the locally defined user groups. To include groups from Red Hat Enterprise Linux Identity Management and in the future from Active Directory, the "id" command is run in a subprocess. * Reference Material Red Hat Storage Administration Guide: https://access.redhat.com/knowledge/docs/Red_Hat_Storage/ Swift Documentation: http://docs.openstack.org/developer/swift/