diff options
author | Raghavendra Talur <rtalur@redhat.com> | 2015-01-13 20:25:34 +0530 |
---|---|---|
committer | Humble Devassy Chirammal <humble.devassy@gmail.com> | 2015-01-28 23:11:17 -0800 |
commit | 32c3d4e7a611f64a1f4eebe5711bcb112b94151f (patch) | |
tree | 23cf65b9f078d5f127a9524de21f2da5f47ef304 | |
parent | abc043312db6307aabf95f593bfa405ac28a2c8a (diff) |
doc: Add doc for rdma transport.
Also corrected few things related to rdma documentation
in other docs.
Change-Id: If35d7e2e884974ff3719f6a9f111f7b45ac8c0ec
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/9443
Tested-by: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Reviewed-by: Ben England <bengland@redhat.com>
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
-rw-r--r-- | doc/admin-guide/en-US/markdown/admin_rdma_transport.md | 70 | ||||
-rw-r--r-- | doc/admin-guide/en-US/markdown/admin_setting_volumes.md | 6 |
2 files changed, 73 insertions, 3 deletions
diff --git a/doc/admin-guide/en-US/markdown/admin_rdma_transport.md b/doc/admin-guide/en-US/markdown/admin_rdma_transport.md new file mode 100644 index 00000000000..872adb31a08 --- /dev/null +++ b/doc/admin-guide/en-US/markdown/admin_rdma_transport.md @@ -0,0 +1,70 @@ +# Introduction + +GlusterFS supports using RDMA protocol for communication between glusterfs clients and glusterfs bricks. +GlusterFS clients include FUSE client, libgfapi clients(Samba and NFS-Ganesha included), gNFS server and other glusterfs processes that communicate with bricks like self-heal daemon, quotad, rebalance process etc. + +NOTE: As of now only FUSE client and gNFS server would support RDMA transport. + + +NOTE: +NFS client to gNFS Server/NFS Ganesha Server communication would still happen over tcp. +CIFS Clients/Windows Clients to Samba Server communication would still happen over tcp. + +# Setup +Please refer to these external documentation to setup RDMA on your machines +http://pkg-ofed.alioth.debian.org/howto/infiniband-howto.html +http://people.redhat.com/dledford/infiniband_get_started.html + +## Creating Trusted Storage Pool +All the servers in the Trusted Storage Pool must have RDMA devices if either RDMA or TCP,RDMA volumes are created in the storage pool. +The peer probe must be performed using IP/hostname assigned to the RDMA device. + +## Ports and Firewall +Process glusterd will listen on both tcp and rdma if rdma device is found. Port used for rdma is 24008. Similarly, brick processes will also listen on two ports for a volume created with transport "tcp,rdma". + +Make sure you update the firewall to accept packets on these ports. + +# Gluster Volume Create + +A volume can support one or more transport types for communication between clients and brick processes. There are three types of supported transport, which are, tcp, rdma, and tcp,rdma. + +Example: To create a distributed volume with four storage servers over InfiniBand: + +`# gluster volume create test-volume transport rdma server1:/exp1 server2:/exp2 server3:/exp3 server4:/exp4` +Creation of test-volume has been successful +Please start the volume to access data. + +# Changing Transport of Volume +To change the supported transport types of a existing volume, follow the procedure: +NOTE: This is possible only if the volume was created with IP/hostname assigned to RDMA device. + + 1. Unmount the volume on all the clients using the following command: +`# umount mount-point` + 2. Stop the volumes using the following command: +`# gluster volume stop volname` + 3. Change the transport type. +For example, to enable both tcp and rdma execute the followimg command: +`# gluster volume set volname config.transport tcp,rdma` + 4. Mount the volume on all the clients. +For example, to mount using rdma transport, use the following command: +`# mount -t glusterfs -o transport=rdma server1:/test-volume /mnt/glusterfs` + +NOTE: +config.transport option does not have a entry in help of gluster cli. +`#gluster vol set help | grep config.transport` +However, the key is a valid one. + +# Mounting a Volume using RDMA + +You can use the mount option "transport" to specify the transport type that FUSE client must use to communicate with bricks. If the volume was created with only one transport type, then that becomes the default when no value is specified. In case of tcp,rdma volume, tcp is the default. + +For example, to mount using rdma transport, use the following command: +`# mount -t glusterfs -o transport=rdma server1:/test-volume /mnt/glusterfs` + +# Transport used by auxillary processes +All the auxillary processes like self-heal daemon, rebalance process etc use the default transport.In case you have a tcp,rdma volume it will use tcp. +In case of rdma volume, rdma will be used. +Configuration options to select transport used by these processes when volume is tcp,rdma are not yet available and will be coming in later releases. + + + diff --git a/doc/admin-guide/en-US/markdown/admin_setting_volumes.md b/doc/admin-guide/en-US/markdown/admin_setting_volumes.md index 395aa2c79a9..f4eebcd50fb 100644 --- a/doc/admin-guide/en-US/markdown/admin_setting_volumes.md +++ b/doc/admin-guide/en-US/markdown/admin_setting_volumes.md @@ -69,7 +69,7 @@ start it before attempting to mount it. - Create a new volume : - `# gluster volume create [stripe | replica | disperse] [transport tcp | rdma | tcp, rdma] ` + `# gluster volume create [stripe | replica | disperse] [transport tcp | rdma | tcp,rdma] ` For example, to create a volume called test-volume consisting of server3:/exp3 and server4:/exp4: @@ -155,7 +155,7 @@ high-availability and high-reliability are critical. 2. Create the replicated volume: - `# gluster volume create [replica ] [transport tcp | rdma tcp,rdma] ` + `# gluster volume create [replica ] [transport tcp | rdma | tcp,rdma] ` For example, to create a replicated volume with two storage servers: @@ -531,7 +531,7 @@ volumes, but using dispersed subvolumes instead of replicated ones. 2. Create the distributed dispersed volume: - `# gluster volume create disperse <count> [redundancy <count>] [transport tcp | rdma tcp,rdma]` + `# gluster volume create disperse <count> [redundancy <count>] [transport tcp | rdma | tcp,rdma]` To create a distributed dispersed volume, the *disperse* keyword and <count> is mandatory, and the number of bricks specified in the |