From c29a9d63b3f36866f1c138fe9b5dd5da48ec720f Mon Sep 17 00:00:00 2001 From: Prashanth Pai Date: Tue, 15 Apr 2014 10:26:31 +0530 Subject: doc: Add doc for network compression xlator Change-Id: Iaed94f1bfe7c97eade95923759980a1d10805663 BUG: 1086752 Signed-off-by: Prashanth Pai Reviewed-on: http://review.gluster.org/7479 Reviewed-by: Humble Devassy Chirammal Reviewed-by: Niels de Vos Tested-by: Gluster Build System --- doc/network_compression.md | 71 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 doc/network_compression.md (limited to 'doc/network_compression.md') diff --git a/doc/network_compression.md b/doc/network_compression.md new file mode 100644 index 000000000..7327591ef --- /dev/null +++ b/doc/network_compression.md @@ -0,0 +1,71 @@ +#On-Wire Compression + Decompression + +The 'compression translator' compresses and decompresses data in-flight +between client and bricks. + +###Working +When a writev call occurs, the client compresses the data before sending it to +brick. On the brick, compressed data is decompressed. Similarly, when a readv +call occurs, the brick compresses the data before sending it to client. On the +client, the compressed data is decompressed. Thus, the amount of data sent over +the wire is minimized. Compression/Decompression is done using Zlib library. + +During normal operation, this is the format of data sent over wire: + +~~~ + + trailer(8 bytes) +~~~ + +The trailer contains the CRC32 checksum and length of original uncompressed +data. This is used for validation. + +###Usage + +Turning on compression xlator: + +~~~ +gluster volume set network.compression on +~~~ + +###Configurable parameters (optional) + +**Compression level** +~~~ +gluster volume set network.compression.compression-level 8 +~~~ + +~~~ +0 : no compression +1 : best speed +9 : best compression +-1 : default compression +~~~ + +**Minimum file size** + +~~~ +gluster volume set network.compression.min-size 50 +~~~ + +Data is compressed only when its size exceeds the above value in bytes. + +**Other paramaters** + +Other less frequently used parameters include `network.compression.mem-level` +and `network.compression.window-size`. More details can about these options +can be found by running `gluster volume set help` command. + +###Known Issues and Limitations + +* Compression translator cannot work with striped volumes. +* Mount point hangs when writing a file with write-behind xlator turned on. To +overcome this, turn off `performance.write-behind` entirely OR +set`performance.strict-write-ordering` to on. +* For glusterfs versions <= 3.5, compression traslator can ONLY work with pure +distribute volumes. This limitation is caused by AFR not being able to +propagate xdata. This issue has been fixed in glusterfs versions > 3.5 + +###TODO +Although zlib offers high compression ratio, it is very slow. We can make the +translator pluggable to add support for other compression methods such as +[lz4 compression](https://code.google.com/p/lz4/) -- cgit