diff options
author | Jeff Darcy <jdarcy@redhat.com> | 2014-04-22 15:37:09 +0000 |
---|---|---|
committer | Jeff Darcy <jdarcy@redhat.com> | 2014-04-22 15:37:09 +0000 |
commit | a827c5eab32a43ade5551259ea56a6a1af7e861b (patch) | |
tree | e6707df68f72baa8645210ba931272285116ad85 /doc/network_compression.md | |
parent | 46d333783a968ab39e0beade9c7a1eec8035f8b1 (diff) | |
parent | 99bfc2a2a1689da1e173cb2f8ef54d2b09ef3a5d (diff) |
Merge branch 'upstream'
Conflicts:
glusterfs.spec.in
xlators/mgmt/glusterd/src/Makefile.am
xlators/mgmt/glusterd/src/glusterd-utils.c
xlators/mgmt/glusterd/src/glusterd.h
Change-Id: I27bdcf42b003cfc42d6ad981bd2bf8180176806d
Diffstat (limited to 'doc/network_compression.md')
-rw-r--r-- | doc/network_compression.md | 71 |
1 files changed, 71 insertions, 0 deletions
diff --git a/doc/network_compression.md b/doc/network_compression.md new file mode 100644 index 000000000..7327591ef --- /dev/null +++ b/doc/network_compression.md @@ -0,0 +1,71 @@ +#On-Wire Compression + Decompression + +The 'compression translator' compresses and decompresses data in-flight +between client and bricks. + +###Working +When a writev call occurs, the client compresses the data before sending it to +brick. On the brick, compressed data is decompressed. Similarly, when a readv +call occurs, the brick compresses the data before sending it to client. On the +client, the compressed data is decompressed. Thus, the amount of data sent over +the wire is minimized. Compression/Decompression is done using Zlib library. + +During normal operation, this is the format of data sent over wire: + +~~~ +<compressed-data> + trailer(8 bytes) +~~~ + +The trailer contains the CRC32 checksum and length of original uncompressed +data. This is used for validation. + +###Usage + +Turning on compression xlator: + +~~~ +gluster volume set <vol_name> network.compression on +~~~ + +###Configurable parameters (optional) + +**Compression level** +~~~ +gluster volume set <vol_name> network.compression.compression-level 8 +~~~ + +~~~ +0 : no compression +1 : best speed +9 : best compression +-1 : default compression +~~~ + +**Minimum file size** + +~~~ +gluster volume set <vol_name> network.compression.min-size 50 +~~~ + +Data is compressed only when its size exceeds the above value in bytes. + +**Other paramaters** + +Other less frequently used parameters include `network.compression.mem-level` +and `network.compression.window-size`. More details can about these options +can be found by running `gluster volume set help` command. + +###Known Issues and Limitations + +* Compression translator cannot work with striped volumes. +* Mount point hangs when writing a file with write-behind xlator turned on. To +overcome this, turn off `performance.write-behind` entirely OR +set`performance.strict-write-ordering` to on. +* For glusterfs versions <= 3.5, compression traslator can ONLY work with pure +distribute volumes. This limitation is caused by AFR not being able to +propagate xdata. This issue has been fixed in glusterfs versions > 3.5 + +###TODO +Although zlib offers high compression ratio, it is very slow. We can make the +translator pluggable to add support for other compression methods such as +[lz4 compression](https://code.google.com/p/lz4/) |