diff options
author | raghavendra talur <raghavendra.talur@gmail.com> | 2015-08-20 15:09:31 +0530 |
---|---|---|
committer | Humble Devassy Chirammal <humble.devassy@gmail.com> | 2015-08-31 02:27:22 -0700 |
commit | 9e9e3c5620882d2f769694996ff4d7e0cf36cc2b (patch) | |
tree | 3a00cbd0cc24eb7df3de9b2eeeb8d42ee9175f88 /Feature Planning/GlusterFS 4.0/Compression Dedup.md | |
parent | f6055cdb4dedde576ed8ec55a13814a69dceefdc (diff) |
Create basic directory structure
All new features specs go into in_progress directory.
Once signed off, it should be moved to done directory.
For now,
This change moves all the Gluster 4.0 feature specs to
in_progress. All other specs are under done/release-version.
More cleanup required will be done incrementally.
Change-Id: Id272d301ba8c434cbf7a9a966ceba05fe63b230d
BUG: 1206539
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/11969
Reviewed-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Reviewed-by: Prashanth Pai <ppai@redhat.com>
Tested-by: Humble Devassy Chirammal <humble.devassy@gmail.com>
Diffstat (limited to 'Feature Planning/GlusterFS 4.0/Compression Dedup.md')
-rw-r--r-- | Feature Planning/GlusterFS 4.0/Compression Dedup.md | 128 |
1 files changed, 0 insertions, 128 deletions
diff --git a/Feature Planning/GlusterFS 4.0/Compression Dedup.md b/Feature Planning/GlusterFS 4.0/Compression Dedup.md deleted file mode 100644 index 7829018..0000000 --- a/Feature Planning/GlusterFS 4.0/Compression Dedup.md +++ /dev/null @@ -1,128 +0,0 @@ -Feature -------- - -Compression / Deduplication - -Summary -------- - -In the never-ending quest to increase storage efficiency (or conversely -to decrease storage cost), we could compress and/or deduplicate data -stored on bricks. - -Owners ------- - -Jeff Darcy <jdarcy@redhat.com> - -Current status --------------- - -Just a vague idea so far. - -Related Feature Requests and Bugs ---------------------------------- - -TBD - -Detailed Description --------------------- - -Compression and deduplication for GlusterFS have been discussed many -times. Deduplication across machines/bricks is a recognized Hard -Problem, with uncertain benefits, and is thus considered out of scope. -Deduplication within a brick is potentially achievable by using -something like -[lessfs](http://sourceforge.net/projects/lessfs/files/ "wikilink"), -which is itself a FUSE filesystem, so one fairly simple approach would -be to integrate lessfs as a translator. There's no similar option for -compression. - -In both cases, it's generally preferable to work on fully expanded files -while they're open, and then compress/dedup when they're closed. Some of -the bitrot or tiering infrastructure might be useful for moving files -between these states, or detecting when such a change is needed. There -are also some interesting interactions with quota, since we need to -count the un-compressed un-deduplicated size of the file against quota -(or do we?) and that's not what the underlying local file system will -report. - -Benefit to GlusterFS --------------------- - -Less \$\$\$/GB for our users. - -Scope ------ - -### Nature of proposed change - -New translators, hooks into bitrot/tiering/quota, probably new daemons. - -### Implications on manageability - -Besides turning these options on or off, or setting parameters, there -will probably need to be some way of reporting the real vs. -compressed/deduplicated size of files/bricks/volumes. - -### Implications on presentation layer - -Should be none. - -### Implications on persistence layer - -If the DM folks ever get their <expletive deleted> together on this -front, we might be able to use some of their stuff instead of lessfs. -That worked so well for thin provisioning and snapshots. - -### Implications on 'GlusterFS' backend - -What's on the brick will no longer match the data that the user stored -(and might some day retrieve). In the case of compression, -reconstituting the user-visible version of the data should be a simple -matter of decompressing via a well known algorithm. In the case of -deduplication, the relevant data structures are much more complicated -and reconstitution will be correspondingly more difficult. - -### Modification to GlusterFS metadata - -Some of the information tracking deduplicated blocks will probably be -stored "privately" in .glusterfs or similar. - -### Implications on 'glusterd' - -TBD - -How To Test ------------ - -TBD - -User Experience ---------------- - -Mostly unchanged, except for performance. As with erasure coding, a -compressed/deduplicated slow tier will usually need to be paired with a -simpler fast tier for overall performance to be acceptable. - -Dependencies ------------- - -External: lessfs, DM, whatever other technology we use to do the -low-level work - -Internal: tiering/bitrot (perhaps changelog?) to track state and detect -changes - -Documentation -------------- - -TBD - -Status ------- - -Still just a vague idea. - -Comments and Discussion ------------------------ |