glusterfs.git, branch v3.7.0

doc: add release notes for glusterfs-3.7.0

2015-05-14T09:09:28+00:00

Change-Id: I4da42edc976ebcc5323270c82dfe445ba56f3eee
BUG: 1199352
Signed-off-by: M S Vishwanath Bhat 
Signed-off-by: Vijay Bellur 
Signed-off-by: Niels de Vos 
Signed-off-by: Raghavendra Bhat 
Signed-off-by: Humble Chirammal 
Reviewed-on: http://review.gluster.org/10781

quota/glusterd: on read call number of byte read should equal to buffer length

2015-05-14T07:59:53+00:00

glusterd is crashing when user try to set limit-usage on quota. Because
in the read call number of byte is going to be read is more then buffer
lenght.

Change-Id: Ie507eb68ebc0d0daa1012baef1bf724e202e3baa
BUG: 1221025
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/10767
Reviewed-by: Anand Nekkunti 
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Tested-by: NetBSD Build System
Reviewed-by: Niels de Vos

rpm: fix dependency on python-gluster

2015-05-12T12:13:02+00:00

The glusterfs-extra-xlators tries to install python-glusterfs. This is
incorrect, the package is called python-gluster.

Cherry picked from commit 5388706e8431114418c0335f24532bf9f8df7588:
> BUG: 1211848
> Change-Id: Id48a63ebe893c2bab90fab37ea3e32e601fae0fa
> Reported-by: Meni Yakove 
> Signed-off-by: Niels de Vos 
> Reviewed-on: http://review.gluster.org/10508
> Reviewed-by: Humble Devassy Chirammal 
> Tested-by: NetBSD Build System
> Tested-by: Gluster Build System 

Change-Id: Id48a63ebe893c2bab90fab37ea3e32e601fae0fa
BUG: 1219787
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10754
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Gaurav Kumar Garg

rpm: marker xlator should part of the server sub-package

2015-05-12T12:11:19+00:00

Currently marker translator is part of extra-xlator package. Installing
only the glusterfs-server package and required dependencies does not
make it possible to create a volume and starte it. Starting the volume
will fail. The brick logs compain that type feature/marker is not valid:

    W [xlator.c:192:xlator_dynload] 0-xlator: /usr/lib64/glusterfs/3.7.0beta1/xlator/features/marker.so: cannot open shared object file: No such file or directory
    E [graph.y:212:volume_type] 0-parser: Volume 'rvstore-marker', line 78: type 'features/marker' is not valid or not found on this machine

Fix is to include the marker translator in the server package. It should
not be part of extra-xlator package.

Backport of:
> Change-Id: I77fe8a0bcf7d6f1dbff62d663e840e9e45805614
> BUG: 1220381
> Reviewed-on: http://review.gluster.org/10752
> Signed-off-by: Gaurav Kumar Garg 

Change-Id: I77fe8a0bcf7d6f1dbff62d663e840e9e45805614
BUG: 1220338
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/10753
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

glusterd/tiering : cksum mismatch for tiered volume

2015-05-10T16:43:03+00:00

        Back port of http://review.gluster.org/#/c/10406

Once we updated the volinfo from orginator node, the hot
type was overwritten with volume type.

Then the same dictionary was sent to peer node to perform
the commit of attach-tier, that will cause hot type to
replace with volume type, eventually end up in cksum
mismatch

Change-Id: I402dceb4d672d0b3a7b91a92f52c1057050dbedc
BUG: 1219845
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10677
Reviewed-by: Niels de Vos 
Tested-by: NetBSD Build System
Tested-by: Gluster Build System

uss : implement statfs fop for snapd

2015-05-10T15:51:54+00:00

    Backport of http://review.gluster.org/#/c/10358/

snapview-client and snapview-server doesnot have statfs
fop implemented

Change-Id: I2cdd4c5784414b0549a01af9a28dbc723b7cdc67
BUG: 1218741
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/10740
Tested-by: NetBSD Build System
Reviewed-by: Niels de Vos 
Reviewed-by: Raghavendra Bhat 
Tested-by: Gluster Build System

features/bit-rot-stub: versioning of objects in write/truncate fop instead of open

2015-05-10T15:14:33+00:00

* This patch brings in the changes where object versioning is done in write and
  truncate fops instead of tracking them in open and create fops. This model
  works for both regular and anonymous fds. It also removes the race associated
  with open calls, create and lookups.

  This patch follows the below method for object versioning and notifications:

  Before sending writev on the fd, increase the ongoing
  version first. This makes anonymous fd write similar to the regular
  fd write by having the ongoing version increased before doing the
  write.

  Do following steps to do versioning:
  1) For anonymous fds set the fd context (so that release is invoked) and add
     the fd context to the list maintained in the inode context.
     For regular fds the above think would have been done in open itself.
  2) Increase the on-disk ongoing version
  3) Increase the in memory ongoing version and mark inode as non-dirty
  3) Once versioning is successfully done send write operation. If
     versioning fails, then fail the write fop.
  5) In writev_cbk mark inode as modified.

> Change-Id: I7104391bbe076d8fc49b68745d2ec29a6e92476c
> BUG: 1207979
> Signed-off-by: Raghavendra Bhat 
> Reviewed-on: http://review.gluster.org/10233
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I4bb86989b5fab02b9ed2950798b1a80e566f1024
BUG: 1220041
Signed-off-by: Raghavendra Bhat 
Reviewed-on: http://review.gluster.org/10722
Reviewed-by: Gaurav Kumar Garg 
Tested-by: NetBSD Build System
Tested-by: Gluster Build System

features/bitrot: scrubber should crawl based on the scrubber frequency value

2015-05-10T13:03:46+00:00

Currently scrubber is crawling all the files continuously. It should
crawl files based on the scrubber frequency which user have set.

By default scrubber crawling frequency value will be biweekly.

Change-Id: I5762a92c1e700134cfe4283d1f631904adbfe31d
BUG: 1220068
Signed-off-by: Gaurav Kumar Garg 
Reviewed-on: http://review.gluster.org/10739
Tested-by: NetBSD Build System
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

features/bitrot: Scrubber pause/resume

2015-05-10T12:29:45+00:00

With logical scan/scrub split, pausing filesystem scrubber is an
override to the thread throttling mechanism, which effectively
throttles "down" number of scrubber threads to zero. This causes
scanner to wait until threads are spawned again (when resumed)
thereby continuing where it left off (since the file tree walk
stack is effectively preserved when the main scanner thread
is waiting for scrubbers to consume scanned entries).

The only catch is when scrubber daemon restarts: file tree walk
stack is lost and scrubbing initiates from root. This is probably
OK for now (can be changed later to persist parent directory
information before entering pause state).

> Change-Id: I5109a749b7fccd0f5367765078f46e6522dd32a1
> BUG: 1208131
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10521
> Reviewed-by: Vijay Bellur 
> Tested-by: Vijay Bellur 

Change-Id: I9b60f2ce24ca3787423a45ec7d502f89215fe45f
Signed-off-by: Venky Shankar 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10721
Tested-by: Gluster Build System 
Reviewed-by: Gaurav Kumar Garg

features/bitrot: Throttle filesystem scrubber

2015-05-10T12:29:31+00:00

This patch introduces multithreaded filesystem scrubber based
on throttling option configured for a particular volume. The
implementation "logically" breaks scanning and scrubbing with
the number of scrubber threads auto-configured depending upon
the throttle configuration. Scanning (crawling) is left single
threaded (per brick) with entries scrubbed in bulk. On reaching
this "bulk" watermark, scanner waits until entries are scrubbed.
Bricks for a particular volume have a set of thread(s) assigned
for scrubbing, with entries for each brick scrubbed in a round
robin fashion to avoid scrub "stalls" when a brick (out of N
bricks) is under active scrubbing.

This mechanism helps us implement "pause/resume" with ease: all
one need to do is to cleanup scrubber threads and let the main
scanner thread "wait" untill scrubbing is resumed (where the
scrubber thread(s) are spawned again), therefore continuing
where we left off (unless we restart the deamons, where crawl
initiates from root directory again, but I guess that's OK).

[
    NOTE:

    Throttling is optional for the signer daemon, without which
    it runs full throttle. However, passing "-DBR_RATE_LIMIT_SIGNER"
    predefined in CFLAGS enables CPU throttling (during checksum
    calculation) thereby avoiding high CPU usage.
]

Subsequent patches would introduce CPU throttling during hash
calculation for scrubber.

> Change-Id: I5701dd6cd4dff27ca3144ac5e3798a2216b39d4f
> BUG: 1207020
> Signed-off-by: Venky Shankar 
> Reviewed-on: http://review.gluster.org/10511
> Tested-by: Gluster Build System 
> Reviewed-by: Vijay Bellur 

Change-Id: I5a125b2d0ac7dafd3e278b7fe4c6c9dd07af76dd
Signed-off-by: Venky Shankar 
BUG: 1220041
Reviewed-on: http://review.gluster.org/10720
Tested-by: Gluster Build System 
Reviewed-by: Gaurav Kumar Garg