glusterfs.git/libglusterfs, branch dht-stale-layout-fixes

build: make GLUSTERD_WORKDIR rely on localstatedir

2014-08-07T08:17:29+00:00

- Break-way from '/var/lib/glusterd' hard-coded previously,
  instead rely on 'configure' value from 'localstatedir'
- Provide 's/lib/db' as default working directory for gluster
  management daemon for BSD and Darwin based installations
- loff_t is really off_t on Darwin
- fix-off the warnings generated by clang on FreeBSD/Darwin
- Now 'tests/*' use GLUSTERD_WORKDIR a common variable for all
  platforms.
- Define proper environment for running tests, define correct PATH
  and LD_LIBRARY_PATH when running tests, so that the desired version
  of glusterfs is used, regardless where it is installed.
  (Thanks to manu@netbsd.org for this additional work)

Change-Id: I2339a0d9275de5939ccad3e52b535598064a35e7
BUG: 1111774
Signed-off-by: Harshavardhana 
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8246
Tested-by: Gluster Build System

libglusterfs: wrong function definition of synclock_destory().

2014-08-06T08:29:28+00:00

synclock_destory() has been prototyped in syncop.h,
how-ever synclock_destroy() is the actual function used in syncop.c.
Correcting this function definition along with few typos.


Change-Id: I35a818190c1d37c303279ca7a820f01895751bd9
BUG: 1075417
Signed-off-by: Humble Chirammal 
Reviewed-on: http://review.gluster.org/8266
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra Talur 
Reviewed-by: soumya k 
Reviewed-by: Poornima G 
Reviewed-by: Vijay Bellur

feature/geo-rep: Keep marker.tstamp's mtime unchangeable during snapshot.

2014-08-04T14:51:07+00:00

Problem:

Geo-replicatoin does a full xsync crawl after snapshot
restoration of slave and master. It does not do history crawl.

Analysis:

Marker creates 'marker.tstamp' file when geo-rep is started
for the first time. The virtual extended attribute
'trusted.glusterfs.volume-mark' is maintained and whenever
it is queried on gluster mount point, marker fills it on
the fly and returns the combination of uuid, ctime of
marker.tstamp and others. So ctime of marker.tstamp, in other
sense 'volume-mark'  marks the geo-rep start time when the
session is freshly created.

From the above, after the first filesystem crawl(xsync) is
done during first geo-rep start, stime should always be less
than 'volume-mark'. So whenever stime is less than volume-mark,
it does full filesystem crawl (xsync).

Root Cause:

When snapshot is restored, marker.tstamp file is freshly
created losing the timestamps, it was originally created with.

Solution:

1. Change is made to depend on mtime instead of ctime.
2. mtime and atime of marker.tstamp is restored back when
   snapshot is created and restored.

Change-Id: I4891b112f4aedc50cfae402832c50c5145807d7a
BUG: 1125918
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8401
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

glusterd: Bump max op version

2014-08-04T05:08:22+00:00

Change-Id: Ic66a0a751233ebbcb65d13f7e3265a046fae9a0b
BUG: 1125431
Signed-off-by: Atin Mukherjee 
Reviewed-on: http://review.gluster.org/8397
Tested-by: Gluster Build System 
Reviewed-by: Humble Devassy Chirammal 
Reviewed-by: Kaushal M

cluster/dht: Modified logic of linkto file deletion on non-hashed

2014-08-01T05:46:34+00:00

Currently whenever dht_lookup_everywhere gets called, if in
dht_lookup_everywhere_cbk, a linkto file is found on non-hashed
subvolume, file is unlinked. But there are cases when this file
is under migration. Under such condition, we should avoid deletion
of file.

When  some other rebalance process changes the layout of parent
such that dst_file (w.r.t. migration) falls on non-hashed node,
then may be lookup could have found it as linkto file but just
before unlink, file  is under migration or already migrated
In such cased unlink can be avoided.

Race:
-------
If we have two bricks (brick-1 and brick-2) with initial file "a"
under BaseDir which is hashed as well as cached on (brick-1).

Assume "a"  hashing gives 44.

                              Brick-1              Brick-2

Initial Setup:               BaseDir/a             BaseDir
                             [1-50]                [51-100]

Now add new-brick Brick-3.

1. Rebalance-1 on node Node-1 (Brick-1 node) will reset
the BaseDir Layout.

2. After that it will perform
a)  Create linkto file on  new-hashed (brick-2)
b)  Perform file migration.

1.Rebalance-1 Fixes the base-layout:
                 Brick-1             Brick-2           Brick-3
                 ---------         ----------         ------------
                 BaseDir/a            BaseDir           BaseDir
                  [1-33]              [34-66]           [67-100]

2. Only a) is     BaseDir/a          BaseDir/a(linkto)   BaseDir
   performed                         Create linktofile

Now rebalance 2 on node-2 jumped in and it will perform
step 1 and 2-a.

After (rebal-2, step-1), it changes the layout of the BaseDir.
                    BaseDir/a     BaseDir/a(link)    BaseDir
                    [67-100]           [1-33]        [34-66]

For  (rebale-2, step-2), It will perform lookup at Brick-3 as w.r.t new
layout 44 falls for brick-3. But lookup will fail.
So  dht_lookup_everywhere gets called.

NOTE: On brick-2 by rebalance-1, a linkto file was created.

Currently that linkto files gets deleted by rebalance-2 lookup as it
is considered as stale linkto file.  But  with patch if rebalance is
already in progress or rebalance is over,  linkto file will not be
unlinked. If rebalance is in progress fd will be  open and if rebalance
is over then linkto file wont be set.

Change-Id: I3fee0d28de3c76197325536a9e30099d2413f079
BUG: 1116150
Signed-off-by: Venkatesh Somyajulu 
Reviewed-on: http://review.gluster.org/8345
Tested-by: Gluster Build System 
Reviewed-by: Raghavendra G 
Reviewed-by: Shyamsundar Ranganathan 
Reviewed-by: Vijay Bellur

Attempt to fix cmockery2 build

2014-07-27T18:57:24+00:00

The current code assumes cmockery2 is installed in default paths.
Use PKG_MODULES_CHECK to find it using pkg-config if it is not. If
not found by pkg-config, try AC_CHECK_LIB.

There are also some build flag adjustement so that local overrides
do not loose the required -I flags.

This includes and enhance http://review.gluster.org/8340/

BUG: 764655
Change-Id: Ide9f77d1e70afe3c1c5c57ae2b93127af6a425f9
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/8365
Reviewed-by: Harshavardhana 
Tested-by: Harshavardhana 
Tested-by: Gluster Build System

features/changelog: Capture "correct" internal FOPs

2014-07-23T23:15:21+00:00

This patch fixes changelog capturing internal FOPs in a cascaded
setup, where the intermediate master would record internal FOPs
(generated by DHT on link()/rename()). This is due to I/O happening
on the intermediate slave on geo-replication's auxillary mount with
client-pid -1. Currently, the internal FOP capturing logic depends
on client pid being non-negative and the presence of a special key
in dictionary. Due to this, internal FOPs on an inter-mediate master
would be recorded in the changelog. Checking client-pid being
non-negative was introduced to capture AFR self-heal traffic in
changelog, thereby breaking cascading setups. By coincidence,
AFR self-heal daemon uses -1 as frame->root->pid thereby making
is hard to differentiate b/w geo-rep's auxillary mount and self-heal
daemon.

Change-Id: Ib7bd71e80dd1856770391edb621ba9819cab7056
BUG: 1122037
Original-Author: Venky Shankar 
Signed-off-by: Kotresh H R 
Reviewed-on: http://review.gluster.org/8347
Reviewed-by: Venky Shankar 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Bump op-version for 3.7.0

2014-07-23T15:13:44+00:00

Change-Id: I4542edeca140bc2252d765b5cfc2e24d1d90cdb1
BUG: 1122398
Reviewed-on: http://review.gluster.org/8354
Reviewed-by: Atin Mukherjee 
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M

Use C-locale for numeric/string conversion routines

2014-07-19T21:15:48+00:00

(strtod, ...) or config file parsing might fail.


Change-Id: I649f29bbf87222399a0c2d1ed5a3bf136c613b9b
BUG: 1117951
Signed-off-by: Anders Blomdell 
Reviewed-on: http://review.gluster.org/8299
Tested-by: Gluster Build System 
Reviewed-by: Prashanth Pai 
Reviewed-by: Vijay Bellur

porting: extattr_remove_fd() doesn't exist

2014-07-18T22:58:55+00:00

Fix typo extattr_remove_fd() --> extattr_delete_fd()

Change-Id: I0dc3399722eb272d3c04836726e22b212ef15eba
BUG: 1111774
Signed-off-by: Harshavardhana 
Reviewed-on: http://review.gluster.org/8319
Tested-by: Gluster Build System