glusterfs.git/xlators/features/changelog/src, branch v3.8.4

changelog/rpc: Fix rpc_clnt_t mem leaks

2016-07-25T06:40:30+00:00

Backport of: http://review.gluster.org/13658

PROBLEM:
   1. Freeing up rpc_clnt object might lead to crashes. Well,
      it was not a necessity to free rpc-clnt object till now
      because all the existing use cases needs to reconnect
      back on disconnects. Hence timer code was not taking
      ref on rpc-clnt object.

      Glusterd had some use-cases that led to crash due to
      ping-timer and they fixed only those code paths that
      involve ping-timer.

      Now, since changelog has an use-case where rpc-clnt
      need to be freed up, we need to fix timer code to take
      refs

   2. In changelog, because of issue 1, only mydata was being
      freed which is incorrect. And there are races where
      rpc-clnt object would access the freed mydata which
      would lead to crashes.

      Since changelog xlator resides on brick side and is long
      living process, if multiple libgfchangelog consumers
      register to changelog and disconnect/reconnect mulitple
      times, it would result in leak of 'rpc-clnt' object
      for every connect/disconnect.

SOLUTION:
   1. Handle ref/unref of 'rpc_clnt' structure in timer
      functions properly.
   2. In changelog, unref 'rpc_clnt' in RPC_CLNT_DISCONNECT
      after disabling timers and free mydata on RPC_CLNT_DESTROY.

RPC SETUP IN CHANGELOG:
   1. changelog xlator initiates rpc server say 'changelog_rpc_server'
   2. libgfchangelog initiates one rpc server say 'libgfchangelog_rpc_server'
   3. libgfchangelog initiates rpc client and connects to 'changelog_rpc_server'
   4. In return changelog_rpc_server initiates a rpc client and connects back
      to 'libgfchangelog_rpc_server'

REF/UNREF HANDLING IN TIMER FUNCTIONS:
Let's say rpc clnt refcount = 1
   1. Take the ref before reigstering callback to timer queue
           >>>>  rpc_clnt_ref (say ref count becomes = 2)
   2. Register a callback to timer say 'callback1'
   3. If register fails:
           >>>> rpc_clnt_unref (ref count = 1)
   4. On timer expiration, 'callback1' gets called. So unref rpc clnt at the end
      in 'callback1'. This is corresponding to ref taken in step 1
           >>>> rpc_clnt_unref (ref count = 1)
   5. The cycle from step-1 to step-4 continues....until timer cancel event happens
   6. timer cancel of say 'callback1'
           If timer cancel fails:
                 Do nothing, Step-4 would have unrefd
           If timer cancel succeeds:
                 >>>> rpc_clnt_unref (ref count = 1)

Change-Id: I91389bc511b8b1a17824941970ee8d2c29a74a09
BUG: 1359364
Signed-off-by: Kotresh HR 
(cherry picked from commit 637ce9e2e27e9f598a4a6c5a04cd339efaa62076)
Reviewed-on: http://review.gluster.org/14994
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Raghavendra G

features/changelog: Change barrier notification mechanism

2016-06-13T11:22:51+00:00

Backport of http://review.gluster.org/#/c/14272/

The barrier notification mechanism was fd based and 'select'
was being used. 'select' breaks when number of fds opened
by brick process exceeds 1024. To avoid this and also
the maintainance of pipe between notify and 'changelog_rollover',
the pipe has been replaced with pthread condition signal
and timed wait mechanism.

> Change-Id: I530ea90d9a06953f8b23b4e12d122872ee1925de
> BUG: 1334314
> Signed-off-by: Kotresh HR 
> Reviewed-on: http://review.gluster.org/14272
(cherry picked from commit be00012e7be55d25870411f3e975db9a8e19c70a)

Change-Id: I530ea90d9a06953f8b23b4e12d122872ee1925de
BUG: 1342083
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/14614
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos 
Smoke: Gluster Build System

features/changelog: Don't modify 'pargfid' in 'resolve_pargfid_to_path'

2016-03-31T04:23:19+00:00

If 'changelog' is enabled and 'changelog.capture-del-path' option is on
it calls 'resolve_pargfid_to_path' which modifies 'pargfid' sent by
caller.  'changelog_unlink' calls this routine directly with
'loc->pargfid' resulting it being modified and point to root instead of
actual pargfid. This is a nasty bug and could cause the deletion of
entry on root directory instead on actual parent when 'loc->path' is
not present.  Hence this fix to make 'pargfid' a const pointer and
'resolve_pargfid' to work on copy of pargfid.

Glusterfind session creation enables these options by default to
capture deleted entry path in changelog.

Thanks Pranith for root causing this.

Change-Id: I1d275a86f91c981b6889bedef93401c039d01d71
BUG: 1321955
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/13845
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Pranith Kumar Karampuri 
Reviewed-by: Venky Shankar 
Reviewed-by: Ravishankar N 
CentOS-regression: Gluster Build System

changelog: fix typecasting of function

2016-03-17T15:22:13+00:00

fix compiler error:
"error: void value not ignored as it ought to be"

while the code tries to typecast return type of
'LOCK_DESTROY(lock)' to (void)

Change-Id: I38ed47f3a12719c7a4a59388c0d021858d4dfe6d
BUG: 1312354
Signed-off-by: Prasanna Kumar Kalever 
Reviewed-on: http://review.gluster.org/13533
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Jeff Darcy

build: export minimum symbols from xlators for correct resolution

2015-12-22T17:15:01+00:00

Revisiting http://review.gluster.org/#/c/11814/, which unintentionally
introduced warnings from libtool about the xlator .so names.

According to [1], the -module option must appear in the Makefile.am
file(s); if -module is defined in a macro, e.g. in configure(.ac),
then libtool will not recognize that this is a module and will emit a
warning.

[1]
http://www.gnu.org/software/automake/manual/automake.html#Libtool-Modules

Change-Id: Ifa5f9327d18d139597791c305aa10cc4410fb078
BUG: 1248669
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/13003
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: soumya k 
Reviewed-by: Niels de Vos

features/changelog: Remove incorrect error message

2015-12-15T03:41:27+00:00

During changelog_rollover, the rename error is
logged even though changelog rollover is successful.
This got mistakenly added as part of commit
64e6836ac8fb96f38a859eea0d8525cd9bd2693f
(http://review.gluster.org/10532). Removing the
same.

Change-Id: I7ac1a16b41de5a6a4d7e714946585dff3ce0e83b
BUG: 1290421
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12940
Reviewed-by: Aravinda VK 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar 
Tested-by: NetBSD Build System

features/changelog: Capture FXATTROP and XATTROP in changelog

2015-11-23T06:49:15+00:00

GEO-REP INTEROP WITH SHARD FEATURE

shard xlator updates size of the file using FXATTROP
or XATTROP. Hence record the same in changelog.

Change-Id: Ie0c21e9326da05ea78dc1ef3fd32a90ef38b4bb9
BUG: 1265148
Signed-off-by: Kotresh HR 
Reviewed-on: http://review.gluster.org/12225
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar

cluster/ec: Mark self-heal fops as internal

2015-11-18T13:16:03+00:00

Change-Id: I8ae7af266d3e00460f0cfdc9389a926e5f2fee36
BUG: 1282761
Signed-off-by: Pranith Kumar K 
Reviewed-on: http://review.gluster.org/12598
Tested-by: Gluster Build System 
Tested-by: NetBSD Build System 
Reviewed-by: Xavier Hernandez

core: use syscall wrappers instead of direct syscalls

2015-10-28T23:34:45+00:00

various xlators and other components are invoking system calls
directly instead of using the libglusterfs/syscall.[ch] wrappers.

If not using the system call wrappers there should be a comment
in the source explaining why the wrapper isn't used.

Change-Id: I8ef94c48728666465abf126c778b70c9e5c00e47
BUG: 1267967
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/12273
Tested-by: Gluster Build System 
Reviewed-by: Jeff Darcy

features/changelog: record mknod if tier-dht linkto is set

2015-10-27T17:14:28+00:00

This is a series of patches which aims to fix geo-replication
in a Tiering Volume.

Problem:
Consider, a file is placed in volume initially and then hot tier is
attached. During any operation on the file, due to lookup a linkto
file is created in hot tier.

Now, any namespace operation carried out on the file is recorded in
both cold and hot tier.
There is a room for races when both changelogs are replayed.

Solution:
So, We are going to replay (namespace related)operations
only in the hot tier.

Why?
a. If the file is directly placed in Hot tier, all fops will be
recorded in HOT tier.

b. If  the file is already present in Cold tier, and if any fop is
carried out, it creates linkto file in Hot tier.
Now, operations like UNLINK, RENAME are captured in Hot tier(by means of linkto file).

This way, we can get both tier's operation in HOT tier itself.

But, We may miss initial Data sync immediately after creating the
file as it is only recording MKNOD. So, if MKNOD encountered
with sticky bit set, queue DATA operation for the corresponding gfid.
( This geo-rep related changes are addressed in this patch: http://review.gluster.org/12326/ )

So, If tier-dht linkto is set, we need to record the corresponding
MKNOD. Earlier this was avoided as it was set as INTERNAL fop.
(This is addressed here in this patch)

Change-Id: I25514fe3e25f68592a8d6361507f8c8a4fcb70b1
BUG: 1266875
Signed-off-by: Saravanakumar Arumugam 
Reviewed-on: http://review.gluster.org/12417
Reviewed-by: Aravinda VK 
Reviewed-by: Kotresh HR 
Tested-by: Gluster Build System 
Reviewed-by: Venky Shankar