glusterfs.git/libglusterfs/src, branch v3.8dev

quota/disperse: handle inode quotas in xattr aggregate

2015-04-14T05:01:18+00:00

with the inode quota feature, quota size is now
increased from 64bit to 192bits which contains
values of 'file size', 'file count' and 'dir count'

This change in quota size xattr needs to be handled
in disperse xattr aggregation

Signed-off-by: vmallika 

Change-Id: I5fd28aa9f5b8b6cba83a98360236417a97ac16ee
BUG: 1207967
Reviewed-on: http://review.gluster.org/10112
Reviewed-by: Pranith Kumar Karampuri 
Tested-by: Gluster Build System 
Reviewed-by: Sachin Pandit 
Reviewed-by: Raghavendra G 
Tested-by: Raghavendra G

glusterd: Replace transaction peers lists

2015-04-13T06:30:02+00:00

Transaction peer lists were used in GlusterD to peers belonging to a
transaction. This was needed to prevent newly added peers performing
partial transactions, which could be incorrect.

This was accomplished by creating a seperate transaction peers list at
the beginning of every transaction. A transaction peers list referenced
the peerinfo data structures of the peers which were present at the
beginning of the transaction. RCU protection of peerinfos referenced by
the transaction peers list is a hard problem and difficult to do
correctly.

To have proper RCU protection of peerinfos, the transaction peers lists
have been replaced by an alternative method to identify peers that
belong to a transaction. The alternative method is to the global peers
list along with generation numbers to identify peers that should belong
to a transaction.

This change introduces a global peer list generation number, and a
generation number for each peerinfo object. Whenever a peerinfo object
is created, the global generation number is bumped, and the peerinfos
generation number is set to the bumped global generation.

With the above changes, the algorithm to identify peers belonging to a
transaction with RCU protection is as follows,
- At the beginning of a transaction, the current global generation
  number is saved
- To identify if a peers belonging to the transaction,
  - Start a RCU read critical section
  - For each peer in the global peers list,
    - If the peers generation number is not greater than the saved
      generation number, continue with the action on the peer
  - End the RCU read critical section

The above algorithm guarantees that,
- The peer list is not modified when a transaction is iterating through
  it
- The transaction actions are only done on peers that were present when
  the transaction started

But, as a transaction could iterate over the peers list multiple times,
the algorithm cannot guarantee that same set of peers will be selected
every time. A peer could get deleted between two iterations of the list
within a transaction. This problem existed with transaction peers list
as well, but unlike before now it will not lead to invalid memory access
and potential crashes. This problem will be addressed seprately.

This change was developed on the git branch at [1]. This commit is a
combination of the following commits on the development branch.
  52ded5b Add timespec_cmp
  44aedd8 Add create timestamp to peerinfo
  7bcbea5 Fix some silly mistakes
  13e3241 Add start time to opinfo
  17a6727 Use timestamp comparisions to identify xaction peers instead
          of a xaction peer list
  3be05b6 Correct check for peerinfo age
  70d5b58 Use read-critical sections for peer list iteration
  ba4dbca Use peerinfo timestamp checks in op-sm instead of xaction peer
          list
  d63f811 Add more peer status checks when iterating peers list in
          glusterd-syncop
  1998a2a Timestamp based peer list traversal of mgmtv3 xactions
  f3c1a42 Remove transaction peer lists
  b8b08ee Remove unused labels
  32e5f5b Remove 'npeers' usage
  a075fb7 Remove 'npeers' from mgmt-v3 framework
  12c9df2 Use generation number instead of timestamps.
  9723021 Remove timespec_cmp
  80ae2c6 Remove timespec.h include
  a9479b0 Address review comments on 10147/4

[1]: https://github.com/kshlm/glusterfs/tree/urcu

Change-Id: I9be1033525c0a89276f5b5d83dc2eb061918b97f
BUG: 1205186
Signed-off-by: Kaushal M 
Reviewed-on: http://review.gluster.org/10147
Tested-by: Gluster Build System 
Reviewed-by: Atin Mukherjee 
Reviewed-by: Anand Nekkunti 
Reviewed-by: Krishnan Parthasarathi 
Tested-by: Krishnan Parthasarathi

build: make contrib/uuid dependency optional

2015-04-10T11:39:52+00:00

On Linux systems we should use the libuuid from the distribution and not
bundle and statically link the contrib/uuid/ bits.

libglusterfs/src/compat-uuid.h has been introduced and should become an
abstraction layer for different UUID APIs. Non-Linux operating systems
should implement their compatibility layer there.

Once all operating systems have an implementation in compat-uuid.h, we
can remove contrib/uuid/ from the repository completely.

Change-Id: I345e5357644be2521685e00358bb8c83c4ea0577
BUG: 1206587
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10129
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

quota/cli: validate quota hard-limit option

2015-04-10T11:21:48+00:00

Quota hard-limit is supported only upto: 9223372036854775807 (int 64)

In CLI, it is allowed to set the value upto 16384PB (unsigned int 64),
this is not a valid value as the xattrop for quota accounting and
the quota enforcer operates on a signed int64 limit value.

This patches fixes the problem in CLI and allows user to set
the hard-limit value only from range 0 - 9223372036854775807

Change-Id: Ifce6e509e1832ef21d3278bacfa5bd71040c8cba
BUG: 1206432
Signed-off-by: vmallika 
Reviewed-on: http://review.gluster.org/10022
Tested-by: Gluster Build System 
Reviewed-by: Kaushal M 
Reviewed-by: Vijay Bellur

libglusterfs: avoid possibility of crash when ctx is null

2015-04-08T18:13:36+00:00

ctx is passed to gf_log_inject_timer_event() and pass through to
 __gf_log_inject_timer_event() where the struct members are getting
dereferenced, and can cause crash if the passed ctx is null. This patch
avoids the issue.

Change-Id: I153dbb5d3744898429139e3d40bb4f0e9093632a
BUG: 1208118
Signed-off-by: Humble Devassy Chirammal 
Reviewed-on: http://review.gluster.org/10102
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

libglusterfs/syncop: Add xdata to all syncop calls

2015-04-08T15:14:59+00:00

This patch adds support for xdata in both the
request and response path of syncops.

Few calls like lookup already had the support;
have renamed variables in few places to maintain
uniformity.

xdata passed downwards is known as xdata_in
and xdata passed upwards is known as xdata_out.

There is an old patch by Jeff Darcy at
http://review.gluster.org/#/c/8769/3 which does the
same for some selected calls. It also brings in
xdata support at gfapi level.

xdata support at gfapi level would be introduced
in subsequent patches.

Change-Id: I340e94ebaf2a38e160e65bc30732e8fe1c532dcc
BUG: 1158621
Signed-off-by: Raghavendra Talur 
Reviewed-on: http://review.gluster.org/9859
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

nfs : introducing logging framework for nfs translator

2015-04-08T13:35:07+00:00

Change-Id: I3a47cdd06595c87da8e822d11683d68b43c11cda
BUG: 1194640
Signed-off-by: Manikandan Selvaganesh 
Reviewed-on: http://review.gluster.org/9945
Tested-by: Gluster Build System 
Reviewed-by: jiffin tony Thottan 
Reviewed-by: Niels de Vos

ctr : Fix for heating of files during promotion/demotion

2015-04-08T10:29:58+00:00

This fix will solve the heating of the files during the promotion
or demotion.
Promotion:
~~~~~~~~~
When a file gets promoted it get the current time stamp
 during creation only, but following writes or reads during the
migration wont heat the file.
Demotion:
~~~~~~~~
When a file gets demoted it get the wind/unwind time stamp is set to
zero. The following writes or reads during the migration wont heat
the file.

What is remaining ?
~~~~~~~~~~~~~~~~~
Bug 1209129 ( https://bugzilla.redhat.com/show_bug.cgi?id=1209129 )
Inspite of this fix there is still a issue remaining, i.e the heat of
the file is not keep intact during a internal rebalance activity i.e
a rebalance within a tier.

Change-Id: I01e82dc226355599732d40e699062cee7960b0a5
BUG: 1207867
Signed-off-by: Joseph Fernandes 
Signed-off-by: Dan Lambright 
Signed-off-by: Joseph Fernandes 
Reviewed-on: http://review.gluster.org/10080
Reviewed-by: Shyamsundar Ranganathan 
Tested-by: Gluster Build System

bitrot/scrub: Scrubber fixes

2015-04-08T07:01:32+00:00

This patch fixes a handful of problem with scrubber which
are detailed below.

Scrubber used to skip objects for verification due to missing
fd iterface to fetch versioning extended attributes. Similar
to the inode interface, an fd based interface in POSIX is now
introduced.

Moreover, this patch also fixes potential false reporting by
scrubber due to:

  An object gets dirtied and signed when scrubber is busy
  calculatingobject checksum. This is fixed by caching the
  signed version when an object is first inspected for
  stalenes, i.e., during pre-compute stage. This version is
  used to verify checksum in the post-compute stage when the
  signatures are compared for possible corruption.

  Side effect of _not_ sending signature length during signing
  resulted in "truncated" signature to be set for an object.
  Now, at the time of signing, the signature length is sent
  and is used in place of invoking strlen() to get signature
  length (which could have possible 00s). The signature length
  itself is not persisted in the signature xattr, but is
  calculated on-the-fly by substracting the xattr length by
  the "structure" header size.

Some of the log entries are made more meaningful (as and aid
for debugging).

Change-Id: I938bee5aea6688d5d99eb2640053613af86d6269
BUG: 1207624
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/10118
Reviewed-by: Raghavendra Bhat 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

cluster/dht: fix tier.c problems found prior to feature freeze

2015-04-06T09:35:37+00:00

This patch resolves tiering translator issues taken from the list
in bug 1203776. These issues have been selected to be fixed
first. The rest will be fixed in a subsequent patch (or are not a
problem).

3. Replace hardcoded #defines of promote/demote file names

6. Use loc_wipe() in migrate_using_query_file()

9. Only promote/demote files on the same node on which they reside.

14. Replace calloc with GF_CALLOC in tier.c and ensure freeing done
properly.

15. Handle if parse_query_str fails

22. Only load gfdb library on server side, remove SQL references
    from client.

Change-Id: I6563b11e58ab2e4c6b1ce44db755781ad6d930fb
BUG: 1203776
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/9987
Tested-by: Gluster Build System 
Reviewed-by: N Balachandran 
Reviewed-by: Niels de Vos