glusterfs.git/xlators/cluster/dht/src/tier.c, branch release-3.7

cluster/tier: downgrade max-cycle-time log message to INFO

2016-05-21T15:48:40+00:00

The "max cycle time" log message was incorrectly logged as
an error. Downgrade it to INFO.

This is a backport of 14336

> Change-Id: Ia7d074423019fa79443bc6ea694148b7b8da455d
> BUG: 1335973
> Signed-off-by: Dan Lambright 

Change-Id: I29514c66781f49d5c36a0d3ad5dee6ab0c0368cd
BUG: 1336470
Signed-off-by: Dan Lambright 
Reviewed-on: http://review.gluster.org/14361
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: N Balachandran

Tier: Avoiding stale entries from causing demotion to stop

2016-03-22T09:06:48+00:00

    back-port of : http://review.gluster.org/#/c/13501/

When the parent GFID is a stale entry, the lookup on this parent
fails and this in turn fails the demotion process.

This patch will make the stale entry error to be skipped.

Situation for pargfid to be stale:
Consider a folder from a tar file. Once the tar file is untared
the files in the tar-file will start to demote.
when the demotion is under progress, if we tend to delete
the actual folder, then the files under it which are
undergoing demotion will do a lookup on the parent which was
deleted and become stale entry. This stale entry fails the
Lookup and this will fail the demotion of the other files(not from
tar) that are supposed to be demoted.

>Change-Id: I3d47c32c4077526d477a25912b0135bab98b23fc
>BUG: 1311178
>Signed-off-by: hari gowtham 
>Reviewed-on: http://review.gluster.org/13501
>Tested-by: hari gowtham 
>Smoke: Gluster Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Dan Lambright 
>NetBSD-regression: NetBSD Build System 

Change-Id: I80a77d149180e2a05b3d7943f370b3ef162e545b
BUG: 1317366
Signed-off-by: Hari Gowtham 
Reviewed-on: http://review.gluster.org/13692
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

TIER: stopping the tierd when the volume goes down

2016-03-22T09:05:14+00:00

        back-port of : http://review.gluster.org/#/c/13646/

If there are large number of files to be migrated
and by this time if the volume goes down, then the tierd
has to be stopped. But on a huge query file list it keeps
checking for each file before stopping. If the volume comes
up before the old tierd dies then due to the
presence of old tierd new one won't be created. After
the old one completes the task, it dies and the status
ends up as failed.

This patch will check if the status is still running and then let
it continue its work. Else it will stop running the tierd.

>Change-Id: I6522a4e2919e84bf502b99b13873795b9274f3cd
>BUG: 1315659
>Signed-off-by: hari gowtham 
>Reviewed-on: http://review.gluster.org/13646
>Tested-by: Dan Lambright 
>Smoke: Gluster Build System 
>NetBSD-regression: NetBSD Build System 
>CentOS-regression: Gluster Build System 
>Reviewed-by: Dan Lambright 

Change-Id: I8326dbe5edaaea921e5401f39d148aac322c78d0
BUG: 1318498
Signed-off-by: hari 
Reviewed-on: http://review.gluster.org/13756
Smoke: Gluster Build System 
Tested-by: hari gowtham 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier: break out of iterating query file once cycle time ends

2016-03-17T04:37:48+00:00

When iterating the query file during migration, tiering should
break out of the loop once cycle time completes. Otherwise it
may be possible to stay in the loop for a long time. If that
happens updates to files will become stale and have not impact
migration.

This is a backport of 13284

> Change-Id: Ib60cf74bc84e8646e6a0da21ff04954b1b83c414
> BUG: 1301227
> Signed-off-by: Dan Lambright 
> Reviewed-on: http://review.gluster.org/13284
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
Signed-off-by: Dan Lambright 

Change-Id: Ia305784b0aa570b70e54f532bf5d68a91b0cf305
BUG: 1318203
Reviewed-on: http://review.gluster.org/13750
Smoke: Gluster Build System 
Reviewed-by: hari gowtham 
Tested-by: Dan Lambright 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier: Additional details in error messages

2016-03-07T22:15:53+00:00

Added file path/gfid when available to the tier log
messages to make debugging easier.

> Change-Id: I22dda329367df2b846dcf254594312c997b66083
> BUG: 1273043
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/13114
> Reviewed-by: mohammed rafi  kc 
> Tested-by: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 

Change-Id: I5cbf8407fb7c493c4be1dda65c0f722ede33487f
BUG: 1311865
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/13518
Smoke: Gluster Build System 
Reviewed-by: mohammed rafi  kc 
Reviewed-by: Dan Lambright 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Tested-by: Dan Lambright

cluster/tier: Create linkfiles to hardlinks correctly

2016-02-27T18:48:50+00:00

There is a bug in the way hardlinks are handled in tiered volumes.
Ideally, the tier linkto files on the cold tier to files that are
hardlinks to each other on the hot tier, should themselves be
hardlinks of each other. As they are not, they end up being files
with the same gfid but different names for the cold tier dht,
and end up overwriting the cached-subvol information stored in the dht inode-ctx.

> Change-Id: Ic658a316836e6a1729cfea848b7d212674b0edd2
> BUG: 1305277
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/13391
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
> Tested-by: Dan Lambright 
(cherry picked from commit eb362c74db84d95aac07febf0d888bd98b3fb2b9)

Change-Id: I5dd98e6d248619147974a630d7d72e1942a1cc83
BUG: 1311836
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/13517
Smoke: Gluster Build System 
Reviewed-by: mohammed rafi  kc 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier : Fixed wrong variable comparison

2016-02-27T18:48:12+00:00

The wrong variable was being checked to determine
the watermark value.

> Change-Id: If4c97fa70b772187f1fcbdf5193e077cb356a8b1
> BUG: 1303895
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/13357
> Smoke: Gluster Build System 
> Reviewed-by: Dan Lambright 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
Signed-off-by: N Balachandran 

Change-Id: I0a98e0efbc093a727912107038477239e6d85765
BUG: 1306129
Reviewed-on: http://review.gluster.org/13516
Smoke: Gluster Build System 
Reviewed-by: mohammed rafi  kc 
Tested-by: mohammed rafi  kc 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

tier:delete the linkfile if data file creation fails

2016-02-22T14:53:07+00:00

If we are creating data file in a hot subvolume
then we will create a linkfile in cold subvolume.
Linkfile creation happens first. If linkfile creation
was successful and data file creation failed, then
linkfile in cold subvolume will become stale

This patch will delete the linkfile as well, if data
file creation fails

Also this code duplicates dht_create to make tier_create

backport of>
>Change-Id: I377a90dad47f288e9576c7323b23cf694a91a7a3
>BUG: 1290677
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12948
>Reviewed-by: N Balachandran 
>Tested-by: NetBSD Build System 
>Reviewed-by: Raghavendra G 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

Change-Id: I3689e8ee387fb6731b4b3a7fe8f8190143482eaa
BUG: 1295359
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/13161
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

tier:unlink during migration

2016-02-22T14:50:54+00:00

files deleted during promotion were not deleting as the
files are moving from hashed to non-hashed

On deleting a file that is undergoing promotion,
the unlink call is not sent to the dst file as the
hashed subvol == cached subvol. This causes
the file to reappear once the migration is complete.

This patch also fixes a problem with stale linkfile
deleting.

Backport of>
>Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5
>BUG: 1282390
>Signed-off-by: Mohammed Rafi KC 
>Reviewed-on: http://review.gluster.org/12829
>Tested-by: NetBSD Build System 
>Tested-by: Gluster Build System 
>Reviewed-by: Dan Lambright 
>Tested-by: Dan Lambright 

(cherry picked from commit b5de382afa8c5777e455c7a376fc4f1f01d782d1)

Change-Id: I951adb4d929926bcd646dd7574f7a2d41d57479d
BUG: 1282388
Signed-off-by: Mohammed Rafi KC 
Reviewed-on: http://review.gluster.org/12991
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Dan Lambright

cluster/tier : Reset watermarks in tier

2016-02-19T15:28:35+00:00

A node which contains only cold bricks and has detected that
the high watermark value has been breached on the hot tier will
never reset the watermark to the correct value. The promotion check
will thus always fail and no promotions will occur from that node.

> Change-Id: I0f0804744cd184c263acbea1ee50cd6010a49ec5
> BUG: 1303895
> Signed-off-by: N Balachandran 
> Reviewed-on: http://review.gluster.org/13341
> Smoke: Gluster Build System 
> NetBSD-regression: NetBSD Build System 
> CentOS-regression: Gluster Build System 
> Reviewed-by: Dan Lambright 
(cherry picked from commit 545f4ed2c7195a21210e6a055c27c1b7a115e18c)

Change-Id: Iba3aa9c57cf5828ab87140c2c8257146a8772836
BUG: 1306129
Signed-off-by: N Balachandran 
Reviewed-on: http://review.gluster.org/13411
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Dan Lambright