summaryrefslogtreecommitdiffstats
path: root/xlators/cluster/dht
Commit message (Collapse)AuthorAgeFilesLines
* dht: report constant directory sizeJeff Darcy2016-03-284-1/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Directory size is meaningless. Every filesystem has its own unpredictable way of increasing or decreasing it, based on internal data structures and even transient conditions. Some filesystems (e.g. ext4) never decrease it at all. Others (e.g. btrfs) don't even report it. Very few programs look at it, and those that do are broken. Unfortunately, one such program is GNU tar, which will complain when it sees different values because at different times we got the value from different DHT subvolumes. To avoid such problems, just report a constant value. Backport of http://review.gluster.org/#/c/13770/ > Change-Id: Id64ce917c75b5f7ff50cb55b6e997f3b3556e7e3 > BUG: 1302948 > Original-author: Shyam <srangana@redhat.com> > Signed-off-by: Jeff Darcy <jdarcy@redhat.com> > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13770 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> > Reviewed-by: Vijay Bellur <vbellur@redhat.com> Change-Id: Id64ce917c75b5f7ff50cb55b6e997f3b3556e7e3 BUG: 1312721 Original-author: Shyam <srangana@redhat.com> Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13831 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: update attr information in refresh layout to avoidSakshi Bansal2016-03-281-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | stale timestamp Consider the scenario where an mkdir has just created the directory but has not healed it yet. A parallel lookup on this entry will find anomalies and trigger a selfheal which will sample the ctime of the directory after the mkdir phase. Meanwhile the mkdir has completed setting the layout and updated the ctime. The selfheal then sees the layout to be healed and returns with the ctime it got after the mkdir phase which has now become stale. However if the lookup happens to unwind before the mkdir then the inode associated with lookup will get linked in the inode table which has the stale ctime. To avoid this selfheal must do an iatt_merge in refresh layout to get the latest timestamp irrespective of whether it needs to heal the layout or not. Backport of http://review.gluster.org/#/c/13781/ > Change-Id: I3634c3978bcc1710705f44b48f3876601682d33e > BUG: 1302948 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13781 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Change-Id: I3634c3978bcc1710705f44b48f3876601682d33e BUG: 1312721 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13830 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* Tier: Avoiding stale entries from causing demotion to stophari gowtham2016-03-222-3/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | back-port of : http://review.gluster.org/#/c/13501/ When the parent GFID is a stale entry, the lookup on this parent fails and this in turn fails the demotion process. This patch will make the stale entry error to be skipped. Situation for pargfid to be stale: Consider a folder from a tar file. Once the tar file is untared the files in the tar-file will start to demote. when the demotion is under progress, if we tend to delete the actual folder, then the files under it which are undergoing demotion will do a lookup on the parent which was deleted and become stale entry. This stale entry fails the Lookup and this will fail the demotion of the other files(not from tar) that are supposed to be demoted. >Change-Id: I3d47c32c4077526d477a25912b0135bab98b23fc >BUG: 1311178 >Signed-off-by: hari gowtham <hgowtham@redhat.com> >Reviewed-on: http://review.gluster.org/13501 >Tested-by: hari gowtham <hari.gowtham005@gmail.com> >Smoke: Gluster Build System <jenkins@build.gluster.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Change-Id: I80a77d149180e2a05b3d7943f370b3ef162e545b BUG: 1317366 Signed-off-by: Hari Gowtham <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/13692 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: hari gowtham <hari.gowtham005@gmail.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* TIER: stopping the tierd when the volume goes downhari gowtham2016-03-221-0/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | back-port of : http://review.gluster.org/#/c/13646/ If there are large number of files to be migrated and by this time if the volume goes down, then the tierd has to be stopped. But on a huge query file list it keeps checking for each file before stopping. If the volume comes up before the old tierd dies then due to the presence of old tierd new one won't be created. After the old one completes the task, it dies and the status ends up as failed. This patch will check if the status is still running and then let it continue its work. Else it will stop running the tierd. >Change-Id: I6522a4e2919e84bf502b99b13873795b9274f3cd >BUG: 1315659 >Signed-off-by: hari gowtham <hgowtham@redhat.com> >Reviewed-on: http://review.gluster.org/13646 >Tested-by: Dan Lambright <dlambrig@redhat.com> >Smoke: Gluster Build System <jenkins@build.gluster.com> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I8326dbe5edaaea921e5401f39d148aac322c78d0 BUG: 1318498 Signed-off-by: hari <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/13756 Smoke: Gluster Build System <jenkins@build.gluster.com> Tested-by: hari gowtham <hari.gowtham005@gmail.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: break out of iterating query file once cycle time endsDan Lambright2016-03-161-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When iterating the query file during migration, tiering should break out of the loop once cycle time completes. Otherwise it may be possible to stay in the loop for a long time. If that happens updates to files will become stale and have not impact migration. This is a backport of 13284 > Change-Id: Ib60cf74bc84e8646e6a0da21ff04954b1b83c414 > BUG: 1301227 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/13284 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ia305784b0aa570b70e54f532bf5d68a91b0cf305 BUG: 1318203 Reviewed-on: http://review.gluster.org/13750 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: hari gowtham <hari.gowtham005@gmail.com> Tested-by: Dan Lambright <dlambrig@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: Additional details in error messagesN Balachandran2016-03-071-37/+41
| | | | | | | | | | | | | | | | | | | | | | | | | Added file path/gfid when available to the tier log messages to make debugging easier. > Change-Id: I22dda329367df2b846dcf254594312c997b66083 > BUG: 1273043 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13114 > Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I5cbf8407fb7c493c4be1dda65c0f722ede33487f BUG: 1311865 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13518 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier: Fix unused-but-set-variable warningRaghavendra Talur2016-03-031-2/+0
| | | | | | | | | | | | | Backport of http://review.gluster.org/13591 Change-Id: I20c205849f511acf4797a8a53ea59bfaad60bd35 BUG: 1314548 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/13598 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
* dht: mkdir must unwind with latest ctimeSakshi Bansal2016-02-292-0/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently fops like mkdir used the the ctime it gets after creating the directory entry. But setting layout also updates the ctime of a directory. Hence DHT must get the ctime after the setxattr call and unwind with the latest ctime to avoid mismatch in time seen by applications like tar. Backport of http://review.gluster.org/#/c/13352/ > Change-Id: Iecbbe3aac5244af5da9788b48ccf299ca56b4bae > BUG: 1302948 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13352 > Smoke: Gluster Build System <jenkins@build.gluster.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1312721 Change-Id: Ie09342d3705b40ce98b2935f05ad4402f74ba069 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13536 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/tier: Create linkfiles to hardlinks correctlyN Balachandran2016-02-273-1/+132
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a bug in the way hardlinks are handled in tiered volumes. Ideally, the tier linkto files on the cold tier to files that are hardlinks to each other on the hot tier, should themselves be hardlinks of each other. As they are not, they end up being files with the same gfid but different names for the cold tier dht, and end up overwriting the cached-subvol information stored in the dht inode-ctx. > Change-Id: Ic658a316836e6a1729cfea848b7d212674b0edd2 > BUG: 1305277 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13391 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit eb362c74db84d95aac07febf0d888bd98b3fb2b9) Change-Id: I5dd98e6d248619147974a630d7d72e1942a1cc83 BUG: 1311836 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13517 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier : Fixed wrong variable comparisonN Balachandran2016-02-271-2/+15
| | | | | | | | | | | | | | | | | | | | | | | | | The wrong variable was being checked to determine the watermark value. > Change-Id: If4c97fa70b772187f1fcbdf5193e077cb356a8b1 > BUG: 1303895 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13357 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I0a98e0efbc093a727912107038477239e6d85765 BUG: 1306129 Reviewed-on: http://review.gluster.org/13516 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> Tested-by: mohammed rafi kc <rkavunga@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht: cleanup dict and free memory in rename code pathSakshi Bansal2016-02-251-0/+4
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/13392/ > Change-Id: I2458e18197bdf7565563a85e9021b5b2850c1825 > BUG: 1303945 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13392 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1304889 Change-Id: Ibda96769be2bd5b2708066d83363633fd7286929 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13513 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: loc should store proper gfidSakshi Bansal2016-02-231-0/+1
| | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/12574 > Change-Id: Ic1393d44a9ed4aaba23d7c9ddea45977b9dae5e4 > BUG: 1281265 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/12574 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1308410 Change-Id: I6577971f3db139eccf3b8ca3b25e66d292a6a150 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13443 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* dht: file rename must take blocking inode locksSakshi Bansal2016-02-221-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/13366 Currently DHT takes non-blocking locks for file rename. Due to this during parallel renames some clients fail with EBUSY or ESTALE errors. Hence to avoid application discontinuity file rename must take blocking inode locks. > Change-Id: I986e9d08b3be359f20b1a3e1564e049b0f3dffd3 > BUG: 1304966 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/13366 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I986e9d08b3be359f20b1a3e1564e049b0f3dffd3 BUG: 1310544 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13481 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* tier/create: store TIER_LINKFILE_GFID in xattr dictionaryMohammed Rafi KC2016-02-221-17/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | In tier_create, a new key TIER_LINKFILE_GFID was introduced to avoid a race in stale linkfile deletion. Storing this key in xattr dictionary instead of using local->params dictionary. Because local->params dictionary was also used to create the file before stale linkfile deletion, that leads posix_create to fail, trying to set the added key as extended attributes Back port of> >Change-Id: I24fecb62b47bee65a1e86103925a67d13304c5df >BUG: 1290677 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/13130 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I301c159ee51c77d3876e30f216d0375d761eb91e BUG: 1295359 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13410 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/create: Dynamically allocate gfid memoryMohammed Rafi KC2016-02-221-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently we are storing the memory as a static pointer. There is a chance to go that variable in out of scope. So we should allocate in Dynamic way. Backport of> >Change-Id: I096876deb8055ac3a44681599591a0a032bc0c24 >BUG: 1290677 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/13102 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ibc5cde24db4846ef3152366cdaca81dd1d0e7b21 BUG: 1295359 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13162 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier:delete the linkfile if data file creation failsMohammed Rafi KC2016-02-224-82/+288
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we are creating data file in a hot subvolume then we will create a linkfile in cold subvolume. Linkfile creation happens first. If linkfile creation was successful and data file creation failed, then linkfile in cold subvolume will become stale This patch will delete the linkfile as well, if data file creation fails Also this code duplicates dht_create to make tier_create backport of> >Change-Id: I377a90dad47f288e9576c7323b23cf694a91a7a3 >BUG: 1290677 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/12948 >Reviewed-by: N Balachandran <nbalacha@redhat.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Raghavendra G <rgowdapp@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I3689e8ee387fb6731b4b3a7fe8f8190143482eaa BUG: 1295359 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13161 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/unlink: symlink failed to unlinkMohammed Rafi KC2016-02-221-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | during unlink of a file, we will get stat just after deleting the file, to see if the file is under migration or not but this stat call will fail for symlink if the actual file was deleted. So it is better not to send stat request from client if it is a symlink as we are not migrating symlink back port of> >Change-Id: Idc033b24fa3522b5261e579889d2195b43419682 >BUG: 1293963 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/13074 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ic53a9ad29722927f44004d2e7289c807c7485df0 BUG: 1295347 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13159 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier:unlink during migrationMohammed Rafi KC2016-02-225-75/+374
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | files deleted during promotion were not deleting as the files are moving from hashed to non-hashed On deleting a file that is undergoing promotion, the unlink call is not sent to the dst file as the hashed subvol == cached subvol. This causes the file to reappear once the migration is complete. This patch also fixes a problem with stale linkfile deleting. Backport of> >Change-Id: I4b02a498218c9d8eeaa4556fa4219e91e7fa71e5 >BUG: 1282390 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/12829 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit b5de382afa8c5777e455c7a376fc4f1f01d782d1) Change-Id: I951adb4d929926bcd646dd7574f7a2d41d57479d BUG: 1282388 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12991 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht: set proper errno when hashed subvol is not foundSakshi Bansal2016-02-211-5/+5
| | | | | | | | | | | | | | | | | | | | | | | Backport of http://review.gluster.org/#/c/12559/ > Change-Id: I0c4c72e2f5a9f8a7c60ef65251c596b54de89479 > BUG: 1279705 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/12559 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Susant Palai <spalai@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I0c4c72e2f5a9f8a7c60ef65251c596b54de89479 BUG: 1285829 Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13351 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Skip subvols if no layout presentN Balachandran2016-02-211-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Running "rm -rf" on a tiered volume sometimes caused the client to crash because dht_readdirp_cbk referenced a NULL layout for the hot tier subvol. Now, entries are skipped if the layout is NULL. This can cause "rm -rf" to fail with ENOTEMPTY rmdir failures. > Change-Id: Idd71a9d0f7ee712899cc7113bbf2cd3dcb25808b > BUG: 1307208 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13440 > Smoke: Gluster Build System <jenkins@build.gluster.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit bfd8f92490454f261330da34565334ed5decd0e1) Change-Id: I34e413d669fdfba343446e84fef5d8e9dd7bd1da BUG: 1308400 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13441 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/tier : Reset watermarks in tierN Balachandran2016-02-191-9/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | A node which contains only cold bricks and has detected that the high watermark value has been breached on the hot tier will never reset the watermark to the correct value. The promotion check will thus always fail and no promotions will occur from that node. > Change-Id: I0f0804744cd184c263acbea1ee50cd6010a49ec5 > BUG: 1303895 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13341 > Smoke: Gluster Build System <jenkins@build.gluster.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit 545f4ed2c7195a21210e6a055c27c1b7a115e18c) Change-Id: Iba3aa9c57cf5828ab87140c2c8257146a8772836 BUG: 1306129 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13411 Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: allow db queries to be interruptableDan Lambright2016-02-183-76/+167
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | A query to the database may take a long time if the database has many entries. The tier daemon also sends IPC calls to the bricks which can run slowly, espcially in RHEL6. While it is possible to track down each such instance, the snapshot feature should not be affected by database operations. It requires no migration be underway. Therefore it is okay to pause tiering at any time except when DHT is moving a file. This fix implements this strategy by monitoring when control passes to DHT to migrate a file using the GF_XATTR_FILE_MIGRATE_KEY trigger. If it is not, the pause operation is successful. > Change-Id: I21f168b1bd424077ad5f38cf82f794060a1fabf6 > BUG: 1287842 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/13104 > Reviewed-by: Joseph Fernandes > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I667e0af24eaa66afefa860c4d73b324e4f39b997 BUG: 1288352 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/13199 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* tier/gfdb : Round-Robin read of query filesJoseph Fernandes2016-02-143-83/+329
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Each brick on a host will get a separate query file. 2. While reading query record from these query files we read them in a Round-Robin manner. 3. When an error occurs during migration we rename it to query file with an time stamp and .err extension for better debugging. Backport of http://review.gluster.org/13293 > Change-Id: I27c4285d24fd695d2d5cbd9fd7db3879d277ecc8 > BUG: 1302772 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/13293 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Tested-by: N Balachandran <nbalacha@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Change-Id: I9448f19c10a82fff9b1b3c9387c9026adc0742f1 BUG: 1306514 Signed-off-by: Joseph Fernandes <josferna@redhat.com> Reviewed-on: http://review.gluster.org/13427 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier/dht : Default value for demote-freq, max files and mbJoseph Fernandes2016-02-142-5/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Default value for tier-demote-frequency is 3600 sec to avoid frequent demotions. Default value for tier-max-mb is 4000 mb Default value for tier-max-files is 10000 files Backport of http://review.gluster.org/13270 > Change-Id: Ie60951c478a7462c425059699ab82511aa13fa0a > BUG: 1300412 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/13270 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ic4458b1b78dc3c6059713f3871e9e2dc8f38b39a BUG: 1306302 Reviewed-on: http://review.gluster.org/13424 Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes Tested-by: Joseph Fernandes NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* dht/quota : heal the limit_objects_key xattr needed for inode-quotaManikandan Selvaganesh2016-02-101-0/+11
| | | | | | | | | | | | | | | | | | | | | | | Whenever a new brick is added, quota related xattr's should be healed but currently, the xattr "quota.limit-objects.<suffix>" needed for inode-quota is not being healed. The patch fixes this issue. Backport of http://review.gluster.org/#/c/13299/ > Change-Id: I1e7b229126f7b058642bbc3fb5c109bfd8925325 > BUG: 1302257 > Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Change-Id: I22fbbc30fbc0788a7cf80671c1f3e2a80c9e65e8 BUG: 1306138 Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com> Reviewed-on: http://review.gluster.org/13418 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/dht: Cleanup dict in dht_do_rename()v3.7.8Vijay Bellur2016-02-081-0/+2
| | | | | | | | | | | | | Change-Id: Ib4b3a843e78eccf5b8e0e7776cd0128013a59a3e BUG: 1304889 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/13322 CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: http://review.gluster.org/13358
* cluster/tier: Ignore quota-deem-statfs for watermark calculationN Balachandran2016-01-311-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | The tier process watermark calculations were incorrect when the quota-deem-statfs option was enabled. We now ignore this while calculating hot tier usage to determine watermark levels. Change-Id: I308bc24432e2fa5ad1d5703e80fc391433538bbb BUG: 1302012 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13288 > Smoke: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> > Tested-by: mohammed rafi kc <rkavunga@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit 9ce86c63e349eaafbd923b0b541d75e929ecbae7) Signed-off-by: Nithya Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13292 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* dht : changing variable type to avoid overflowSakshi Bansal2016-01-291-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | For layout computation we find total size of the cluster and store it in an unsigned 32 bit variable. For large clusters this value may overflow which leads to wrong computations and for some bricks the layout may overflow. Hence using unsigned 64 bit to handle large values. > Backport of http://review.gluster.org/12597 > Change-Id: I7c3ba26ea2c4158065ea9e74705a7ede1b6759c7 > BUG: 1282751 > Signed-off-by: Sakshi Bansal <sabansal@redhat.com> > Reviewed-on: http://review.gluster.org/12597 > Reviewed-by: Susant Palai <spalai@redhat.com> > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> BUG: 1294969 Change-Id: Ia66587c6ae4aa3a25b3ac73920b514c1d3c4c2cb Signed-off-by: Sakshi Bansal <sabansal@redhat.com> Reviewed-on: http://review.gluster.org/13127 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* cluster/dht : Ftruncate on migrating file fails with EINVALN Balachandran2016-01-299-81/+341
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | What: If dht_open is called on a migrating file after the inode_ctx is set, subsequent FOPs on that fd do not open the fd on the dst subvol. This is seen when the open-ftruncate-close sequence is repeatedly called on a migrating file. A second call to the sequence described above causes dht_truncate_cbk to call dht_truncate2 as the dht_inode_ctx was already set by the first call. As dht_rebalance_in_progress_check is not called, the fd is not opened on the dst subvol. On a distributed-replicate volume, this causes AFR to open the fd using afr_fix_open, but with the wrong flags, causing posix_ftruncate to fail with EINVAL. The fix: We require fd specific information to make a decision while handling migrating files. Set the fd_ctx to indicate the fd has been opened on the dst subvol and check if it has been set while processing Phase1/Phase2 checks in the FOP callback functions. > Change-Id: I43cdcd8017b4a11e18afdd210469de7cd9a5ef14 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/12985 > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I99f8aceec105f16631def06a263f0561954d14b3 BUG: 1293827 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13071 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* quota: limit xattr for subdir not healed on newly added bricksvmallika2016-01-251-1/+19
| | | | | | | | | | | | | | | | | | | | | | | | This is a backport of http://review.gluster.org/#/c/13100/ DHT after creating missing directory, tries to heal the xattrs. This xattrs operation fails as INTERNAL FOP key was not set > Change-Id: I819d373cf7073da014143d9ada908228ddcd140c > BUG: 1294479 > Signed-off-by: vmallika <vmallika@redhat.com> Change-Id: I30345e919bc9ae882fd0e159c7b03ffc50ed5ef7 BUG: 1294608 Signed-off-by: vmallika <vmallika@redhat.com> Reviewed-on: http://review.gluster.org/13108 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Tested-by: Raghavendra G <rgowdapp@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
* cluster/dht : Rebalance process crashes due to double fd_unrefN Balachandran2016-01-241-6/+11
| | | | | | | | | | | | | | | | | | | | | | | | | The dst_fd was being unrefed twice in case the call to __dht_rebalance_create_dst_file failed. > BUG: 1296611 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/13193 > Reviewed-by: Susant Palai <spalai@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Tested-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 36fcaf275952202ce3f1e621d3b3a34d58054c99) Change-Id: I56c5aff7fa3827887e67936b8aa1ecbd1a08a9e9 BUG: 1297309 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/13212 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Smoke: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* cluster/dht: Handle failure in getxattrSusant Palai2016-01-241-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | Problem: Currently even if we have received xattrs from any one of the subvolume, we unwind with error in case the last subvol (which unwinds) received a negative response. To handle the case check if any of the subvolume has received a response and pass it down. BUG: 1296108 Change-Id: I5279442fabd500934d04509d83ae87fdd69388e2 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/12845 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: http://review.gluster.org/13184 Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* all: reduce "inline" usageKaleb S KEITHLEY2016-01-186-17/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | There are three kinds of inline functions: plain inline, extern inline, and static inline. All three have been removed from .c files, except those in "contrib" which aren't our problem. Inlines in .h files, which are overwhelmingly "static inline" already, have generally been left alone. Over time we should be able to "lower" these into .c files, but that has to be done in a case-by-case fashion requiring more manual effort. This part was easy to do automatically without (as far as I can tell) any ill effect. In the process, several pieces of dead code were flagged by the compiler, and were removed. backport of Change-Id: I56a5e614735c9e0a6ee420dab949eac22e25c155, http://review.gluster.org/11769, BUG: 1245331 Change-Id: Iba1efb0bc578ea4a5e9bf76b7bd93dc1be9eba44 BUG: 1283302 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/12646 Smoke: Gluster Build System <jenkins@build.gluster.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* tier: Demotion failed if the file was renamed when it was in coldMohammed Rafi KC2015-12-301-11/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During migration if the file is present we just open the file in hashed subvol. Now if the linkfile present on hashed is just linkfile to another subvol, we actually open in hashed subvol. But subsequent operation will go to linkto subvol ie, to non-hashed subvol. This operation will get failed since we haven't opened d on non-hashed. Backport of> >Change-Id: I9753ad3a48f0384c25509612ba76e7e10645add3 >BUG: 1292067 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/12980 >Tested-by: NetBSD Build System <jenkins@build.gluster.org> >Reviewed-by: Susant Palai <spalai@redhat.com> >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit d2f48214d436be633efb1136ee951b0736935143) Change-Id: I562ac4ba73e6b572bc1be91e55a029fa75262b33 BUG: 1293342 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/13045 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: do not block in synctask created from pause tierDan Lambright2015-12-253-18/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We had run sleep() in the pause tier callback. Blocking within a synctask is dangerous. The sleep() call does not inform the synctask scheduler that a thread is no longer running. It therefore believes it is running. If a second synctask already exists, it may not be able to run. This occurs if the thread limit in the pool has been reached. Note the pool size only grows when a synctask is created, not when it is moved from wait state to run state, as is the case when an FOP completes. When the tier is paused during migration, synctasks already exist waiting for responses to FOPs to the server with high probability. The fix is to yield() in the RPC callback, which will place the synctask into the wait queue and free up a thread for the FOP callback. A timer wakes the callback after sufficient time has elapsed for the pause to occur. This is a backport of 12987. > Change-Id: I6a947ee04c6e5649946cb6d8207ba17263a67fc6 > BUG: 1267950 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/12987 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I9bb7564d57d59abb98e89c8d54c8b1ff4a5fc3f3 BUG: 1274100 Reviewed-on: http://review.gluster.org/13078 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/dht : Properly free file descriptors during data migrationJoseph Fernandes2015-12-221-6/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | While tier migration, free src and dst fd's when create of destination or open of source fails. Backport of http://review.gluster.org/12969 > Change-Id: I62978a669c6c9fbab5fed9df2716b9b2ba00ddf1 > BUG: 1291566 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12969 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I4d745076b3258a64584778ba88dd665ddd907d27 BUG: 1293348 Reviewed-on: http://review.gluster.org/13046 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* Tier: "tier start force" command implementationhari gowtham2015-12-222-3/+3
| | | | | | | | | | | | | | | | | | | | | back port of : http://review.gluster.org/#/c/12983/ The start command doesnt restart the tier deamon if the deamon is running at one node. hence to bring up the tierd on the nodes where the deamon is down, the force command is implemented. It skips the check for tierd running. >Change-Id: I0037d3e5ecfe56637d0da201a97903c435d26436 >BUG: 1292112 >Signed-off-by: hari gowtham <hgowtham@redhat.com> Change-Id: Idaca442c1a41ded8bf555a6e34eed0ebb9ea4034 BUG: 1293698 Signed-off-by: hari <hgowtham@redhat.com> Reviewed-on: http://review.gluster.org/13069 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: fix tier-max-files bookeeping and helpDan Lambright2015-12-202-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The option tier-max-files represents the maximum number of files trasferred by a node in a gives cycle. Fix help message to reflect the "per node" aspect. The code transferred one more file per cycle than the given value. Also change the default values of max file and max bytes to very large values, effectively we will not throttle migration unless the administrator requests it via CLI. This is a backport of 12984. > Change-Id: Ic2949ed3d8c35afe7c9ae4db72195603cfb2e28f > BUG: 1292671 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/12984 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Change-Id: Ibb234976f912ff4b20e894f2984c63792acfe814 BUG: 1292945 Reviewed-on: http://review.gluster.org/13004 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* dht/rebalance: Use seperate return variable for destination cleanupJoseph Fernandes2015-12-151-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | Use seperate local return variable for destination cleanup, without messing with the function return variable. Backport http://review.gluster.org/12956 > Change-Id: Iaea9ed2927234fdb888aef7a31ec362090e98196 > BUG: 1290975 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12956 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I2df0e29321647fd6eb1719eb3cb7d657fbed3793 BUG: 1291002 Reviewed-on: http://review.gluster.org/12959 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/dht: files are still going to decommissioned subvolMohammed Rafi KC2015-12-111-3/+27
| | | | | | | | | | | | | | | | | | | | | | | | After detach tier start, creates are still going to hot tier. Because when creating data files we are not checking for decommissioned bricks. Backpor of> >Change-Id: I8e28258d9b2367dcc8ad6e5e91d0e54d92fdf771 >BUG: 1289602 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> >Reviewed-on: http://review.gluster.org/12914 >Tested-by: Gluster Build System <jenkins@build.gluster.com> >Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit cb41de7c4a7d620dbab645f2b105286b68bde9a9) Change-Id: I56e4551657a234ac51fa2513dc46a48c19c4fac7 BUG: 1289602 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12937 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com>
* tier : Spawn promotion or demotion thread depending on local bricksJoseph Fernandes2015-12-091-22/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | Spawn Promotion or Demotion depending if there are any Cold or Hot bricks present localy. IF the local HOT brick list is empty dont spawn demote thread. IF the local COLD brick list is empty dont spawn promote thread. Backport of http://review.gluster.org/12912 > Change-Id: I524730e59414dd156c78ec0bd7a3629212697e6e > BUG: 1289578 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12912 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: Ic3113934051c7a751ae56508e00d098d010f4c0e BUG: 1290048 Reviewed-on: http://review.gluster.org/12928 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/dht: Fix mem leak from lookup response dictJoseph Fernandes2015-12-091-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | Fixing memory leak from response dict during a parent lookup to get the path. Backport of http://review.gluster.org/12867 > Change-Id: I60c23d0b25e7f763f0e53c40e71ee053aba6d555 > BUG: 1288019 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12867 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Joseph Fernandes > Tested-by: Joseph Fernandes Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I77dfbb097e0200ace4c1adcf48560caf283007d7 BUG: 1288992 Reviewed-on: http://review.gluster.org/12889 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier : Fix double free in tier processN Balachandran2015-12-081-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | The tier process tries to free ipc_ctr_params twice if the syncop_ipc call in tier_process_ctr_query fails. ipc_ctr_params is freed when ctr_ipc_in_dict is freed. But ctr_ipc_out_dict is NULL when syncop_ipc fails, causing GF_FREE to be called on a non-NULL ipc_ctr_params ptr again. > Change-Id: Ia15f36dfbcd97be5524588beb7caad5cb79efdb4 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/12890 > Reviewed-by: Joseph Fernandes > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > (cherry picked from commit 06818a0fd69bb0d6daabde73e5c3cc2661a70854) Change-Id: Ida2da0416272ff1b04cf51f76467e27b62121f41 BUG: 1289414 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/12904 Reviewed-by: Joseph Fernandes Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* tier/tier: Ignoring status of already migrated filesJoseph Fernandes2015-12-061-5/+16
| | | | | | | | | | | | | | | | | | | | | | | Ignore the status of already migrated files and in the process don't count. Backport http://review.gluster.org/12758 > Change-Id: Idba6402508d51a4285ac96742c6edf797ee51b6a > BUG: 1276141 > Signed-off-by: Joseph Fernandes <josferna@redhat.com> > Reviewed-on: http://review.gluster.org/12758 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Joseph Fernandes <josferna@redhat.com> Change-Id: I44b7b965ecbc34159c2233af1a74762fd410dcaf BUG: 1262860 Reviewed-on: http://review.gluster.org/12884 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: fix loading tier.so into glusterdN Balachandran2015-12-047-41/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | The glusterd process loads the shared libraries of client translators. This failed for tiering due to a reference to dht_methods which is defined as a global variable which is not necessary. The global variable has been removed and this is now a member of dht_conf and is now initialised in the *_init calls. > Change-Id: Ifa0a21e3962b5cd8d9b927ef1d087d3b25312953 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/12863 > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Reviewed-by: Dan Lambright <dlambrig@redhat.com> >Tested-by: Dan Lambright <dlambrig@redhat.com> (cherry picked from commit 96fc7f64da2ef09e82845a7ab97574f511a9aae5) Change-Id: If3cc908ebfcd1f165504f15db2e3079d97f3132e BUG: 1288352 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/12877 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: readdirp to cold tier onlyDan Lambright2015-11-246-109/+507
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is possible a file would get migrated in the middle of a readdir operation. If there are four subvolumes A,B,C,D, and if readdir reads them in order and reaches subvol B, then, if a file is moved from D to A, it will not be included in the readdir output. This phenonema has pre-existed in DHT migration but is more apparent in tiering. When a file is moved off the hashed subvolume a T file is created. For tiering, we will make the cold subvolume the hashed subvolume. This will ensure the creation of a T file. Readdir will not skip T files in the tier translator. Making the cold subvolume the hashed subvolume ensures the T files created on promotions or creates will be less likely to fill the volume. Creates still put the data on the hot subvolume. This is a backport of 12530 > Change-Id: Ifde557d3d0e94a4570ca9f115adee3db2ee75407 > BUG: 1281598 > Signed-off-by: Dan Lambright <dlambrig@redhat.com> > Reviewed-on: http://review.gluster.org/12530 > Tested-by: Gluster Build System <jenkins@build.gluster.com> > Tested-by: NetBSD Build System <jenkins@build.gluster.org> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Signed-off-by: Dan Lambright <dlambrig@redhat.com> Conflicts: xlators/cluster/dht/src/tier.c Change-Id: I5720a4cd04ae5088e5d7d23439b0f90d6bbc6265 BUG: 1283923 Reviewed-on: http://review.gluster.org/12722 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: Do not delete linkto file on demotionN Balachandran2015-11-192-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | The current DHT migration code will always delete the src linkto file after migration as dht always moves files to the hashed subvol. This is not the case in tiering. The lack of linkto files causes rename to fail leaving 2 files with the same name but different gfids on the volume. Modified to leave the linkto file behind if the source volume is the hashed subvolume. > Change-Id: I2b99f7d34b4b719aee6232dc40c6a8f8ba88225d > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: http://review.gluster.org/12551 > Reviewed-by: Dan Lambright <dlambrig@redhat.com> > Tested-by: Dan Lambright <dlambrig@redhat.com> Change-Id: I210b94cdae0409c87af8ba198e3cd263a6c85190 BUG: 1283480 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: http://review.gluster.org/12655 Tested-by: Gluster Build System <jenkins@build.gluster.com> Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* cluster/tier: Disallow detach commit when detach in progressDan Lambright2015-11-162-4/+1
| | | | | | | | | | | | | 1. Check if detach is running, disallow detach commit if so. 2. Cleanup shutdown of tier daemon on detach: do not rerun fix-layout, do not send incorrect status back to glusterd. Change-Id: I97202f748773c1176396a4ffd32a4c7fa9b9c1bc BUG: 1264441 Signed-off-by: Dan Lambright <dlambrig@redhat.com> Reviewed-on: http://review.gluster.org/12272 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Joseph Fernandes
* dht: update cached subvolume during readdirp cbkMohammed Rafi KC2015-11-091-32/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | back port of http://review.gluster.org/#/c/12449/ This reverts commit bb2370514598a99e6ab268af81df57dc16caa2c5. issue and impact: readdirp_cbk was not resetting the layout for files, this causes problem if the files is moved from one cached subvolume and if the layout was not proper, then there is chance to fail entry fops if the fops executed with out a lookup. Because the cached subvolume will not change and the application assumes the presence of file in cached subvol. so it fails with ENOENT. The patch preset the layout information in readdirp cbk for each files in the entry. That leaves the problem the commit bb2370514598a99e6ab268af81df57dc16caa2c5 try to fix. We will fix the problem in a separate patch. Change-Id: Ia09f2b5edbacaeb31cf1b730c27b76ca1c93f1a8 BUG: 1279095 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12538 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Dan Lambright <dlambrig@redhat.com> Tested-by: Dan Lambright <dlambrig@redhat.com>
* dht: heal directory path if the directory is not presentMohammed Rafi KC2015-11-093-7/+246
| | | | | | | | | | | | | | | | | | | | | back port of http://review.gluster.org/#/c/12376/ After a successful nameless lookup if the directory is not present on any of the subvol, then we will get the path of the directory and will recursively send a named lookp on each parent directory. This will help particularly for the scenarios like add brick and attach-tier. Change-Id: I97f3178adb08b88910f709f0acf1d86c147be017 BUG: 1279095 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: http://review.gluster.org/12539 Tested-by: NetBSD Build System <jenkins@build.gluster.org> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
generated by cgit (git 2.34.1) at 2025-07-14 09:25:02 +0000