summaryrefslogtreecommitdiffstats
path: root/xlators
Commit message (Collapse)AuthorAgeFilesLines
* server/auth: add option for strict authenticationMohammed Rafi KC2018-04-246-12/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When this option is enabled, we will check for a matching username and password, if not found then the connection will be rejected. This also does a checksum validation of volfile The option is invalid when SSL/TLS is in use, at which point the SSL/TLS certificate user name is used to validate and hence authorize the right user. This expects TLS allow rules to be setup correctly rather than the default *. This option is not settable, as a result this cannot be enabled for volumes using the CLI. This is used with the shared storage volume, to restrict access to the same in non-SSL/TLS environments to the gluster peers only. Tested: ./tests/bugs/protocol/bug-1321578.t ./tests/features/ssl-authz.t - Ran tests on volumes with and without strict auth checking (as brick vol file needed to be edited to test, or rather to enable the option) - Ran tests on volumes to ensure existing mounts are disconnected when we enable strict checking Change-Id: I2ac4f0cfa5b59cc789cc5a265358389b04556b59 fixes: bz#1570428 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Signed-off-by: ShyamsundarR <srangana@redhat.com>
* shared storage: Prevent mounting shared storage from non-trusted clientMohammed Rafi KC2018-04-241-0/+21
| | | | | | | | | | | | | | gluster shared storage is a volume used for internal storage for various features including ganesha, geo-rep, snapshot. So this volume should not be exposed to the client, as it is a special volume for internal use. This fix wont't generate non trusted volfile for shared storage volume. Change-Id: I8ffe30ae99ec05196d75466210b84db311611a4c updates: bz#1570428 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
* features/index: Choose different base file on EMLINK errorPranith Kumar K2018-04-101-18/+9
| | | | | | | | Change-Id: I4648816af908539efdc2528608aa2ebf7f0d0e2f fixes: bz#1553777 BUG: 1553777 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> (cherry picked from commit bb12f2109a01856e8184e13cf984210d20155b13)
* cluster/ec: send list-node-uuids request to all subvolumesXavi Hernandez2018-04-051-1/+1
| | | | | | | | | | | | | | | The xattr trusted.glusterfs.list-node-uuids was only sent to a single subvolume. This was returning null uuids from the other subvolumes as if they were down. This fix forces that xattr to be requested from all subvolumes. Backport of: > BUG: 1561406 Change-Id: If62eb39a6857258923ba625e153d4ad79018ea2f BUG: 1561732 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* glusterd : introduce timer in mgmt_v3_lockGaurav Yadav2018-03-204-14/+238
| | | | | | | | | | | | | | | | | | | | Problem: In a multinode environment, if two of the op-sm transactions are initiated on one of the receiver nodes at the same time, there might be a possibility that glusterd may end up in stale lock. Solution: During mgmt_v3_lock a registration is made to gf_timer_call_after which release the lock after certain period of time >Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843 >BUG: 1499004 >Signed-off-by: Gaurav Yadav <gyadav@redhat.com> Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843 BUG: 1557304 Signed-off-by: Gaurav Yadav <gyadav@redhat.com>
* glusterd/ganesha : create/remove export file only from the node which ↵Jiffin Tony Thottan2018-03-163-3/+3
| | | | | | | | | | | | performs ganesha.enable As part of volume set ganesha.enable on the ganesha export configuration file will be created/removed using "create-export-ganesha.sh". This performed from the nodes which are part of ganesha cluster. But it is not need since the file is saved in shared storage and consumed by the nodes in the ganesha cluster. Change-Id: I2583899972b47d451a90711940293004a3af4690 BUG: 1555195 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
* cluster/ec: avoid delays in self-healXavi Hernandez2018-03-154-46/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Self-heal creates a thread per brick to sweep the index looking for files that need to be healed. These threads are started before the volume comes online, so nothing is done but waiting for the next sweep. This happens once per minute. When a replace brick command is executed, the new graph is loaded and all index sweeper threads started. When all bricks have reported, a getxattr request is sent to the root directory of the volume. This causes a heal on it (because the new brick doesn't have good data), and marks its contents as pending to be healed. This is done by the index sweeper thread on the next round, one minute later. This patch solves this problem by waking all index sweeper threads after a successful check on the root directory. Additionally, the index sweep thread scans the index directory sequentially, but it might happen that after healing a directory entry more index entries are created but skipped by the current directory scan. This causes the remaining entries to be processed on the next round, one minute later. The same can happen in the next round, so the heal is running in bursts and taking a lot to finish, specially on volumes with many directory levels. This patch solves this problem by immediately restarting the index sweep if a directory has been healed. Backport of: > BUG: 1547662 Change-Id: I58d9ab6ef17b30f704dc322e1d3d53b904e5f30e BUG: 1555203 Signed-off-by: Xavi Hernandez <jahernan@redhat.com>
* glusterd/ganesha : change voltype for ganesha.enable in volume option tableJiffin Tony Thottan2018-02-271-1/+1
| | | | | | | | | | The voltype defined for ganesha.enable is features/ganesha. But ganesha xlator was removed from client stack long back. Now it is defined as part of glusterd. So reflecting the same on the volume option table. Change-Id: Ifedd7493020b77bd54edfdbdd9c799d93b24d0aa BUG: 1486542 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
* glusterd: fix tier-enabled flag op-version checkAtin Mukherjee2018-02-211-2/+2
| | | | | | | | | | | | | | tier-enabled flag in volinfo structure was introduced in 3.10, however while writing this value to the glusterd store was done with a wrong op-version check which results into volume checksum failure during upgrades. >Change-Id: I4330d0c4594eee19cba42e2cdf49a63f106627d4 >BUG: 1544600 >Signed-off-by: Atin Mukherjee <amukherj@redhat.com> Change-Id: I4330d0c4594eee19cba42e2cdf49a63f106627d4 BUG: 1544461 Signed-off-by: hari gowtham <hgowtham@redhat.com>
* cluster/dht: Add migration checks to dht_(f)xattropN Balachandran2018-01-308-53/+372
| | | | | | | | | | | | | | | | The dht_(f)xattrop implementation did not implement migration phase1/phase2 checks which could cause issues with rebalance on sharded volumes. This does not solve the issue where fops may reach the target out of order. > Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c > BUG: 1471031 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c BUG: 1498081 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* posix: delete stale gfid handles in nameless lookupRavishankar N2018-01-181-1/+16
| | | | | | | | | | | ..in order for self-heal of symlinks to work properly (see BZ for details). Backport of https://review.gluster.org/#/c/19070/ Signed-off-by: Ravishankar N <ravishankar@redhat.com> Change-Id: I9a011d00b07a690446f7fd3589e96f840e8b7501 BUG: 1534848
* storage/posix: Handle [f]xattrop xdataPranith Kumar K2018-01-152-26/+43
| | | | | | | | | | | | | Updates #251 Change-Id: I13d89c3b5dc39aa0a232a70be8ec6b64394cfa6e Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17740 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> (cherry picked from commit 73b2b7fe57069eb85485465fb92c52a97d4d411c)
* glusterd: Nullify pmap entry for bricks belonging to same portAtin Mukherjee2018-01-031-1/+1
| | | | | | | | | | | | | Commit 30e0b86 tried to address all the stale port issues glusterd had in case of a brick is abruptly killed. For brick multiplexing case because of a bug the portmap entry was not getting removed. This patch addresses the same. >mainline patch : https://review.gluster.org/#/c/19119/ Change-Id: Ib020b967a9b92f1abae9cab9492f0cacec59aaa1 BUG: 1530450 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* mount/fuse: use fstat in getattr implementation if any opened fd is availableRaghavendra G2018-01-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The restriction of using fds opened by the same Pid means fds cannot be shared across threads of multithreaded application. Note that fops from kernel have different Pid for different threads. Imagine following sequence of operations: * Turn off performance.open-behind * Thread t1 opens an fd - fd1 - on file "file". Let's assume nodeid of "file" is "nodeid-file". * Thread t2 does RENAME ("newfile", "file"). Let's assume nodeid of "newfile" as "nodeid-newfile". * t2 proceeds to do fstat (fd1) The above set of operations can sometimes result in ESTALE/ENOENT errors. RENAME overwrites "file" with "newfile" changing its nodeid from "nodeid-file" to "nodeid-newfile" and post RENAME, "nodeid-file" is removed from the backend. If fstat carries nodeid-file as argument, which can happen if lookup has not refreshed the nodeid of "file" and since t2 doesn't have an fd opened, fuse_getattr_resume uses STAT which will fail as "nodeid-file" no longer exists. Since the above set of operations and sharing of fds across multiple threads are valid, this is a bug. The fix is to use any fd opened on the inode. In this specific example fuse_getattr_resume will find fd1 and winds down the call as fstat (fd1) which won't fail. Cross-checked with "Miklos Szeredi" <mszeredi.at.redhat.dot.com> for any security issues with this solution and he approves the solution. Thanks to "Miklos Szeredi" <mszeredi.at.redhat.dot.com> for all the pointers and discussions. >Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c >BUG: 1510401 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 8b57378e5596f287a7b9d106dd6fb56a624b42ee) Change-Id: I88dd29b3607cd2594eee9d72a1637b5346c8d49c BUG: 1529086 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* Revert "mount/fuse: report ESTALE as ENOENT"Raghavendra G2018-01-021-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 26d16b90ec7f8acbe07e56e8fe1baf9c9fa1519e. Consider rename (index.new, store.idx) and open (store.idx) being executed in parallel. When we break down operations following sequence is possible. * lookup (store.idx) - as part of open(store.idx) returns gfid1 as the result. * rename (index.new, store.idx) changes gfid of store.idx to gfid2. Note that gfid2 was the nodeid of index.new. Since rename is successful, gfid2 is associated with store.idx. * open (store.idx) resumes and issues open fop to glusterfs with gfid1. open in glusterfs fails as gfid1 doesn't exist and the error returned by glusterfs to kernel-fuse is ENOENT. * kernel passes back the same error to application as a result to open. This error could've been prevented if kernel retries open with gfid2. Interestingly kernel do retry open when it receives ESTALE error. Even though failure to find gfid resulted in ESTALE error, commit 26d16b90ec7f8acb converted that error to ENOENT while sending an error reply to kernel. This prevented kernel from retrying open resulting in error. >Change-Id: I2e752ca60dd8af1b989dd1d29c7b002ee58440b4 >BUG: 1500269 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 019a55e708375d2b1e576fcc948a691bcdc5c749) Change-Id: I2e752ca60dd8af1b989dd1d29c7b002ee58440b4 BUG: 1529089 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* mount/fuse: never fail open(dir) with ENOENTRaghavendra G2018-01-021-0/+7
| | | | | | | | | | | | | | | open(dir) being an operation on inode should never fail with ENOENT. If gfid is not present, the appropriate error is ESTALE. This will enable kernel to retry open after a revalidate lookup. >Change-Id: I8d07d2ebb5a0da6c3ea478317442cb42f1797a4b >BUG: 1500269 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit fb4b914ce84bc83a5f418719c5ba7c25689a9251) Change-Id: I8d07d2ebb5a0da6c3ea478317442cb42f1797a4b BUG: 1529089 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* performance/write-behind: fix bug while handling short writesRaghavendra G2017-12-261-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The variabled "fulfilled" in wb_fulfill_short_write is not reset to 0 while handling every member of the list. This has some interesting consequences: * If we break from the loop while processing last member of the list head->winds, req is reset to head as the list is a circular one. However, head is already fulfilled and can potentially be freed. So, we end up adding a freed request to wb_inode->todo list. This is the RCA for the crash tracked by the bug associated with this patch (Note that we saw "holder" which is freed in todo list). * If we break from the loop while processing any of the last but one member of the list head->winds, req is set to next member in the list, skipping the current request, even though it is not entirely synced. This can lead to data corruption. The fix is very simple and we've to change the code to make sure "fulfilled" reflects whether the current request is fulfilled or not and it doesn't carry history of previous requests in the list. >Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8 >BUG: 1528558 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 0bc22bef7f3c24663aadfb3548b348aa121e3047) Change-Id: Ia3d6988175a51c9e08efdb521a7b7938b01f93c8 BUG: 1529096 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd: Free up svc->conn on volume deleteAtin Mukherjee2017-12-071-0/+4
| | | | | | | | | | | | Daemons like snapd, tierd and gfproxyd are maintained on per volume basis and on a volume delete we should destroy the rpc connection established for them. >mainline patch : https://review.gluster.org/#/c/18957 Change-Id: Id1440e39da07b990fdb9b207df18da04b1ca8014 BUG: 1523050 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* features/locks: Fix memory leaksXavier Hernandez2017-11-275-5/+11
| | | | | | | | | Backport of: > BUG: 1515161 Change-Id: Ic1d2e17a7d14389b6734d1b88bd28c0a2907bbd6 BUG: 1517682 Signed-off-by: Xavier Hernandez <jahernan@redhat.com>
* cluster/afr: Honor default timeout of 5min for analyzing split-brain fileskarthik-us2017-11-271-1/+5
| | | | | | | | | | | | | | | | | | | | Problem: After setting split-brain-choice option to analyze the file to resolve the split brain using the command "setfattr -n replica.split-brain-choice -v "choiceX" <path-to-file>" should allow to access the file from mount for default timeout of 5mins. But the timeout was not honored and was able to access the file even after the timeout. Fix: Call the inode_invalidate() in afr_set_split_brain_choice_cbk() so that it will triger the cache invalidate after resetting the timer and the split brain choice. So the next calls to access the file will fail with EIO. Change-Id: I698cb833676b22ff3e4c6daf8b883a0958f51a64 BUG: 1514388 Signed-off-by: karthik-us <ksubrahm@redhat.com> (cherry picked from commit 933ec57ccda2c1ba5ce6f207313c3b6802e67ca3)
* glusterd: use sys_lstat instead of lstatJeff Darcy2017-11-011-1/+1
| | | | | | | | | | | | | Showed up in 0symbol-check.t while testing something else. Might as well fix it now. > Signed-off-by: Jeff Darcy <jdarcy@redhat.com> > Reviewed-on: https://review.gluster.org/16820 > Reviewed-by: Prashanth Pai <ppai@redhat.com> (cherry picked from commit 9ed98f23564387c5b436a0c6ec6d4393f970dcb9) BUG: 1508036 Change-Id: Ic6b8214de6f486187afc4987c5ffbbca02c8997f
* glusterd: delete source brick only once in reset-brick commit forceAtin Mukherjee2017-10-311-1/+1
| | | | | | | | | | | | | | While stopping the brick which is to be reset and replaced delete_brick flag was passed as true which resulted glusterd to free up to source brick before the actual operation. This results commit force to fail failing to find the source brickinfo. > mainline patch : https://review.gluster.org/#/c/18581/ Change-Id: I1aa7508eff7cc9c9b5d6f5163f3bb92736d6df44 BUG: 1507880 Signed-off-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 0fb8acaa6ff80c43e46deac0ce66b29ae0df0ca4)
* glusterd: clean up portmap on brick disconnectAtin Mukherjee2017-10-314-16/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | GlusterD's portmap entry for a brick is cleaned up when a PMAP_SIGNOUT event is initiated by the brick process at the shutdown. But if the brick process crashes or gets killed through SIGKILL then this event is not initiated and glusterd ends up with a stale port. Since GlusterD's portmap traversal happens both ways, forward for allocation and backward for registry search, there is a possibility that glusterd might end up running with a stale port for a brick which eventually will end up with clients to fail to connect to the bricks. Solution is to clean up the port entry in case the process is down as part of the brick disconnect event. Although with this the handling PMAP_SIGNOUT event becomes redundant in most of the cases, but this is the safeguard method to avoid glusterd getting into the stale port issues. This patch also needs to bring in the changes from change id I705f101739ab1647ff52a92820d478354407264a which is needed for the compilation to go through. > mainline patch : https://review.gluster.org/#/c/18541/ > https://review.gluster.org/#/c/17129/ Change-Id: I04c5be6d11e772ee4de16caf56dbb37d5c944303 BUG: 1507749 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* glusterd: persist brickinfo's port change into glusterd's storeGaurav Yadav2017-10-315-10/+62
| | | | | | | | | | | | | | | | | | Problem: Consider a case where node reboot is performed and prior to reboot brick was listening to 49153. Post reboot glusterd assigned 49152 to brick and started the brick process but the new port was never persisted. Now when glusterd restarts glusterd always read the port from its persisted store i.e 49153 however pmap signin happens with the correct port i.e 49152. Fix: Make sure when glusterd_brick_start is called, glusterd_store_volinfo is eventually invoked. Change-Id: Ic0efbd48c51d39729ed951a42922d0e59f7115a1 BUG: 1507752 Signed-off-by: Gaurav Yadav <gyadav@redhat.com>
* md-cache: Use correct xattr keynames for virtual glusterfs ACLs.Günther Deschner2017-10-251-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The "glusterfs.posix_acl." prefix does not catch the glusterfs posix acl xattr keynames which are * "glusterfs.posix.acl" and * "glusterfs.posix.default_acl" Using the GF_POSIX_ACL_ACCESS and GF_POSIX_ACL_DEFAULT defines directly is the savest option. Guenther > Change-Id: I5aba64b26b6cbec850ea02316dd9f069400e857f > BUG: 1476295 > Signed-off-by: Guenther Deschner <gd@samba.org> > Reviewed-on: https://review.gluster.org/17909 > Reviewed-by: Michael Adam <obnox@samba.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Niels de Vos <ndevos@redhat.com> > Tested-by: Niels de Vos <ndevos@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > (cherry picked from commit 5fe8555800cbc9818e7c976f63499795a378cd8d) > Signed-off-by: Günther Deschner <gd@samba.org> Change-Id: I5aba64b26b6cbec850ea02316dd9f069400e857f BUG: 1499890 Signed-off-by: Günther Deschner <gd@samba.org>
* libgfchangelog: Fix possible null pointer dereferenceKotresh HR2017-10-251-6/+6
| | | | | | | | | | | | | | | If pthread_attr_init fails, gf_msg uses this->name where 'this' is not initialized yet. This patch fixes the same. > Change-Id: Ie004cbe1015a0d62fc3b5512e8954c5606eeeb5f > Signed-off-by: Kotresh HR <khiremat@redhat.com> > BUG: 1505325 (cherry picked from commit 738c38f0efa7b4d4dab0cf23d00589d68e4eb88d) Change-Id: Ie004cbe1015a0d62fc3b5512e8954c5606eeeb5f Signed-off-by: Kotresh HR <khiremat@redhat.com> BUG: 1503405
* glusterd: Gluster should keep PID file in correct locationGaurav Kumar Garg2017-10-256-24/+152
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently Gluster keeps process pid information of all the daemons and brick processes in Gluster configuration file directory (ie., /var/lib/glusterd/*). These pid files should be seperate from configuration files. Deletion of the configuration file directory might result into serious problems. Also, /var/run/gluster is the default placeholder directory for pid files. So, with this fix Gluster will keep all process pid information of all processes in /var/run/gluster/* directory. > Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4 > BUG: 1258561 > Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> > Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> > Reviewed-on: https://review.gluster.org/13580 > Tested-by: MOHIT AGRAWAL <moagrawa@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> > (Cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d) BUG: 1491059 Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* features/worm: Adding check to newloc when doing renameluneo72017-10-251-2/+16
| | | | | | | | | | | | | | | | | | | | | | | Problem: Since rename didn't check if newloc exists and it's retention state it was possible to rename a new file that wasn't in retention over a existing file that was in read-only state. Cherry picked from commit 00a4dc0: > Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229 > BUG: 1484490 > Signed-off-by: luneo7 <luneo7@gmail.com> > Reviewed-on: https://review.gluster.org/18104 > Tested-by: jiffin tony Thottan <jthottan@redhat.com> > Tested-by: Prashanth Pai <ppai@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Prashanth Pai <ppai@redhat.com> > Reviewed-by: Karthik U S <ksubrahm@redhat.com> > Reviewed-by: Amar Tumballi <amarts@redhat.com> Change-Id: I63c6bbabb7bb456ebedf201cc77b878ffda62229 BUG: 1480788 Signed-off-by: luneo7 <luneo7@gmail.com>
* md-cache: avoid checking the xattr value buffer with string functions.Günther Deschner2017-10-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | xattrs may very well contain binary, non-text data with leading 0 values. Using strcmp for checking empty values is not the appropriate thing to do: In the best case, it might treat a binary xattr value starting with 0 from being cached (and hence also from being reported back with xattr). In the worst case, we might read beyond the end of a data blob that does contain any zero byte. We fix this by checking the length of the data blob and checking the first byte against 0 if the length is one. > Signed-off-by: Guenther Deschner <gd@samba.org> > Pair-Programmed-With: Michael Adam <obnox@samba.org> > Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11 > BUG: 1476324 > Reviewed-on: https://review.gluster.org/17910 > Reviewed-by: Michael Adam <obnox@samba.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Poornima G <pgurusid@redhat.com> > Tested-by: Poornima G <pgurusid@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > (cherry picked from commit ab4ffdac9dec1867f2d9b33242179cf2b347319d) Change-Id: If723c465a630b8a37b6be58782a2724df7ac6b11 BUG: 1499893 Signed-off-by: Günther Deschner <gd@samba.org>
* afr: don't check for file size in afr_mark_source_sinks_if_file_emptyRavishankar N2017-10-051-6/+7
| | | | | | | | | | ... for AFR_METADATA_TRANSACTION and just mark source and sinks if metadata is the same. (cherry picked from commit 24637d54dcbc06de8a7de17c75b9291fcfcfbc84) Change-Id: I69e55d3c842c7636e3538d1b57bc4deca67bed05 BUG: 1496321 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* cluster/dht: EBADF handling for fremovexattr and fsetxattrN Balachandran2017-10-033-3/+44
| | | | | | | | | | | | | | | | | Add EBADF handling for dht_fremovexattr and dht_fsetxattr. > BUG: 1476665 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: https://review.gluster.org/17999 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> (cherry picked from commit 747a08d34e2a1e94d7fce68a3577370288bb1955) Change-Id: Ide0d5812dae79655d2565157e5baabcd753b4309 BUG: 1467010 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* mount/fuse: Make event-history feature configurableKrutika Dhananjay2017-10-023-14/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | ... and disable it by default. Backport of: > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485 > Reviewed-on: https://review.gluster.org/18242 > BUG: 1467614 > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6 This is because having it disabled seems to improve performance. This could be due to the lock contention by the different epoll threads on the circular buff lock in the fop cbks just before writing their response to /dev/fuse. Just to provide some data - wrt ovirt-gluster hyperconverged environment, I saw an increase in IOPs by 12K with event-history disabled for randrom read workload. Usage: mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT OR glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485 BUG: 1495430 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* afr: auto-resolve split-brains for zero-byte filesRavishankar N2017-10-023-0/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/#/c/18283/ Problems: As described in BZ 1491670, renaming hardlinks can result in data/mdata split-brain of the DHT link-to files (T files) without any mismatch of data and metadata. As described in BZ 1486063, for a zero-byte file with only dirty bits set, arbiter brick will likely be chosen as the source brick. Fix: For zero byte files in split-brain, pick first brick as a) data source if file size is zero on all bricks. b) metadata source if metadata is the same on all bricks In arbiter case, if file size is zero on all bricks and there are no pending afr xattrs, pick 1st brick as data source. Change-Id: I0270a9a2f97c3b21087e280bb890159b43975e04 BUG: 1496321 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reported-by: Rahul Hinduja <rhinduja@redhat.com> Reported-by: Mabi <mabi@protonmail.ch>
* dht: add FOP check to dht_file_setattr_cbkRavishankar N2017-09-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Problem: bug-797171.7 loaded error-gen xlator on the brick which sent EBADF for a non fd-based fop, namely setattr. This caused dht_check_and_open_fd_on_subvol_task() to crash as local->fd was NULL. Fix: Call dht_check_and_open_fd_on_subvol_task() from dht_file_setattr_cbk only for dht_fsetattr and not dht_setattr or dht_setattr2 > Reviewed-on: https://review.gluster.org/18208 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Susant Palai <spalai@redhat.com> > Reviewed-by: Amar Tumballi <amarts@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> (cherry picked from commit 47188e9eac59de416a5c86c7ec7540ed6aaa1c98) Signed-off-by: Ravishankar N <ravishankar@redhat.com> Change-Id: Iab4999e213bf2065804f3f8237e470ad454e3c99 BUG: 1497122
* posix: add sanity checks for removing the gfid symlink for directoriesRavishankar N2017-09-231-14/+56
| | | | | | | | | | | | | Backport of https://review.gluster.org/17945 ...during mkdir and rmdir. Otherwise, during entry self-heal, the directory could be left out without a .glusterfs symlink causing fops like opendir, readdir to fail. The only chance the missing symlink will be created is when a fresh lookup comes on it. Change-Id: I2e1cf1bce8962ea80187edd8f6d73e0a09cf9f8e BUG: 1491966 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* afr: Prevent null gfids in self-heal entry re-creationRavishankar N2017-09-171-3/+11
| | | | | | | | | | | | | | | | | | > Reviewed-on: https://review.gluster.org/17981 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Amar Tumballi <amarts@redhat.com> > Reviewed-by: Karthik U S <ksubrahm@redhat.com> (cherry picked from commit bead74a6e085001225bc0704bad1a5db36dd75a1) Change-Id: I5acb8bd0a19fc4e764d61e349bb690b5236ee610 BUG: 1491985 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/18300 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* afr: heal metadata in discover code pathRavishankar N2017-09-172-17/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | ****************************************************** Backport of: https://review.gluster.org/18202 Also added loc_is_nameless() to libglusterfs since the patch that introduced it in master was not backported to release-3.10. Note: 18202 is a squash of 17850 and 18187 in master. ****************************************************** During graph switch, if fuse sends nameless (gfid) lookups, afr takes the discover code path to serve it. If there are pending metadata heals, they do not happen unless an inode refresh happens as a part of discover (which is not guaranteed to happen always). This patch fixes it by attempting metadata heal as a part of discover, just like how it is done in lookup code path. Change-Id: I87c493045b9225741cad173bf3f645848697032e BUG: 1492010 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/18304 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: fix invalid memory reference returnedXavier Hernandez2017-09-171-2/+9
| | | | | | | | | | | | | | | | | | > BUG: 1490897 > Reviewed-on: https://review.gluster.org/18263 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> > Reviewed-by: Gaurav Yadav <gyadav@redhat.com> Change-Id: I0823c7b33060b48040c1d86ad346a5f6e15bc190 BUG: 1491166 Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-on: https://review.gluster.org/18281 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* posix: add null gfid checksRavishankar N2017-09-174-0/+50
| | | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/17975 ...in file/dir creation and lookup codepaths. The check is relaxed for fops coming from trash xlator at the moment until trash has client side logic to send the create fops with gfid-req. Also fixed the missing trash pid assignment in creates sent by trash xlator. Without this, truncated files won't be moved to .trashcan. Change-Id: Ieddd7f0634850e7c7010e4fbb4ad1eead35888c8 BUG: 1491985 Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/18302 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* afr: check validity of afr_replyRavishankar N2017-09-175-47/+71
| | | | | | | | | | | | | | | | | | | | | | ...in various self-heal code paths. Originally found by Pranith in __afr_selfheal_name_impunge () Also change __afr_selfheal_assign_gfid() to send lookup only on those bricks that don't have a gfid matching that of the source. > Reviewed-on: https://review.gluster.org/18065 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> (cherry picked from commit d594900dbca92c356152be65fce16f77c402117c) Change-Id: I70a2ccd750a2af92c5fc36e0eefb2b6125404b4a BUG: 1491995 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Signed-off-by: Ravishankar N <ravishankar@redhat.com> Reviewed-on: https://review.gluster.org/18303 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* features/shard: Return aggregated size in stbuf of LINK fopKrutika Dhananjay2017-09-171-2/+42
| | | | | | | | | | | | | | | | Backport of: > Change-Id: I42df7679d63fec9b4c03b8dbc66c5625f097fac0 > Reviewed-on: https://review.gluster.org/18209 > BUG: 1488546 > cherry-picked from 91430817ce5bcbeabf057e9c978485728a85fb2b Change-Id: I42df7679d63fec9b4c03b8dbc66c5625f097fac0 BUG: 1488719 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/18212 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* features/shard: Increment counts in locksPranith Kumar K2017-09-171-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/18203 Problem: Because create_count/eexist_count are incremented without locks, all the shards may not be created because call_count will be lesser than what it needs to be. This can lead to crash in shard_common_inode_write_do() because inode on which we want to do fd_anonymous() is NULL Fix: Increment the counts in frame->lock >Change-Id: Ibc87dcb1021e9f4ac2929f662da07aa7662ab0d6 >BUG: 1488354 >Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Change-Id: Ibc87dcb1021e9f4ac2929f662da07aa7662ab0d6 BUG: 1488391 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/18206 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* glusterd : glusterd fails to start when peer's network interface is downGaurav Yadav2017-09-173-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: glusterd fails to start on nodes where glusterd tries to come up even before network is up. Fix: On startup glusterd tries to resolve brick path which is based on hostname/ip, but in the above scenario when network interface is not up, glusterd is not able to resolve the brick path using ip_address or hostname With this fix glusterd will use UUID to resolve brick path. >Reviewed-on: https://review.gluster.org/17813 >Smoke: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Prashanth Pai <ppai@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Change-Id: Icfa7b2652417135530479d0aa4e2a82b0476f710 BUG: 1482857 Signed-off-by: Gaurav Yadav <gyadav@redhat.com> Reviewed-on: https://review.gluster.org/18063 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
* cluster/dht: Check for open fd only on EBADFN Balachandran2017-09-174-160/+160
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | DHT fd based fops used to check if the fd was open on the cached subvol before winding the call. However, this introduced a performance regression of about 30% for reads. This check was introduced to handle cases where files were migrated while IOs were happening. As this is not the common case, dht will now check if the fd is open on the cached subvol only if the call fails with EBADF. This will prevent a performance hit where a rebalance is not running. > BUG: 1476665 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: https://review.gluster.org/17976 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Amar Tumballi <amarts@redhat.com> > Reviewed-by: Susant Palai <spalai@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808 BUG: 1467010 Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/18057 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* cluster/ec: Node uuid xattr support update for ECSunil Kumar Acharya2017-09-132-6/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The change in EC to return list of node uuids for GF_XATTR_NODE_UUID_KEY was causing problems with geo-rep. Fix: This patch will allow to get the single node uuid as it was doing before with the key "GF_XATTR_NODE_UUID_KEY", and will also allow to get the list of node uuids by using a new key "GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve the problem with geo-rep and any other features which were depending on this. >BUG: 1462790 >Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb >Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> >Reviewed-on: https://review.gluster.org/17594 >Smoke: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> BUG: 1487647 Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> Reviewed-on: https://review.gluster.org/17667 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* xlators/ganesha : remove ganesha xlator code from 3.10Jiffin Tony Thottan2017-09-071-19/+0
| | | | | | | | | | | | | | | The commit e4a4043 have removed ganesha xlator from glusterfs codebase. But while reverting back ganesha changes, the Makefile.am in xlators/ganesha got resurrected. Change-Id: I6efaacaf1fe426da974608ddac5eae4a43800983 BUG: 1486542 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: https://review.gluster.org/18147 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* cluster/ec: return all node uuids from all subvolumesXavier Hernandez2017-09-012-105/+141
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | EC was retuning the UUID of the brick with smaller value. This had the side effect of not evenly balancing the load between bricks on rebalance operations. This patch modifies the common functions that combine multiple subvolume values into a single result to take into account the subvolume order and, optionally, other subvolumes that could be damaged. This makes easier to add future features where brick order is important. It also makes possible to easily identify the originating brick of each answer, in case some brick will have an special meaning in the future. >Change-Id: Iee0a4da710b41224a6dc8e13fa8dcddb36c73a2f >BUG: 1366817 >Signed-off-by: Xavier Hernandez <xhernandez@datalab.es> >Reviewed-on: https://review.gluster.org/17297 >Smoke: Gluster Build System <jenkins@build.gluster.org> >NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Ashish Pandey <aspandey@redhat.com> >Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> BUG: 1487042 Change-Id: Iee0a4da710b41224a6dc8e13fa8dcddb36c73a2f Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> Reviewed-on: https://review.gluster.org/18148 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* nfs: add NULL check for call state in nfs3_call_state_wipeJiffin Tony Thottan2017-08-111-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refcounting added for nfs call state in https://review.gluster.org/17696. This is based on assumption that call state won't NULL when it is freed. But currently gluster nfs server is crashing in different scenarios at nfs3_getattr() with following bt #0 0x00007ff1cfea9205 in _gf_ref_put (ref=ref@entry=0x0) at refcount.c:36 #1 0x00007ff1c1997455 in nfs3_call_state_wipe (cs=cs@entry=0x0) at nfs3.c:559 #2 0x00007ff1c1998931 in nfs3_getattr (req=req@entry=0x7ff1bc0b26d0, fh=fh@entry=0x7ff1c2f76ae0) at nfs3.c:962 #3 0x00007ff1c1998c8a in nfs3svc_getattr (req=0x7ff1bc0b26d0) at nfs3.c:987 #4 0x00007ff1cfbfd8c5 in rpcsvc_handle_rpc_call (svc=0x7ff1bc03e500, trans=trans@entry=0x7ff1bc0c8020, msg=<optimized out>) at rpcsvc.c:695 #5 0x00007ff1cfbfdaab in rpcsvc_notify (trans=0x7ff1bc0c8020, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:789 #6 0x00007ff1cfbff9e3 in rpc_transport_notify (this=this@entry=0x7ff1bc0c8020, event=event@entry=RPC_TRANSPORT_MSG_RECEIVED, data=data@entry=0x7ff1bc0038d0) at rpc-transport.c:538 #7 0x00007ff1c4a2e3d6 in socket_event_poll_in (this=this@entry=0x7ff1bc0c8020, notify_handled=<optimized out>) at socket.c:2306 #8 0x00007ff1c4a3097c in socket_event_handler (fd=21, idx=9, gen=19, data=0x7ff1bc0c8020, poll_in=1, poll_out=0, poll_err=0) at socket.c:2458 #9 0x00007ff1cfe950f6 in event_dispatch_epoll_handler (event=0x7ff1c2f76e80, event_pool=0x5618154d5ee0) at event-epoll.c:572 #10 event_dispatch_epoll_worker (data=0x56181551cbd0) at event-epoll.c:648 #11 0x00007ff1cec99e25 in start_thread () from /lib64/libpthread.so.0 #12 0x00007ff1ce56634d in clone () from /lib64/libc.so.6 This patch add previous NULL check move from __nfs3_call_state_wipe() to nfs3_call_state_wipe() Cherry picked from commit 111d6bda9259126b0429113c9b8ba479958a4398: > Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea > BUG: 1479030 > Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> > Reviewed-on: https://review.gluster.org/17989 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Niels de Vos <ndevos@redhat.com> Change-Id: I2d73632f4be23f14d8467be3d908b09b3a2d87ea BUG: 1480594 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/18027 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* cluster/rebalance: Fix hardlink migration failuresSusant Palai2017-08-112-15/+82
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A brief about how hardlink migration works: - Different hardlinks (to the same file) may hash to different bricks, but their cached subvol will be same. Rebalance picks up the first hardlink, calculates it's hash(call it TARGET) and set the hashed subvolume as an xattr on the data file. - Now all the hardlinks those come after this will fetch that xattr and will create linkto files on TARGET (all linkto files for the hardlinks will be hardlink to each other on TARGET). - When number of hardlinks on source is equal to the number of hardlinks on TARGET, the data migration will happen. RACE:1 Since rebalance is multi-threaded, the first lookup (which decides where the TARGET subvol should be), can be called by two hardlink migration parallely and they may end up creating linkto files on two different TARGET subvols. Hence, hardlinks won't be migrated. Fix: Rely on the xattr response of lookup inside gf_defrag_handle_hardlink since it is executed under synclock. RACE:2 The linkto files on TARGET can be created by other clients also if they are doing lookup on the hardlinks. Consider a scenario where you have 100 hardlinks. When rebalance is migrating 99th hardlink, as a result of continuous lookups from other client, linkcount on TARGET is equal to source linkcount. Rebalance will migrate data on the 99th hardlink itself. On 100th hardlink migration, hardlink will have TARGET as cached subvolume. If it's hash is also the same, then a migration will be triggered from TARGET to TARGET leading to data loss. Fix: Make sure before the final data migration, source is not same as destination. RACE:3 Since a hardlink can be migrating to a non-hashed subvolume, a lookup from other client or even the rebalance it self, might delete the linkto file on TARGET leading to hardlinks never getting migrated. This will be addressed in a different patch in future. > Change-Id: If0f6852f0e662384ee3875a2ac9d19ac4a6cea98 > BUG: 1469964 > Signed-off-by: Susant Palai <spalai@redhat.com> > Reviewed-on: https://review.gluster.org/17755 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: N Balachandran <nbalacha@redhat.com> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Signed-off-by: Susant Palai <spalai@redhat.com> Change-Id: If0f6852f0e662384ee3875a2ac9d19ac4a6cea98 BUG: 1473141 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: https://review.gluster.org/17838 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* cluster/dht: fix on demand migration files from clientSusant Palai2017-08-114-20/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On demand migration of files i.e. migration done by clients triggered by a setfattr was broken. Dependency on defrag led to crash when migration was triggered from client. Note: This functionality is not available for tiered volumes. Migration from tier served client will fail with ENOTSUP. usage (But refer to the steps mentioned below to avoid any issues) : setfattr -n "trusted.distribute.migrate-data" -v "1" <filename> The purpose of fixing the on-demand client migration was to give a workaround where the user has lots of empty directories compared to files and want to do a remove-brick process. Here are the steps to trigger file migration for remove-brick process from client. (This is highly recommended to follow below steps as is) Let's say it is a replica volume and user want to remove a replica pair named brick1 and brick2. (Make sure healing is completed before you run these steps) Step-1: Start remove-brick process - gluster v remove-brick <volname> brick1 brick2 start Step-2: Kill the rebalance daemon - ps aux | grep glusterfs | grep rebalance\/ | awk '{print $2}' | xargs kill Step-3: Do a fresh mount as mentioned here - glusterfs -s ${localhostname} --volfile-id rebalance/$volume-name /tmp/mount/point Step-4: Go to one of the bricks (among brick1 and brick2) - cd <brick1 path> Step-5: Run the following command. - find . -not \( -path ./.glusterfs -prune \) -type f -not -perm 01000 -exec bash -c 'setfattr -n "distribute.fix.layout" -v "1" ${mountpoint}/$(dirname '{}')' \; -exec setfattr -n "trusted.distribute.migrate-data" -v "1" ${mountpoint}/'{}' \; This command will ignore the linkto files and empty directories. Do a fix-layout of the parent directory. And trigger a migration operation on the files. Step-6: Once this process is completed do "remove-brick force" - gluster v remove-brick <volname> brick1 brick2 force Note: Use the above script only when there are large number of empty directories. Since the script does a crawl on the brick side directly and avoids directories those are empty, the time spent on fixing layout on those directories are eliminated(even if the script does not do fix-layout on empty directories, post remove-brick a fresh layout will be built for the directory, hence not affecting application continuity). Detailing the expectation for hardlink migartion with this patch: Hardlink is migrated only for remove-brick process. It is highly essential to have a new mount(step-3) for the hardlink migration to happen. Why?: setfattr operation is an inode based operation. Since, we are doing setfattr from fuse mount here, inode_path will try to build path from the linked dentries to the inode. For a file without hardlinks the path construction will be correct. But for hardlinks, the inode will have multiple dentries linked. Without fresh mount, inode_path will always get the most recently linked dentry. e.g. if there are three hardlinks named dir1/link1, dir2/link2, dir3/link3, on a client where these hardlinks are looked up, inode_path will always return the path dir3/link3 if dir3/link3 was looked up most recently. Hence, we won't be able to create linkto files for all other hardlinks on destination (read gf_defrag_handle_hardlink for more details on hardlink migration). With a fresh mount, the lookup and setfattr become serialized. e.g. link2 won't be looked up until link1 is looked up and migrated. Hence, inode_path will always have the correct path, in this case link1 dentry is picked up(as this is the most recently looked up inode) and the path is built right. Note: If you run the above script on an existing mount(all entries looked up), hard links may not be migrated, but there should not be any other issue. Please raise a bug, if you find any issue. Tests: Manual > Change-Id: I9854cdd4955d9e24494f348fb29ba856ea7ac50a > BUG: 1450975 > Signed-off-by: Susant Palai <spalai@redhat.com> > Reviewed-on: https://review.gluster.org/17115 > NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> > Signed-off-by: Susant Palai <spalai@redhat.com> Change-Id: I9854cdd4955d9e24494f348fb29ba856ea7ac50a BUG: 1473140 Signed-off-by: Susant Palai <spalai@redhat.com> Reviewed-on: https://review.gluster.org/17837 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-16 17:27:36 +0000