summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* doc: Added release 6.6 notesv6.6hari gowtham2019-10-251-0/+56
| | | | | | | Fixes: bz#1762237 Change-Id: If339505ca0436947b814d9b0bc0b78f92d5d3317 Signed-off-by: hari gowtham <hgowtham@redhat.com>
* test: fix suspicous non-root geo-rep test failuresSunny Kumar2019-10-251-1/+1
| | | | | | | | | | | | | | | Export of env variable is required for ssh-copy-id command. Backport of: > fixes: bz#1765426 > Change-Id: Icaf7a848cb8f4ae9f887d885a8c5bb71f26633b4 > Signed-off-by: Sunny Kumar <sunkumar@redhat.com> fixes: bz#1765433 Change-Id: Icaf7a848cb8f4ae9f887d885a8c5bb71f26633b4 Signed-off-by: Sunny Kumar <sunkumar@redhat.com> (cherry picked from commit bfebfa9f2ec9dfc5dbf4a68c3518f98364ebc461)
* geo-rep: Fix Permission denied traceback on non root setupKotresh HR2019-10-243-16/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: While syncing rename of directory in hybrid crawl, geo-rep crashes as below. Traceback (most recent call last): File "/usr/local/libexec/glusterfs/python/syncdaemon/repce.py", line 118, in worker res = getattr(self.obj, rmeth)(*in_data[2:]) File "/usr/local/libexec/glusterfs/python/syncdaemon/resource.py", line 588, in entry_ops src_entry = get_slv_dir_path(slv_host, slv_volume, gfid) File "/usr/local/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 687, in get_slv_dir_path [ENOENT], [ESTALE]) File "/usr/local/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 546, in errno_wrap return call(*arg) PermissionError: [Errno 13] Permission denied: '/bricks/brick1/b1/.glusterfs/8e/c0/8ec0fcd4-d50f-4a6e-b473-a7943ab66640' Cause: Conversion of gfid to path for a directory uses readlink on backend .glusterfs gfid path. But this fails for non root user with permission denied. Fix: Use gfid2path interface to get the path from gfid Backport of: > Patch: https://review.gluster.org/23570 > Change-Id: I9d40c713a1b32cea95144cbc0f384ada82972222 > BUG: 1763439 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: I9d40c713a1b32cea95144cbc0f384ada82972222 fixes: bz#1764183 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* cluster/ec: fix EIO error for concurrent writes on sparse filesXavi Hernandez2019-10-242-10/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | EC doesn't allow concurrent writes on overlapping areas, they are serialized. However non-overlapping writes are serviced in parallel. When a write is not aligned, EC first needs to read the entire chunk from disk, apply the modified fragment and write it again. The problem appears on sparse files because a write to an offset implicitly creates data on offsets below it (so, in some way, they are overlapping). For example, if a file is empty and we read 10 bytes from offset 10, read() will return 0 bytes. Now, if we write one byte at offset 1M and retry the same read, the system call will return 10 bytes (all containing 0's). So if we have two writes, the first one at offset 10 and the second one at offset 1M, EC will send both in parallel because they do not overlap. However, the first one will try to read missing data from the first chunk (i.e. offsets 0 to 9) to recombine the entire chunk and do the final write. This read will happen in parallel with the write to 1M. What could happen is that half of the bricks process the write before the read, and the half do the read before the write. Some bricks will return 10 bytes of data while the otherw will return 0 bytes (because the file on the brick has not been expanded yet). When EC tries to recombine the answers from the bricks, it can't, because it needs more than half consistent answers to recover the data. So this read fails with EIO error. This error is propagated to the parent write, which is aborted and EIO is returned to the application. The issue happened because EC assumed that a write to a given offset implies that offsets below it exist. This fix prevents the read of the chunk from bricks if the current size of the file is smaller than the read chunk offset. This size is correctly tracked, so this fixes the issue. Also modifying ec-stripe.t file for Test #13 within it. In this patch, if a file size is less than the offset we are writing, we fill zeros in head and tail and do not consider it strip cache miss. That actually make sense as we know what data that part holds and there is no need of reading it from bricks. Backport of: > Patch:https://review.gluster.org/#/c/glusterfs/+/23066/ > Change-Id: Ic342e8c35c555b8534109e9314c9a0710b6225d6 > BUG: 1730715 > Signed-off-by: Xavi Hernandez <xhernandez@redhat.com> (cherry picked from commit b01a43586c5abc23a874e5528a063c508f952cbd) Change-Id: Ic342e8c35c555b8534109e9314c9a0710b6225d6 Fixes: bz#1739451 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* features/shard: Send correct size when reads are sent beyond file sizeKrutika Dhananjay2019-10-242-0/+31
| | | | | | | Change-Id: I0cebaaf55c09eb1fb77a274268ff564e871b743b fixes bz#1737141 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> (cherry picked from commit 51237eda7c4b3846d08c5d24d1e3fe9b7ffba1d4)
* geo-rep: Fix config upgrade on non-participating nodeKotresh HR2019-10-223-1/+181
| | | | | | | | | | | | | | | | | | | After upgrade, if the config files are of old format, it gets migrated to new format. Monitor process migrates it. Since monitor doesn't run on nodes where bricks are not hosted, it doesn't get migrated there. So this patch fixes the config upgrade on nodes which doesn't host bricks. This happens during config either on get/set/reset. Backport of: > Patch: https://review.gluster.org/23555 > Change-Id: Ibade2f2310b0f3affea21a3baa1ae0eb71162cba > Signed-off-by: Kotresh HR <khiremat@redhat.com> > BUG: 1762220 Change-Id: Ibade2f2310b0f3affea21a3baa1ae0eb71162cba fixes: bz#1763028 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* tests : test case for non-root geo-rep setupSunny Kumar2019-10-221-0/+251
| | | | | | | | | | | | | | | Added test case for non-root geo-rep setup. Backport of: > Patch: https://review.gluster.org/22902 > Change-Id: Ib6ebee79949a9f61bdc5c7b5e11b51b262750e98 > BUG: 1717827 > Signed-off-by: Sunny Kumar <sunkumar@redhat.com> > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: Ib6ebee79949a9f61bdc5c7b5e11b51b262750e98 fixes: bz#1764178 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix the name of changelog archive fileKotresh HR2019-10-222-8/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Background: The processed changelogs are archived each month in a single tar file. The default format is "archive_YYYYMM.tar" which is specified as "%%Y%%m" in configuration file. Problem: The created changelog archive file didn't have corresponding year and month. It created as "archive_%Y%m.tar" on python2 only systems. Cause and Fix: Geo-rep expects "%Y%m" after the ConfigParser reads it from config file. Since it was "%%Y%%m" in config file, geo-rep used to get correct value "%Y%m" in python3 and "%%Y%%m" in python2 which is incorrect. The fix can be to use "%Y%m" in config file but that fails in python3. So the fix is to use "RawConfigParser" in geo-rep and use "%Y%m". This works both in python2 and python3. Backport of: > Patch: https://review.gluster.org/23248 > Change-Id: Ie5b7d2bc04d0d53cd1769e064c2d67aaf95d557c > BUG: 1741890 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: Ie5b7d2bc04d0d53cd1769e064c2d67aaf95d557c fixes: bz#1764176 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix Config Get RaceAravinda VK2019-10-221-6/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When two threads(sync jobs) in Geo-rep worker calls `gconf.get` and `gconf.getr`(realtime) at the sametime, `getr` resets the conf object and other one gets None. Thread Lock is introduced to fix the issue. ``` File "/usr/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 368, in twrap tf(*aargs) File "/usr/libexec/glusterfs/python/syncdaemon/master.py", line 1987, in syncjob po = self.sync_engine(pb, self.log_err) File "/usr/libexec/glusterfs/python/syncdaemon/resource.py", line 1444, in rsync rconf.ssh_ctl_args + \ AttributeError: 'NoneType' object has no attribute 'split' ``` Backport of: > Patch: https://review.gluster.org/23158 > Change-Id: I9c245e5c36338265354e158f5baa32b119eb2da5 > BUG: 1737484 > Signed-off-by: Aravinda VK <avishwan@redhat.com> Change-Id: I9c245e5c36338265354e158f5baa32b119eb2da5 fixes: bz#1764174 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Test case for upgrading config fileShwetha K Acharya2019-10-222-6/+18
| | | | | | | | | | | | | | | | | | Added test case for the patch https://review.gluster.org/#/c/glusterfs/+/22894/4 Also updated if else structure in gsyncdconfig.py to avoid repeated occurance of values in new configfile. Backport of: > Patch: https://review.gluster.org/22982 > BUG: 1707731 > Change-Id: If97e1d37ac52dbd17d47be6cb659fc5a3ccab6d7 > Signed-off-by: Shwetha K Acharya <sacharya@redhat.com> fixes: bz#1764171 Change-Id: If97e1d37ac52dbd17d47be6cb659fc5a3ccab6d7 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep : fix gluster command path for non-root sessionSunny Kumar2019-10-222-5/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | Problem: gluster command not found. Cause: In Volinfo class we issue command 'gluster vol info' to get information about volume like getting brick_root to perform various operation. When geo-rep session is configured for non-root user Volinfo class fails to issue gluster command due to unavailability of gluster binary path for non-root user. Solution: Use config value 'slave-gluster-command-dir'/'gluster-command-dir' to get path for gluster command based on caller. Backport of: > Patch: https://review.gluster.org/22920 > BUG: 1722740 > Change-Id: I4ec46373da01f5d00ecd160c4e8c6239da8b3859 > Signed-off-by: Sunny Kumar <sunkumar@redhat.com> fixes: bz#1764172 Change-Id: I4ec46373da01f5d00ecd160c4e8c6239da8b3859 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Upgrading config file to new versionShwetha K Acharya2019-10-222-0/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | - configuration handling is enhanced with patch https://review.gluster.org/#/c/glusterfs/+/18257/ - hence, the old configurations are not applied when Geo-rep session is created in the old version and upgraded. This patch solves the issue. It, - checks if the config file is old. - parses required values from old config file and stores in new config file, which ensures that configerations are applied on upgrade. - stores old config file as backup. - handles changes in options introduced in https://review.gluster.org/#/c/glusterfs/+/18257/ Backport of: > Patch: https://review.gluster.org/22894 > BUG: bz#1707731 > Change-Id: Iad8da6c1e1ae8ecf7c84dfdf8ea3ac6966d8a2a0 > Signed-off-by: Shwetha K Acharya <sacharya@redhat.com> updates: bz#1764171 Change-Id: Iad8da6c1e1ae8ecf7c84dfdf8ea3ac6966d8a2a0 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* dht: Rebalance causing IO Error - File descriptor in bad stateMohit Agrawal2019-10-175-17/+116
| | | | | | | | | | | | | | | | | | | | | Problem : When a file is migrated, dht attempts to re-open all open fds on the new cached subvol. Earlier, if dht had not opened the fd, the client xlator would be unable to find the remote fd and would fall back to using an anon fd for the fop. That behavior changed with https://review.gluster.org/#/c/glusterfs/+/15804, causing fops to fail with EBADFD if the fd was not available on the cached subvol. The client xlator returns EBADFD if the remote fd is not found but dht only checks for EBADF before re-opening fds on the new cached subvol. Solution: Handle EBADFD at dht code path to avoid the issue >Change-Id: I43c51995cdd48d05b12e4b2889c8dbe2bb2a72d8 >Fixes: bz#1758579 >(cherry picked from commit 9314a9fbf487614c736cf6c4c1b93078d37bb9df) >(Reviewed on upstream https://review.gluster.org/#/c/glusterfs/+/23518/) Change-Id: I43c51995cdd48d05b12e4b2889c8dbe2bb2a72d8 Fixes: bz#1761907
* cluster/afr: Heal entries when there is a source & no healed_sinkskarthik-us2019-10-172-0/+104
| | | | | | | | | | | | | | | | | | | | Problem: In a situation where B1 blames B2, B2 blames B1 and B3 doesn't blame anything for entry heal, heal will not complete even though we have clear source and sinks. This will happen because while doing afr_selfheal_find_direction() only the bricks which are blamed by non-accused bricks are considered as sinks. Later in __afr_selfheal_entry_finalize_source() when it tries to mark all the non-sources as sinks it fails to do so because there won't be any healed_sinks marked, no witness present and there will be a source. Fix: If there is a source and no healed_sinks, then reset all the locked sources to 0 and healed sinks to 1 to do conservative merge. Change-Id: If40d8bc95d52a52b2730f55bdcf135109b421548 Fixes: bz#1760706 Signed-off-by: karthik-us <ksubrahm@redhat.com>
* afr: support split-brain CLI for replica 3Ravishankar N2019-10-173-2/+115
| | | | | | | | | | | | | | | | | | | | | | | | | Ever since we added quorum checks for lookups in afr via commit bd44d59741bb8c0f5d7a62c5b1094179dd0ce8a4, the split-brain resolution commands would not work for replica 3 because there would be no readables for the lookup fop. The argument was that split-brains do not occur in replica 3 but we do see (data/metadata) split-brain cases once in a while which indicate that there are a few bugs/corner cases yet to be discovered and fixed. Fortunately, commit 8016d51a3bbd410b0b927ed66be50a09574b7982 added GF_CLIENT_PID_GLFS_HEALD as the pid for all fops made by glfsheal. If we leverage this and allow lookups in afr when pid is GF_CLIENT_PID_GLFS_HEALD, split-brain resolution commands will work for replica 3 volumes too. Likewise, the check is added in shard_lookup as well to permit resolving split-brains by specifying "/.shard/shard-file.xx" as the file name (which previously used to fail with EPERM). Change-Id: I3c543dea79caf7cfbc1633e9089cb1cdd2538ba9 Fixes: bz#1760792 Signed-off-by: Ravishankar N <ravishankar@redhat.com> (cherry picked from commit 47dbd753187f69b3835d2e42fdbe7485874c4b3e)
* system/posix-acl: update ctx only if iatt is non-NULLHomma2019-10-151-0/+8
| | | | | | | | | | | | | | We need to safe-guard against possible zero'ing out of iatt structure in acl ctx, which can cause many issues. > fixes: bz#1668286 > Change-Id: Ie81a57d7453a6624078de3be8c0845bf4d432773 > Signed-off-by: Amar Tumballi <amarts@redhat.com> > (cherry picked from commit 6bf9637a93011298d032332ca93009ba4e377e46) Change-Id: I992d25f1c1282d50aa0232d01586d2df2216551c fixes: bz#1741402 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* packaging: remove leftover bd cruft in rpm .specKaleb S. KEITHLEY2019-10-101-9/+3
| | | | | | | | $subject Change-Id: Ifd8e3340ae6868ea1fd2d0b53f02b093cc23f2cb Fixes: bz#1760361 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* perf/write-behind: Clear frame->local on conflict errorN Balachandran2019-10-041-0/+4
| | | | | | | | | | | | | | | WB saves the wb_inode in frame->local for the truncate and ftruncate fops. This value is not cleared in case of error on a conflicting write request. FRAME_DESTROY finds a non-null frame->local and tries to free it using mem_put. However, wb_inode is allocated using GF_CALLOC, causing the process to crash. credit: vpolakis@gmail.com Change-Id: I217f61470445775e05145aebe44c814731c1b8c5 fixes: bz#1755679 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* fuse: add missing GF_FREE to fuse_interruptCsaba Henk2019-09-271-1/+4
| | | | | | Change-Id: Id7e003e4a53d0a0057c1c84e1cd704c80a6cb015 fixes: bz#1753571 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* ctime/rebalance: Heal ctime xattr on directory during rebalanceKotresh HR2019-09-2717-53/+629
| | | | | | | | | | | | | | | | | | | | | | | After add-brick and rebalance, the ctime xattr is not present on rebalanced directories on new brick. This patch fixes the same. Note that ctime still doesn't support consistent time across distribute sub-volume. This patch also fixes the in-memory inconsistency of time attributes when metadata is self healed. Backport of: > Patch: https://review.gluster.org/23127 > Change-Id: Ia20506f1839021bf61d4753191e7dc34b31bb2df > BUG: 1734026 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Patch: https://review.gluster.org/23127 Change-Id: Ia20506f1839021bf61d4753191e7dc34b31bb2df fixes: bz#1752413 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep : fix mountbroker setupSunny Kumar2019-09-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Unable to setup mountbroker root directory while creating geo-replication session for non-root user. Casue: With patch[1] which defines the max-port for glusterd one extra sapce got added in field of 'option max-port'. [1]. https://review.gluster.org/#/c/glusterfs/+/21872/ In geo-rep spliting of key-value pair form vol file was done on the basis of space so this additional space caused "ValueError: too many values to unpack". Solution: Use split so that it can treat consecutive whitespace as a single separator. Backport of https://review.gluster.org/#/c/glusterfs/+/22716/ >Fixes: bz#1709248 >Change-Id: Ia22070a43f95d66d84cb35487f23f9ee58b68c73 >Signed-off-by: Sunny Kumar <sunkumar@redhat.com> >(cherry picked from commit 3dd03146bb7037ae2ebea0579d0b81be27fdd927) Fixes: bz#1750228 Signed-off-by: Sunny Kumar <sunkumar@redhat.com> Change-Id: Ia22070a43f95d66d84cb35487f23f9ee58b68c73
* gfapi: 'glfs_h_creat_open' - new API to create handle and open fdSoumya Koduri2019-09-276-0/+291
| | | | | | | | | | | | | | | | | | | | | Right now we have two separate APIs, one - 'glfs_h_creat_handle' to create handle & another - 'glfs_h_open' to create a glfd to return to application Having two separate routines can result in access errors while trying to create and write into a read-only file. Since a fd is opened even during file/directory creation, introducing a new API to make these two operations atomic i.e, which can create both handle & fd and pass them to application This is backport of below mainline patch - - https://review.gluster.org/#/c/glusterfs/+/23448/ - bz#1753569 Change-Id: Ibf513fcfcdad175f4d7eb6fa7a61b8feec6d33b5 fixes: bz#1755785 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* dht: Custom xattrs are not healed in case of add-brickroot2019-09-272-8/+68
| | | | | | | | | | | | | | | | | | | | | | Problem: If any custom xattrs are set on the directory before add a brick, xattrs are not healed on the directory after adding a brick. Solution: xattr are not healed because dht_selfheal_dir_mkdir_lookup_cbk checks the value of MDS and if MDS value is not negative selfheal code path does not take reference of MDS xattrs.Change the condition to take reference of MDS xattr so that custom xattrs are populated on newly added brick Backport of: > Patch: https://review.gluster.org/22520 > BUG: bz#1702299 > Change-Id: Id14beedb98cce6928055f294e1594b22132e811c > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> fixes: bz#1753561 Change-Id: Id14beedb98cce6928055f294e1594b22132e811c Signed-off-by: Kotresh HR <khiremat@redhat.com>
* storage/posix: set the op_errno to proper errno during gfid setRaghavendra Bhat2019-09-231-0/+1
| | | | | | | | | | In posix_gfid_set, the proper error is not captured in one of the failure cases. Change-Id: I1c13f0691a15d6893f1037b3a5fe385a99657e00 Fixes: bz#1746118 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com> (cherry picked from commit ed7a3793073670e787063c47e55010fc7c963064)
* core/syncop: Bail out if frame creation failsSoumya Koduri2019-09-231-0/+6
| | | | | | | | | | | | | | There could be cases (either due to insufficient memory or corrupted mem-pool) due to which frame creation fails. Bail out with error in such cases. This is the backport of below mainline fix - > Fixes: bz#1748448 > review url: https://review.gluster.org/#/c/glusterfs/+/23350/ Change-Id: I8cc0a5852f6f04d2bac991e4eb79ecb42577da11 Fixes: bz#1751557 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* afr/lookup: Pass xattr_req in while doing a selfheal in lookupMohammed Rafi KC2019-09-235-5/+69
| | | | | | | | | | | | | | | | | We were not passing xattr_req when doing a name self heal as well as a meta data heal. Because of this, some xdata was missing which causes i/o errors Backport of > https://review.gluster.org/#/c/glusterfs/+/23024/ >Change-Id: Ibfb1205a7eb0195632dc3820116ffbbb8043545f >Fixes: bz#1728770 >Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Fixes: bz#1749307 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> (cherry picked from commit d026f0bcfd301712e4f0671ccf238f43f2e6dd30) Change-Id: Ibfb1205a7eb0195632dc3820116ffbbb8043545f
* rpc: Update address family if it is not provide in cmd-line argumentsMohit Agrawal2019-09-091-1/+12
| | | | | | | | | | | | | | | | | | | | Problem: After enabling transport-type to inet6 and passed ipv6 transport.socket.bind-address in glusterd.vol clients are not started. Solution: Need to update address-family based on remote-address for all gluster client process > Change-Id: Iaa3588cd87cebc45231bfd675745c1a457dc9b31 > Fixes: bz#1747746 > Credits: Amgad Saleh <amgad.saleh@nokia.com> > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> > (cherry picked from commit 80b8cfe3f1386606bada97a76a0cad7acdf6b877) Change-Id: Iaa3588cd87cebc45231bfd675745c1a457dc9b31 Fixes: bz#1739320 Credits: Amgad Saleh <amgad.saleh@nokia.com> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* glusterd: IPV6 hostname address is not parsed correctlyMohit Agrawal2019-09-091-5/+11
| | | | | | | | | | | | | | | | | | Problem: IPV6 hostname address is not parsed correctly in function glusterd_check_brick_order Solution: Update the code to parse hostname address > Change-Id: Ifb2f83f9c6e987b2292070e048e97eeb51b728ab > Fixes: bz#1747746 > Credits: Amgad Saleh <amgad.saleh@nokia.com> > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> > (cherry picked from commit 6563ffb04d7ba51a89726e7c5bbb85c7dbc685b5) Change-Id: Ifb2f83f9c6e987b2292070e048e97eeb51b728ab Fixes: bz#1739320 Credits: Amgad Saleh <amgad.saleh@nokia.com> Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* afr: wake up index healer threadsRavishankar N2019-09-056-11/+66
| | | | | | | | | | | | | | | (Backport of https://review.gluster.org/#/c/glusterfs/+/23288/) ...whenever shd is re-enabled after disabling or there is a change in `cluster.heal-timeout`, without needing to restart shd or waiting for the current `cluster.heal-timeout` seconds to expire. See BZ 1743988 for more details. Change-Id: Ia5ebd7c8e9f5b54cba3199c141fdd1af2f9b9bfe fixes: bz#1743988 Reported-by: Glen Kiessling <glenk1973@hotmail.com> Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* tests: fix spurious failure of bug-1402841.t-mt-dir-scan-race.tRavishankar N2019-09-051-4/+5
| | | | | | | | | | | | | | | | | | | | | | | Problem: Since commit 600ba94183333c4af9b4a09616690994fd528478, shd starts healing as soon as it is toggled from disabled to enabled. This was causing the following line in the .t to fail on a 'fast' machine (always on my laptop and sometimes on the jenkins slaves). EXPECT_NOT "^0$" get_pending_heal_count $V0 because by the time shd was disabled, the heal was already completed. Fix: Increase the no. of files to be healed and make it a variable called FILE_COUNT, should we need to bump it up further because the machines become even faster. Also created pending metadata heals to increase the time taken to heal a file. fixes: bz#1749157 Change-Id: I5a26b08e45b8c19bce3c01ce67bdcc28ed48198d Signed-off-by: Ravishankar N <ravishankar@redhat.com> (cherry picked from commit 724c657995a2e148243eeb78c68b620c6d7714a5)
* ctime: Fix ctime issue with utime family of syscallsKotresh HR2019-08-304-52/+68
| | | | | | | | | | | | | | | | | | When atime|mtime is updated via utime family of syscalls, ctime is not updated. This patch fixes the same. Backport of: > Patch: https://review.gluster.org/23177/ > Change-Id: I7f86d8f8a1e06a332c3449b5bbdbf128c9690f25 > BUg: 1738786 > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 95f71df31dc73d85df722b0e7d3a7eb1e0237e7f) Change-Id: I7f86d8f8a1e06a332c3449b5bbdbf128c9690f25 Signed-off-by: Kotresh HR <khiremat@redhat.com> fixes: bz#1746138
* event: rename event_XXX with gf_ prefixedXiubo Li2019-08-2815-90/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I hit one crash issue when using the libgfapi. In the libgfapi it will call glfs_poller() --> event_dispatch() in file api/src/glfs.c:721, and the event_dispatch() is defined by libgluster locally, the problem is the name of event_dispatch() is the extremly the same with the one from libevent package form the OS. For example, if a executable program Foo, which will also use and link the libevent and the libgfapi at the same time, I can hit the crash, like: kernel: glfs_glfspoll[68486]: segfault at 1c0 ip 00007fef006fd2b8 sp 00007feeeaffce30 error 4 in libevent-2.0.so.5.1.9[7fef006ed000+46000] The link for Foo is: lib_foo_LADD = -levent $(GFAPI_LIBS) It will crash. This is because the glfs_poller() is calling the event_dispatch() from the libevent, not the libglsuter. The gfapi link info : GFAPI_LIBS = -lacl -lgfapi -lglusterfs -lgfrpc -lgfxdr -luuid If I link Foo like: lib_foo_LADD = $(GFAPI_LIBS) -levent It will works well without any problem. And if Foo call one private lib, such as handler_glfs.so, and the handler_glfs.so will link the GFAPI_LIBS directly, while the Foo won't and it will dlopen(handler_glfs.so), then the crash will be hit everytime. The link info will be: foo_LADD = -levent libhandler_glfs_LIBADD = $(GFAPI_LIBS) I can avoid the crash temporarily by linking the GFAPI_LIBS in Foo too like: foo_LADD = $(GFAPI_LIBS) -levent libhandler_glfs_LIBADD = $(GFAPI_LIBS) But this is ugly since the Foo won't use any APIs from the GFAPI_LIBS. And in some cases when the --as-needed link option is added(on many dists it is added as default), then the crash is back again, the above workaround won't work. Backport of: > https://review.gluster.org/#/c/glusterfs/+/23110/ > Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa > Fixes: #699 > Signed-off-by: Xiubo Li <xiubli@redhat.com> Change-Id: I38f0200b941bd1cff4bf3066fca2fc1f9a5263aa updates: bz#1740525 Signed-off-by: Xiubo Li <xiubli@redhat.com> (cherry picked from commit 799edc73c3d4f694c365c6a7c27c9ab8eed5f260)
* ctime: Fix incorrect realtime passed to frame->root->ctimeKotresh HR2019-08-284-1/+26
| | | | | | | | | | | | | | | | | | | | On systems that don't support "timespec_get"(e.g., centos6), it was using "clock_gettime" with "CLOCK_MONOTONIC" to get unix epoch time which is incorrect. This patch introduces "timespec_now_realtime" which uses "clock_gettime" with "CLOCK_REALTIME" which fixes the issue. Backport of: > Patch: https://review.gluster.org/23274/ > Change-Id: I57be35ce442d7e05319e82112b687eb4f28d7612 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > BUG: 1743652 (cherry picked from commit d14d0749340d9cb1ef6fc4b35f2fb3015ed0339d) Change-Id: I57be35ce442d7e05319e82112b687eb4f28d7612 Signed-off-by: Kotresh HR <khiremat@redhat.com> fixes: bz#1726175
* performance/md-cache: Do not skip caching of null character xattr valuesAnoop C S2019-08-283-20/+45
| | | | | | | | | | | | | | | | | | | | | | Null character string is a valid xattr value in file system. But for those xattrs processed by md-cache, it does not update its entries if value is null('\0'). This results in ENODATA when those xattrs are queried afterwards via getxattr() causing failures in basic operations like create, copy etc in a specially configured Samba setup for Mac OS clients. On the other side snapview-server is internally setting empty string("") as value for xattrs received as part of listxattr() and are not intended to be cached. Therefore we try to maintain that behaviour using an additional dictionary key to prevent updation of entries in getxattr() and fgetxattr() callbacks in md-cache. Credits: Poornima G <pgurusid@redhat.com> Change-Id: I7859cbad0a06ca6d788420c2a495e658699c6ff7 Fixes: bz#1743782 Signed-off-by: Anoop C S <anoopcs@redhat.com> (cherry picked from commit b4b683736367d93daad08a5ee6ca95778c07c5a4)
* afr: restore timestamp of parent dir during entry-healRavishankar N2019-08-262-0/+80
| | | | | | | Fixes: bz#1741044 Change-Id: I29e338bac62104233a6f80212df8d0fb016affda Signed-off-by: Ravishankar N <ravishankar@redhat.com> (cherry picked from commit 8e9c53ebf16705b9a1db2fc486dc24a5cb244ddd)
* glusterd: ./tests/bugs/glusterd/bug-1595320.t is failingMohit Agrawal2019-08-261-1/+1
| | | | | | | | | | | | | | | | | | Problem: sometime ./tests/bugs/glusterd/bug-1595320.t is failing is failing at the time of checking brick_process after sending a kill signal to brick process Solution: Wait sometime after just sending a kill signal to brick process to make sure brick process is stopped > Change-Id: Iee9e91284618abfc62a550d47e4f9117785def58 > Fixes: bz#1743200 > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> > (cherry picked from commit 8f1620ad7f5d3d040fee55c5f873349800e2268d) Change-Id: Iee9e91284618abfc62a550d47e4f9117785def58 Fixes: bz#1745421 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* rpc: glusterd start is failed and throwing an error Address already in useMohit Agrawal2019-08-191-7/+37
| | | | | | | | | | | | | | | | | | | | Problem: Some of the .t are failed due to bind is throwing an error EADDRINUSE Solution: After killing all gluster processes .t is trying to start glusterd but somehow if kernel has not cleaned up resources(socket) then glusterd startup is failed due to bind system call failure.To avoid the issue retries to call bind 10 times to execute system call succesfully > Change-Id: Ia5fd6b788f7b211c1508c1b7304fc08a32266629 > Fixes: bz#1743020 > Signed-off-by: Mohit Agrawal <moagrawal@redhat.com> > (cherry picked from commit c370c70f77079339e2cfb7f284f3a2fb13fd2f97) Change-Id: Ia5fd6b788f7b211c1508c1b7304fc08a32266629 Fixes: bz#1743219 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* protocol/client: propagte GF_EVENT_CHILD_PING only for connections to brickRaghavendra G2019-08-091-4/+12
| | | | | | | | | | | | | | | | | | | | | | | | Two reasons: * ping responses from glusterd may not be relevant for Halo replication. Instead, it might be interested in only knowing whether the brick itself is responsive. * When a brick is killed, propagating GF_EVENT_CHILD_PING of ping response from glusterd results in GF_EVENT_DISCONNECT spuriously propagated to parent xlators. These DISCONNECT events are from the connections client establishes with glusterd as part of its reconnect logic. Without GF_EVENT_CHILD_PING, the last event propagated to parent xlators would be the first DISCONNECT event from brick and hence subsequent DISCONNECTS to glusterd are not propagated as protocol/client prevents same event being propagated to parent xlators consecutively. propagating GF_EVENT_CHILD_PING for ping responses from glusterd would change the last_sent_event to GF_EVENT_CHILD_PING and hence protocol/client cannot prevent subsequent DISCONNECT events Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Fixes: bz#1739335 Change-Id: I50276680c52f05ca9e12149a3094923622d6eaef (cherry picked from commit 5d66eafec581fb3209af74595784be8854ca40a4)
* doc: Added release 6.5 notesv6.5Hari Gowtham2019-08-071-0/+28
| | | | | | | Fixes: bz#1737311 Change-Id: I1f54bc9b04ee05c8ab802d92c1471cbfb1aa92a8 Signed-off-by: Hari Gowtham <hgowtham@redhat.com>
* features/utime: always update ctime at setattrKinglong Mee2019-08-072-13/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | For the nfs EXCLUSIVE mode create may sets a later time to mtime (at verifier), it should not set to ctime for storage.ctime does not allowed set ctime to a earlier time. /* Earlier, mdata was updated only if the existing time is less * than the time to be updated. This would fail the scenarios * where mtime can be set to any time using the syscall. Hence * just updating without comparison. But the ctime is not * allowed to changed to older date. */ According to kernel's setattr, always set ctime at setattr, and doesnot set ctime from mtime at storage.ctime. Backport of: > Patch: https://review.gluster.org/23154 > Change-Id: I5cfde6cb7f8939da9617506e3dc80bd840e0d749 > BUG: 1737288 > Signed-off-by: Kinglong Mee <kinglongmee@gmail.com> Change-Id: I5cfde6cb7f8939da9617506e3dc80bd840e0d749 fixes: bz#1737746 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* posix/ctime: Fix race during lookup ctime xattr healKotresh HR2019-08-071-18/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: Ctime heals the ctime xattr ("trusted.glusterfs.mdata") in lookup if it's not present. In a multi client scenario, there is a race which results in updating the ctime xattr to older value. e.g. Let c1 and c2 be two clients and file1 be the file which doesn't have the ctime xattr. Let the ctime of file1 be t1. (from backend, ctime heals time attributes from backend when not present). Now following operations are done on mount c1 -> ls -l /mnt/file1 | c2 -> ls -l /mnt/file1;echo "append" >> /mnt/file1; The race is that the both c1 and c2 didn't fetch the ctime xattr in lookup, so both of them tries to heal ctime to time 't1'. If c2 wins the race and appends the file before c1 heals it, it sets the time to 't1' and updates it to 't2' (because of append). Now c1 proceeds to heal and sets it to 't1' which is incorrect. Solution: Compare the times during heal and only update the larger time. This is the general approach used in ctime feature but got missed with healing legacy files. Backport of: > Patch: https://review.gluster.org/23131 > BUG: 1734299 > Change-Id: I930bda192c64c3d49d0aed431ce23d3bc57e51b7 > Signed-off-by: Kotresh HR <khiremat@redhat.com> fixes: bz#1737745 Change-Id: I930bda192c64c3d49d0aed431ce23d3bc57e51b7 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix mount broker setup issueKotresh HR2019-08-071-3/+6
| | | | | | | | | | | | | | | | | | | | | | Even the use builtin 'type' command as in patch [1] causes issues if argument in question is not part of PATH environment variable for that user. This patch fixes the same by doing source /etc/profile. This was already being used in another part of script. [1] https://review.gluster.org/23089 Backport of: > Patch: https://review.gluster.org/23136/ > Change-Id: Iceb78835967ec6a4350983eec9af28398410c002 > BUG: 1734738 > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 84f7794547522463841068063b22fd3a8d8fca2b) Change-Id: Iceb78835967ec6a4350983eec9af28398410c002 fixes: bz#1737712 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* features/utime: Fix mem_put crashPranith Kumar K2019-08-061-1/+3
| | | | | | | | | | | | | | | | | | | | Problem: When frame->local is not null FRAME_DESTROY calls mem_put on it. Since the stub is already destroyed in call_resume(), it leads to crash Fix: Set frame->local to NULL before calling call_resume() Backport of: > Patch: https://review.gluster.org/23091 > BUG: 1593542 > Change-Id: I0f8adf406f4cefdb89d7624ba7a9d9c2eedfb1de > Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> fixes: bz#1733885 Change-Id: I0f8adf406f4cefdb89d7624ba7a9d9c2eedfb1de Signed-off-by: Kotresh HR <khiremat@redhat.com>
* ctime: Set mdata xattr on legacy filesKotresh HR2019-08-0616-57/+475
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: The files which were created before ctime enabled would not have "trusted.glusterfs.mdata"(stores time attributes) xattr. Upon fops which modifies either ctime or mtime, the xattr gets created with latest ctime, mtime and atime, which is incorrect. It should update only the corresponding time attribute and rest from backend Solution: Creating xattr with values from brick is not possible as each brick of replica set would have different times. So create the xattr upon successful lookup if the xattr is not created Note To Reviewers: The time attributes used to set xattr is got from successful lookup. Instead of sending the whole iatt over the wire via setxattr, a structure called mdata_iatt is sent. The mdata_iatt contains only time attributes. Backport of: > Patch: https://review.gluster.org/22936 > Change-Id: I5e535631ddef04195361ae0364336410a2895dd4 > BUG: 1593542 > Signed-off-by: Kotresh HR <khiremat@redhat.com> Change-Id: I5e535631ddef04195361ae0364336410a2895dd4 updates: bz#1733885 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* geo-rep: Fix mount broker setup issueKotresh HR2019-08-051-3/+3
| | | | | | | | | | | | | | | | | | | | | The patch [1] added validation in gverify.sh to check if the gluster binary exists on slave by executing gluster directly on slave. But for non-root users, even though gluster binary is present, path is not found when executed via ssh. Hence validate the gluster binary using bash builtin 'type' command. [1] https://review.gluster.org/19224 Backport of: > Patch: https://review.gluster.org/23089/ > Change-Id: I93ca62c0c5b1e16263e586ddbbca8407d60ca126 > BUG: 1731920 > Signed-off-by: Kotresh HR <khiremat@redhat.com> (cherry picked from commit 2aa731a259ea457c07494e3c3edf6d5f7c02fe77) Change-Id: I93ca62c0c5b1e16263e586ddbbca8407d60ca126 fixes: bz#1733880 Signed-off-by: Kotresh HR <khiremat@redhat.com>
* gfapi: Fix deadlock while processing upcallSoumya Koduri2019-08-021-33/+131
| | | | | | | | | | | | | | | | | | | As mentioned in bug1733166, there could be potential deadlock while processing upcalls depending on how each xlator choose to act on it. The right way of fixing such issues is to change rpc callback communication process. - https://github.com/gluster/glusterfs/issues/697 Till then, making changes in gfapi layer to avoid any I/O processing. This is backport of below mainline patch > https://review.gluster.org/#/c/glusterfs/+/23108/ > bz#1733166 Change-Id: I2079e95339e5d761d5060707f4555cfacab95c83 fixes: bz#1736341 Signed-off-by: Soumya Koduri <skoduri@redhat.com>
* features/snapview-server: obtain the list of snapshots inside the lockRaghavendra Bhat2019-08-011-1/+1
| | | | | | | | | The current list of snapshots from priv->dirents is obtained outside the lock. Change-Id: I8876ec0a38308da5db058397382fbc82cc7ac177 Fixes: bz#1731509 (cherry picked from commit 8e795617fd6f5193d0d52a336059ce1a28108c0e)
* cluster/dht: Fix directory perms during selfhealN Balachandran2019-07-291-3/+5
| | | | | | | | | | | | | | Fixed a bug in the revalidate code path that wiped out directory permissions if no mds subvol was found. Backport of: > Change-Id: I8b4239ffee7001493c59d4032a2d3062586ea115 > fixes: bz#1716830 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I8b4239ffee7001493c59d4032a2d3062586ea115 fixes: bz#1716848 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* glusterd: do not mark skip_locking as true for geo-rep operationsSanju Rakonde2019-07-241-2/+7
| | | | | | | | | | | | | | | | | | | | | | We need to send the commit req to peers in case of geo-rep operations even though it is a no volname operation. In commit phase peers try to set the txn_opinfo which will fail because it is a no volname operation where we don't require a commit phase. We mark skip_locking as true for no volname operations, but we have to give an exception to geo-rep operations, so that they can set txn_opinfo in commit phase. Please refer to detailed RCA at the bug: 1730545 fixes: bz#1730545 Cherrypicked from https://review.gluster.org/#/c/glusterfs/+/23034/ > Change-Id: I9f2478b12a281f6e052035c0563c40543493a3fc > Signed-off-by: Sanju Rakonde <srakonde@redhat.com> Change-Id: I9f2478b12a281f6e052035c0563c40543493a3fc Signed-off-by: Sanju Rakonde <srakonde@redhat.com>
* doc: Added release notes for 6.4v6.4hari gowtham2019-07-171-0/+40
| | | | | | | Fixes: bz#1726935 Change-Id: If682cd271c07fdcb62d77ff007ff502ee4a28501 Signed-off-by: hari gowtham <hgowtham@redhat.com>