summaryrefslogtreecommitdiffstats
path: root/libglusterfs
Commit message (Collapse)AuthorAgeFilesLines
* glusterd: manage upgrade to current masterAmar Tumballi2019-02-041-5/+5
| | | | | | | | | | | | | | | | | | | | | | Scenarios tested: * Upgrade the node when there are stripe / tiering and regular type of volumes are present. - All volumes are started fine (as the change was not on brick volfile) - For tier, the functionality may not even work, as changetimerecorder is not present. - 'gluster volume info' properly shows as 'NOT SUPPORTED' for stripe and tier type of volume. * Upgrade in a rolling upgrade scenario, where an old version is able to connect to higher master. - on a normal volume, if the volfile-server was new, the newer client volfiles needed to have utime xlator conditionally. - with this one change, all other changes seem to work fine. Change-Id: Ib2d3b69dafa02b2c695a735b13c1aa70aba07cb8 updates: bz#1635688 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* mount/fuse: expose auto-invalidation as a mount optionRaghavendra Gowdappa2019-02-021-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Auto invalidation is necessary when same (meta)data is shared/access across multiple mounts. However, if (meta)data is not shared, all relevant I/O goes through the cache of single mount and hence is coherent with (meta)data on bricks always. So, fuse-auto-invalidation can be disabled for this case which gives a huge performance boost for workloads that write data and then immediately read the data they just wrote. From glusterfs --help, <snip> --auto-invalidation[=BOOL] controls whether fuse-kernel can auto-invalidate attribute, dentry and page-cache. Disable this only if same files/directories are not accessed across two different mounts concurrently [default: "on"] </snip> Details on how disabling auto-invalidation helped to reduce pgbench init times can be found at [1]. Time taken for pgbench init of scale 8000 was 8340s. That will be an improvement of 86% (59280s vs 8340s) with auto-invalidations turned off along with other optimizations. Just disabling auto-invalidation contributed 56% improvement by reducing the total time taken by 33260s. [1] https://www.spinics.net/lists/gluster-devel/msg25907.html Change-Id: I0ed730dba9064bd9c576ad1800170a21e100e1ce Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com> updates: bz#1664934
* core: make gf_thread_create() easier to useXavi Hernandez2019-02-015-59/+105
| | | | | | | | | | | | | | This patch creates a specific function to set the thread name using a string format and a variable argument list, like printf(). This function is used to set the thread name from gf_thread_create(), which now accepts a variable argument list to create the full name. It's not necessary anymore to use a local array to build the name of the thread. This is done automatically. Change-Id: Idd8d01fd462c227359b96e98699f8c6d962dc17c Updates: bz#1193929 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* core: move "dict is NULL" logs to DEBUG log levelMilind Changire2019-02-011-2/+2
| | | | | | | | | Too many logs get printed if dict_ref() and dict_unref() are passed NULL pointer. fixes: bz#1671213 Change-Id: I18afd849d64318f68baa7b549ee310dac0e1e786 Signed-off-by: Milind Changire <mchangir@redhat.com>
* syncop: remove unnecessary call to gf_backtrace_save()Xavi Hernandez2019-01-312-2/+0
| | | | | | | | | | | | A call to gf_backtrace_save() was done on each context switch of a synctask. The backtrace is generated writing to the filesystem, so it can have an important impact on latency. The generated backtrace was not used anywhere, so it's been removed. Change-Id: I399a93b932c5b6e981c696c72c3e1ef44710ba52 Updates: bz#1193929 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* core: heketi-cli is throwing error "target is busy"Mohit Agrawal2019-01-314-4/+3
| | | | | | | | | | | | | | | | | | | | Problem: When rpc-transport-disconnect happens, server_connection_cleanup_flush_cbk() is supposed to call rpc_transport_unref() after open-files on that transport are flushed per transport.But open-fd-count is maintained in bound_xl->fd_count, which can be incremented/decremented cumulatively in server_connection_cleanup() by all transport disconnect paths. So instead of rpc_transport_unref() happening per transport, it ends up doing it only once after all the files on all the transports for the brick are flushed leading to rpc-leaks. Solution: To avoid races maintain fd_cnt at client instead of maintaining on brick Credits: Pranith Kumar Karampuri Change-Id: I6e8ea37a61f82d9aefb227c5b3ab57a7a36850e6 fixes: bz#1668190 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* Multiple files: reduce work while under lock.Yaniv Kaul2019-01-294-11/+12
| | | | | | | | | | | | | | | | | Mostly, unlock before logging. In some cases, moved different code that was not needed to be under lock (for example, taking time, or malloc'ing) to be executed before taking the lock. Note: logging might be slightly less accurate in order, since it may not be done now under the lock, so order of logs is racy. I think it's a reasonable compromise. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I2438710016afc9f4f62a176ef1a0d3ed793b4f89
* socket: fix issue on concurrent handle of a socketZhang Huan2019-01-281-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Found an issue on concurrent invoke of event handler to the same socket fd, causing memory corruption. This issue arises after applying commit "socket: Remove redundant in_lock in incoming message handling" that removes priv->in_lock to serialize socket read. The following call sequence describes how concurrent socket event handle happens. thread 1 thread 2 thread 3 epoll_wait() return (slot->in_handler is 0) call select_on_epoll() and epoll_ctl() on fd epoll_wait() return slot->in_handler++ (slot->in_handler is 1) slot->in_handler++ (slot->in_handler is 2) call handler() call handler() Fix this issue by skip invoke of handler if there is already a handler inprogress. Change-Id: I437126ac772debcadb00993a948919c931cd607b updates: bz#1467614 Signed-off-by: Zhang Huan <zhanghuan@open-fs.com>
* afr/self-heal:Fix wrong type checkingRavishankar N2019-01-244-1/+28
| | | | | | | | | | gf_dirent struct has d_type variable which should check with DT_DIR istead of IA_IFDIR or IA_IFDIR has to compare with entry->d_stat.ia_type Change-Id: Idf1059ce2a590734bc5b6adaad73604d9a708804 updates: bz#1653359 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
* core: move logs which are only developer relevant to DEBUG levelAmar Tumballi2019-01-233-5/+5
| | | | | | | | | | | We had only changed the log level to DEBUG in release branch earlier. But considering 90%+ of our deployments happen in same env, we can look at these specific logs on need basis. With this change, the master branch will be easier to debug with lesser logs. Change-Id: I4157a7ec7d5ec9c2948b2bbc1e4cb8317f28d6b8 Updates: bz#1666833 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* rpc: use address-family option from vol fileMilind Changire2019-01-223-2/+19
| | | | | | | | | | | | | | | | | This patch helps enable IPv6 connections in the cluster. The default address-family is IPv4 without using this option explicitly. When address-family is set to "inet6" in the /etc/glusterfs/glusterd.vol file, the mount command-line also needs to have -o xlator-option="transport.address-family=inet6" added to it. This option also gets added to the brick command-line. Snapshot and gfapi use-cases should also use this option to pass in the inet6 address-family. Change-Id: I97db91021af27bacb6d7578e33ea4817f66d7270 fixes: bz#1635863 Signed-off-by: Milind Changire <mchangir@redhat.com>
* common-utils: make vector a const parameterXiubo Li2019-01-221-1/+1
| | | | | | | | To avoid the warning and preparing for adding writesame support. Updates: #617 Change-Id: I0710b1e4c240368a9bf52968bddc6e250ae2028d Signed-off-by: Xiubo Li <xiubli@redhat.com>
* core: Feature added to accept CidrIp in auth.allowRinku Kothiya2019-01-183-6/+37
| | | | | | | | | | | | | | | Added functionality to gluster volume set auth.allow command to accept CIDR IP addresses. Modified few functions to isolate cidr feature so that it prevents other gluster commands such as peer probe to use cidr format ip. The functions are modified in such a way that they have an option to enable accepting of cidr format for other gluster commands if required in furture. updates: bz#1138841 Change-Id: Ie6734002a7078f1820e5df42d404411cce945e8b Credits: Mohit Agrawal Signed-off-by: Rinku Kothiya <rkothiya@redhat.com>
* lock: Add fencing supportSusant Palai2019-01-171-0/+2
| | | | | | | | | | | | | | | | | design reference: https://review.gluster.org/#/c/glusterfs-specs/+/21925/ This patch adds the lock preempt support. Note: The current model stores lock enforcement information as separate xattr on disk. There is another effort going in parallel to store this in stat(x) of the file. This patch is self sufficient to add fencing support. Based on the availability of the stat(x) support either I will rebase this patch or we can modify the necessary bits post merging this patch. Change-Id: If4a42f3e0afaee1f66cdb0360ad4e0c005b5b017 updates: #466 Signed-off-by: Susant Palai <spalai@redhat.com>
* core: Resolve dict_leak at the time of destroying graphMohit Agrawal2019-01-146-12/+2
| | | | | | | | | | | | Problem: In gluster code some of the places it call's get_new_dict to create a dictionary without taking reference so at the time of dict_unref it has become a leak Solution: To resolve the same call dict_new instead of get_new_dict updates bz#1650403 Change-Id: I3ccbbf5af07079a4fa09aad2cd0458c8625b2f06 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* fix 32-bit-build-smoke warningsIraj Jamali2019-01-113-8/+8
| | | | | | | fixes: bz#1622665 Change-Id: I777d67b1b62c284c62a02277238ad7538eef001e Signed-off-by: Iraj Jamali <ijamali@redhat.com>
* libglusterfs/common-utils.c: Fix buffer size for checksum computationVarsha Rao2019-01-111-2/+2
| | | | | | | | | | | | | | Problem: When quorum count option is updated, the change is not reflected in the nfs-server.vol file. This is because in get_checksum_for_file(), when the last part of the file read has size less than buffer size, the read buffer stores old data value along with correct data value. Solution: Pass the bytes read instead of fixed buffer size, for calculating checksum. Change-Id: I4b641607c8a262961b3f3da0028a54e08c3f8589 fixes: bz#1657744 Signed-off-by: Varsha Rao <varao@redhat.com>
* Revert "iobuf: Get rid of pre allocated iobuf_pool and use per thread mem pool"Amar Tumballi2019-01-083-72/+877
| | | | | | | | | This reverts commit b87c397091bac6a4a6dec4e45a7671fad4a11770. There seems to be some performance regression with the patch and hence recommended to have it reverted. Updates: #325 Change-Id: Id85d6203173a44fad6cf51d39b3e96f37afcec09
* gfapi: new api glfs_statx as linux's statxShyamsundarR2019-01-071-4/+13
| | | | | | | Change-Id: I44dd6ceef0954ae7fc13f920e84d81bbd3f6a774 Updates: #389 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Signed-off-by: ShyamsundarR <srangana@redhat.com>
* multiple-files: clang-scan fixesAmar Tumballi2018-12-311-3/+3
| | | | | | updates: bz#1622665 Change-Id: I9f3a75ed9be3d90f37843a140563c356830ef945 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* libglusterfs/src/mem-types.h: remove unused common enums from mem-types.hYaniv Kaul2018-12-301-136/+98
| | | | | | | | | | | | | | | | | They were not used at all, just taking space. I've also marked all those that are not common really, but used in just one place - they probably should move there (in follow-up patches) As a test, I've removed from the stripe xlator unused private enums and moved one that was in the common list, but only used in the stripe code, to be a private enum. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I1158dc1d259f1fd3f69904336c46c9d83cea799f
* all: handle USE_AFTER_FREE warningsAmar Tumballi2018-12-202-3/+8
| | | | | | | | | | | | | | | | | | | * we shouldn't be using 'local' after DHT_STACK_UNWIND() as it frees the content of local. Add a 'goto out' or similar logic to handle the situation. * fix possible overlook of unref(dict), instead of unref(xdata). * make coverity happy by re-ordering unref in meta-defaults. * gfid-access: re-order dictionary allocation so we don't have to do a extra unref. * other obvious errors reported. updates: bz#789278 Change-Id: If05961ee946b0c4868df19861d7e4a927a2a2489 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* posix: use synctask for janitorPoornima G2018-12-191-0/+6
| | | | | | | | | | | | | | With brick mux, the number of threads increases as the number of bricks increases. As an initiative to reduce the number of threads in brick mux scenario, replacing janitor thread to use synctask infra. Now close() and closedir() handle by separate janitor thread which is linked with glusterfs_ctx. Updates #475 Change-Id: I0c4aaf728125ab7264442fde59f3d08542785f73 Signed-off-by: Poornima G <pgurusid@redhat.com>
* cluster/afr: Allow lookup on root if it is from ADD_REPLICA_MOUNTkarthik-us2018-12-181-1/+2
| | | | | | | | | | | | | | | | | | | | | Problem: When trying to convert a plain distribute volume to replica-3 or arbiter type it is failing with ENOTCONN error as the lookup on the root will fail as there is no quorum. Fix: Allow lookup on root if it is coming from the ADD_REPLICA_MOUNT which is used while adding bricks to a volume. It will try to set the pending xattrs for the newly added bricks to allow the heal to happen in the right direction and avoid data loss scenarios. Note: This fix will solve the problem of type conversion only in the case where the volume was mounted at least once. The conversion of non mounted volumes will still fail since the dht selfheal tries to set the directory layout will fail as they do that with the PID GF_CLIENT_PID_NO_ROOT_SQUASH set in the frame->root. Change-Id: Ic511939981dad118cc946754341318b164954b3b fixes: bz#1655854 Signed-off-by: karthik-us <ksubrahm@redhat.com>
* iobuf: Get rid of pre allocated iobuf_pool and use per thread mem poolPoornima G2018-12-183-877/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current implementation of iobuf_pool has two problems: - prealloc of 12.5MB memory, this limits the scale factor of the gluster processes due to RAM requirements - lock contention, as the current implementation has one global iobuf_pool lock. Credits for debugging and addressing the same goes to Krutika Dhananjay <kdhananj@redhat.com>. Issue: #410 Hence changing the iobuf implementation to use per thread mem pool. This may theoritically appear to cause perf dip as there is no preallocation. But per thread mem pool will not have significant perf impact as the last allocated memory is kept alive for subsequent allocs, for some time. The worst case would be if iobufs requested are of random sizes each time. The best case is, if we get iobuf request of the same size. From the perf tests, this patch did not seem to cause any perf decrease. Note that, with this patch, the rdma performance is going to degrade drastically. In one of the previous patchsets we had fixes to not degrade rdma perf, but rdma is not supported and also not tested [1]. Hence the decision was to not have code in rdma that is not tested and not supported. [1] https://lists.gluster.org/pipermail/gluster-users.old/2018-July/034400.html Updates: #325 Change-Id: Ic2ef3bd498f9250dea25f25ba0c01fde19584b27 Signed-off-by: Poornima G <pgurusid@redhat.com>
* mem-pool: Add api to mem_get based on requested sizePoornima G2018-12-174-27/+127
| | | | | | | | | | | | | | | Currently mem-pool implementation provides api to get from the mem pool based on the struct type. This is to retain api compatibility with the old implementation of mem pool. Internally in the mem pool structure there is a mapping from struct to size based pools. In this patch, we are adding new APIs to fetch memory from mem pool, given a size. Change-Id: Ib220ee45ebd134a7be8f6482db5a592dbb7b9211 Updates: #325 Signed-off-by: Poornima G <pgurusid@redhat.com>
* fuse: add --lru-limit optionAmar Tumballi2018-12-144-36/+238
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The inode LRU mechanism is moot in fuse xlator (ie. there is no limit for the LRU list), as fuse inodes are referenced from kernel context, and thus they can only be dropped on request of the kernel. This might results in a high number of passive inodes which are useless for the glusterfs client, causing a significant memory overhead. This change tries to remedy this by extending the LRU semantics and allowing to set a finite limit on the fuse inode LRU. A brief history of problem: When gluster's inode table was designed, fuse didn't have any 'invalidate' method, which means, userspace application could never ask kernel to send a 'forget()' fop, instead had to wait for kernel to send it based on kernel's parameters. Inode table remembers the number of times kernel has cached the inode based on the 'nlookup' parameter. And 'nlookup' field is not used by no other entry points (like server-protocol, gfapi etc). Hence the inode_table of fuse module always has to have lru-limit as '0', which means no limit. GlusterFS always had to keep all inodes in memory as kernel would have had a reference to it. Again, the reason for this is, kernel's glusterfs inode reference was pointer of 'inode_t' structure in glusterfs. As it is a pointer, we could never free it (to prevent segfault, or memory corruption). Solution: In the inode table, handle the prune case of inodes with 'nlookup' differently, and call a 'invalidator' method, which in this case is fuse_invalidate(), and it sends the request to kernel for getting the forget request. When the kernel sends the forget, it means, it has dropped all the reference to the inode, and it will send the forget with the 'nlookup' parameter too. We just need to make sure to reduce the 'nlookup' value we have when we get forget. That automatically cause the relevant prune to happen. Credits: Csaba Henk, Xavier Hernandez, Raghavendra Gowdappa, Nithya B fixes: bz#1560969 Change-Id: Ifee0737b23b12b1426c224ec5b8f591f487d83a2 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* Multiple posix related files: several modificationsYaniv Kaul2018-12-142-6/+4
| | | | | | | | | | | | | | | | | | | | | | | | Just looked at posix.c and related code and performed some changes and cleanups. The only important one is #3 below, but surely the others (#2 and #4) need careful review. Changes to other files are as they were related to code paths in posix.c. I'll send a separate patch for other posix related files. Main changes: 1. Proper initializtion for parameters, where it made sense. 2. Logged outside the lock in several places. 3. Moved from CALLOC to MALLOC where it made sense. 4. Aligned structures. 5. moved dictionary functions to use _sizen where possible. (dict_get() -> dict_get_sizen() for example) Compile-tested only! Change-Id: Ia84699fb495e06d095339c91c1ba770d1393bb6c updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
* xlator: make 'xlator_api' mandatoryAmar Tumballi2018-12-132-145/+16
| | | | | | | | | | | | | | * Remove the options to load old symbol. * keep only 'xlator_api' symbol from being exported using xlator.sym * add xlator_api to all the xlators where its missing NOTE: This covers all the xlators which has at least a test case to validate its loading. If there is a translator, which doesn't have any test, then we should probably remove that from codebase. fixes: #164 Change-Id: Ibcdc8c9844cda6b4463d907a15813745d14c1ebb Signed-off-by: Amar Tumballi <amarts@redhat.com>
* [geo-rep]: Worker still ACTIVE after killing bricksMohit Agrawal2018-12-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | Problem: In changelog xlator after destroying listener it call's unlink to delete changelog socket file but socket file reference is not cleaned up from process memory Solution: 1) To cleanup reference completely from process memory serialize transport cleanup for changelog and then unlink socket file 2) Brick xlator will notify GF_EVENT_PARENT_DOWN to next xlator only after cleanup all xprts Test: To test the same run below steps 1) Setup some volume and enable brick mux 2) kill anyone brick with gf_attach 3) check changelog socket for specific to killed brick in lsof, it should cleanup completely fixes: bz#1600145 Change-Id: Iba06cbf77d8a87b34a60fce50f6d8c0d427fa491 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* core: move invalid port logs to DEBUG log levelMilind Changire2018-12-121-1/+1
| | | | | | | | | Stop spamming "invalid port" logs in case sysadmin has reserved a large number of ports. Change-Id: I244ef7693560cc404b36cadc6b05d92ec0e908d3 fixes: bz#1656517 Signed-off-by: Milind Changire <mchangir@redhat.com>
* copy_file_range support in GlusterFSRaghavendra Bhat2018-12-1217-6/+369
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * libglusterfs changes to add new fop * Fuse changes: - Changes in fuse bridge xlator to receive and send responses * posix changes to perform the op on the backend filesystem * protocol and rpc changes for sending and receiving the fop * gfapi changes for performing the fop * tools: glfs-copy-file-range tool for testing copy_file_range fop - Although, copy_file_range support has been added to the upstream fuse kernel module, no release has been made yet of a kernel which contains the support. It is expected to come in the upcoming release of linux-4.20 So, as of now, executing copy_file_range fop on a fused based filesystem results in fuse kernel module sending read on the source fd and write on the destination fd. Therefore a small gfapi based tool has been written to be able test the copy_file_range fop. This tool is similar (in functionality) to the example program given in copy_file_range man page. So, running regular copy_file_range on a fuse mount point and running gfapi based glfs-copy-file-range tool gives some idea about how fast, the copy_file_range (or reflink) can be. On the local machine this was the result obtained. mount -t glusterfs workstation:new /mnt/glusterfs [root@workstation ~]# cd /mnt/glusterfs/ [root@workstation glusterfs]# ls file [root@workstation glusterfs]# cd [root@workstation ~]# time /tmp/a.out /mnt/glusterfs/file /mnt/glusterfs/new real 0m6.495s user 0m0.000s sys 0m1.439s [root@workstation ~]# time glfs-copy-file-range $(hostname) new /tmp/glfs.log /file /rrr OPEN_SRC: opening /file is success OPEN_DST: opening /rrr is success FSTAT_SRC: fstat on /rrr is success copy_file_range successful real 0m0.309s user 0m0.039s sys 0m0.017s This tool needs following arguments 1) hostname 2) volume name 3) log file path 4) source file path (relative to the gluster volume root) 5) destination file path (relative to the gluster volume root) "glfs-copy-file-range <hostname> <volume> <log file path> <source> <destination>" - Added a testcase as well to run glfs-copy-file-range tool * io-stats changes to capture the fop for profiling * NOTE: - Added conditional check to see whether the copy_file_range syscall is available or not. If not, then return ENOSYS. - Added conditional check for kernel minor version in fuse_kernel.h and fuse-bridge while referring to copy_file_range. And the kernel minor version is kept as it is. i.e. 24. Increment it in future when there is a kernel release which contains the support for copy_file_range fop in fuse kernel module. * The document which contains a writeup on this enhancement can be found at https://docs.google.com/document/d/1BSILbXr_knynNwxSyyu503JoTz5QFM_4suNIh2WwrSc/edit Change-Id: I280069c814dd21ce6ec3be00a884fc24ab692367 updates: #536 Signed-off-by: Raghavendra Bhat <raghavendra@redhat.com>
* libglusterfs: Move devel headers under glusterfs directoryShyamsundarR2018-12-05130-386/+395
| | | | | | | | | | | | | | | | | | | | | | | | libglusterfs devel package headers are referenced in code using include semantics for a program, this while it works can be better especially when dealing with out of tree xlator builds or in general out of tree devel package usage. Towards this, the following changes are done, - moved all devel headers under a glusterfs directory - Included these headers using system header notation <> in all code outside of libglusterfs - Included these headers using own program notation "" within libglusterfs This change although big, is just moving around the headers and making it correct when including these headers from other sources. This helps us correctly include libglusterfs includes without namespace conflicts. Change-Id: Id2a98854e671a7ee5d73be44da5ba1a74252423b Updates: bz#1193929 Signed-off-by: ShyamsundarR <srangana@redhat.com>
* libglusterfs/src/iobuf.c: small refactor to re-use code.Yaniv Kaul2018-12-051-23/+13
| | | | | | | | | | No functional changes (I hope). Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Ifbec21c18a6dbe27c5271db156bff4d30ca85dbf
* rpc: bump up server.event-threadsMilind Changire2018-12-041-1/+1
| | | | | | | | | | | | | | | Problem: A single event-thread causes performance issues in the system. Solution: Bump up event-threads to 2 to make the system more performant. This helps in making the system more responsive and helps avoid the ping-timer-expiry problem as well. However, setting the event-threads to 2 is not the only thing required to avoid ping-timer-expiry issues. Change-Id: Idb0fd49e078db3bd5085dd083b0cdc77b59ddb00 fixes: bz#1653277 Signed-off-by: Milind Changire <mchangir@redhat.com>
* server: Resolve memory leak path in server_initMohit Agrawal2018-12-033-0/+34
| | | | | | | | | | | | | | Problem: 1) server_init does not cleanup allocate resources while it is failed before return error 2) dict leak at the time of graph destroying Solution: 1) free resources in case of server_init is failed 2) Take dict_ref of graph xlator before destroying the graph to avoid leak Change-Id: I9e31e156b9ed6bebe622745a8be0e470774e3d15 fixes: bz#1654917 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* rpcsvc: provide each request handler thread its own queueRaghavendra Gowdappa2018-11-294-28/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | A single global per program queue is contended by all request handler threads and event threads. This can lead to high contention. So, reduce the contention by providing each request handler thread its own private queue. Thanks to "Manoj Pillai"<mpillai@redhat.com> for the idea of pairing a single queue with a fixed request-handler-thread and event-thread, which brought down the performance regression due to overhead of queuing significantly. Thanks to "Xavi Hernandez"<xhernandez@redhat.com> for discussion on how to communicate the event-thread death to request-handler-thread. Thanks to "Karan Sandha"<ksandha@redhat.com> for voluntarily running the perf benchmarks to qualify that performance regression introduced by ping-timer-fixes is fixed with this patch and patiently running many iterations of regression tests while RCAing the issue. Thanks to "Milind Changire"<mchangir@redhat.com> for patiently running the many iterations of perf benchmarking tests while RCAing the regression caused by ping-timer-expiry fixes. Change-Id: I578c3fc67713f4234bd3abbec5d3fbba19059ea5 Fixes: bz#1644629 Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com>
* core: ctx calls naked calloc()Kaleb S. KEITHLEY2018-11-281-1/+1
| | | | | | | | | | | | | | | | liblglusterfs provides wrapper functions MALLOC/__gf_default_malloc, CALLOC/__gf_default_calloc, and REALLOC/__gf_default_realloc for those few places outside of mempool.c that need to call malloc/calloc/realloc directly. Notable exceptions are "contrib" code, e.g. rbtree and timer-wheel, and perhaps parsers generated by yacc+lex. But even parsers can be fixed to at least call the wrappers mentioned above, if not our own allocators. Change-Id: Ib8069815eba9b6c04c3adaf59727ec8d8795c4d1 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* libglusterfs: rename macros roof and floor to not conflict with math.hRaghavendra Gowdappa2018-11-281-2/+2
| | | | | | Change-Id: I666eeb63ebd000711b3f793b948d4e0c04b1a242 Signed-off-by: Raghavendra Gowdappa <rgowdapp@redhat.com> Updates: bz#1644629
* mem-pool: minor fix and clarificationXavi Hernandez2018-11-281-3/+10
| | | | | | | | | | A comment has been added to pool_destructor() function to explain why locks are not needed there. Also, the initialization of 'poison' field has been moved inside a locked region for further safety and clarity. Change-Id: Idbf23bda7f9228d60c644a1bea4b6c2cfc582090 updates: bz#1193929 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* glusterd: perform store operation in cleanup lockAtin Mukherjee2018-11-271-0/+1
| | | | | | | | | All glusterd store operation and cleanup thread should work under a critical section to avoid any partial store write. Change-Id: I4f12e738f597a1f925c87ea2f42565dcf9ecdb9d Fixes: bz#1652430 Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
* libglusterfs/src/dict.c : consistent initialization of parameters.Yaniv Kaul2018-11-271-60/+29
| | | | | | | | | | | | | | | | Some were assigned NULL, for no good reason, some were assigned proper initial value. Made them all consistent, as much as possible, to be assigned reasonable initial values. No expected functional changes (and I also assume the compiler already did most of this work behind the scenes anyway, so no performance implications either). Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I2bc0d4f2221124b5f9ef6150c86b7259074e7013
* libglusterfs: fix memory corruption caused by per-thread mem poolsXavi Hernandez2018-11-261-57/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There was a race in the per-thread memory pool management that could lead to memory corruption. The race appeared when the following sequence of events happened: 1. Thread T1 allocated a memory object O1 from its own private pool P1 2. T1 terminates and P1 is marked to be destroyed 3. The mem-sweeper thread is woken up and scans all private pools 4. It detects that P1 needs to be destroyed and starts releasing the objects from hot and cold lists. 5. Thread T2 releases O1 6. O1 is added to the hot list of P1 The problem happens because steps 4 and 6 are protected by diferent locks, so they can run concurrently. This means that both T1 and T2 are modifying the same list at the same time, potentially causing corruption. This patch fixes the problem using the following approach: 1. When an object is released, it's only returned to the hot list of the corresponding memory pool if it's not marked to be destroyed. Otherwise the memory is released to the system. 2. Object release and mem-sweeper thread synchronize access to the deletion mark of the memory pool to prevent simultaneous access to the list. Some other minor adjustments are made to reduce the lengths of the locked regions. Fixes: bz#1651165 Change-Id: I63be3893f92096e57f54a6150e0461340084ddde Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* Coverity fix for calling risky function - fscanfPoornima G2018-11-242-7/+7
| | | | | | | | | | | | | | | fscanf with %s reads a word, there is no restriction on the length of that word, and the caller is required to pass a sufficiently large buffer for storing thw word. If the input word exceeds the buffer size, it will cause buffer overflow. To fix this, use fscanf with width parameter. Width specifies the maximum number of characters to be read in the current reading operation. Change-Id: If250abf5eb637b9fc2a79047e3599f83254cd4e5 updates: bz#1193929 Signed-off-by: Poornima G <pgurusid@redhat.com>
* core: create a constant for default network timeoutXavi Hernandez2018-11-231-0/+2
| | | | | | | | | | A new constant named GF_NETWORK_TIMEOUT has been defined and all references to the hard-coded timeout of 42 seconds have been replaced with this constant. Change-Id: Id30f5ce4f1230f9288d9e300538624bcf1a6da27 fixes: bz#1652852 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* libglusterfs/src/common-utils.h: faster mem_0filled() functionYaniv Kaul2018-11-201-10/+22
| | | | | | | | | | | based on the amusing discussion @ https://rusty.ozlabs.org/?p=560 Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: I1cac54067eb44801b216d5620fc5ee2c89befdd0
* core: fix strncpy warningsKaleb S. KEITHLE2018-11-152-3/+3
| | | | | | | | | | | | | | | | | | | Since gcc-8.2.x (fedora-28 or so) gcc has been emitting warnings about buggy use of strncpy. e.g. warning: ‘strncpy’ output truncated before terminating nul copying as many bytes from a string as its length and warning: ‘strncpy’ specified bound depends on the length of the source argument Since we're copying string fragments and explicitly null terminating use memcpy to silence the warning Change-Id: I413d84b5f4157f15c99e9af3e154ce594d5bcdc1 updates: bz#1193929 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* core: Portmap entries showing stale brick entries when bricks are downMohit Agrawal2018-11-121-1/+4
| | | | | | | | | | | | | | | | Problem: pmap is showing stale brick entries after down the brick because of glusterd_brick_rpc_notify call gf_is_service_running before call pmap_registry_remove to ensure about brick instance. Solutiom: 1) Change the condition in gf_is_pid_running to ensure about process existence, use open instead of access to achieve the same 2) Call search_brick_path_from_proc in __glusterd_brick_rpc_notify along with gf_is_service_running Change-Id: Ia663ac61c01fdee6c12f47c0300cdf93f19b6a19 fixes: bz#1646892 Signed-off-by: Mohit Agrawal <moagrawal@redhat.com>
* libglusterfs multiple files: remove dead initilizationYaniv Kaul2018-11-1110-55/+13
| | | | | | | | | | | | | Per newer GCC releases and clang-scan, some trivial dead initialization (values that were set but were never read) were removed. Compile-tested only! updates: bz#1193929 Signed-off-by: Yaniv Kaul <ykaul@redhat.com> Change-Id: Ia9959b2ff87d2e9cb46864e68ffe7dccb984db34
* coverity: ignore tainted access reported in gf_freeShyamsundarR2018-11-081-1/+7
| | | | | | | | | | | | | Earlier commit had the annotation incorrect, and also did not wrap the sanitization in a separate function. (see commit 39a1db1) The issues are corrected in this patch, and also a coverity stand alone run has been tested to ensure the annotations are respected by coverity. Change-Id: I4a93b6981e2ff4bba9a29e590b17da248931c8ae Updates: bz#789278 Signed-off-by: ShyamsundarR <srangana@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-17 20:12:15 +0000