summaryrefslogtreecommitdiffstats
path: root/api
Commit message (Collapse)AuthorAgeFilesLines
* tests/gfapi : add test case for nameless lookups in glfs_resolve_component()Jiffin Tony Thottan2017-08-021-1/+2
| | | | | | | | | | | | | Plus address pending comment to add check for entry "" in glfs_resolve_component() Change-Id: I6063f776ce1cd76cb4c1b1f621b064f3dcc91e5c BUG: 1460514 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: https://review.gluster.org/17844 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com>
* gfapi: mark glfs_ipc() for internal use onlyNiels de Vos2017-07-265-20/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | The 3.7 version of glfs_ipc() has never been used by external applications. There is little use for internal xlator communication that is triggered from outside of core GlusterFS executables. This function has now been removed from libgfapi.so. For Gluster 4.0 a new variation for glfs_ipc() has been added. The function expects dict_t parameters, which are currently not available for external applications. There is no sense in providing glfs_ipc() for non-core GlusterFS executables. Therefore, glfs_ipc() has been marked as private, and the declaration is now in the glfs-internal.h header. The Python test case (tests/features/ipctest.py) is not correct and will be re-written in C to prevent portability issues. This test is currently disabled (commit d26f0bac149d495fa93710c3f7b6b63c36cb8387). Change-Id: Idbfe35570d34d45ce8b6b43084627a552ac21f59 Fixes: #269 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17854 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* mem-pool: initialize pthread_key_t pool_key in mem_pool_init_early()Niels de Vos2017-07-191-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is not possible to call pthread_key_delete for the pool_key that is intialized in the constructor for the memory pools. This makes it difficult to do a full cleanup of all the resources in mem_pools_fini(). For this, the initialization of pool_key should be moved to mem_pool_init(). However, the glusterfsd binary has a rather complex initialization procedure. The memory pools need to get initialized partially to get mem_get() functionality working. But, the pool_sweeper thread can get killed in case it is started before glusterfsd deamonizes. In order to solve this, mem_pools_init() is split into two pieces: 1. mem_pools_init_early() for initializing the basic structures 2. mem_pools_init_late() to start the pool_sweeper thread With the split of mem_pools_init(), and placing the pthread_key_create() in mem_pools_init_early(), it is now possible to correctly cleanup the pool_key with pthread_key_delete() in mem_pools_fini(). It seems that there was no memory pool initialization in the CLI. This has been added as well now. Without it, the CLI will not be able to call mem_get() successfully which results in a hang of the process. Change-Id: I1de0153dfe600fd79eac7468cc070e4bd35e71dd BUG: 1470170 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17779 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* libglusterfs: Name threads on creationRaghavendra Talur2017-07-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set names to threads on creation for easier debugging. Output of top -H -p <PID-OF-GLUSTERFSD> Before: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd After: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustertimer 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustermemsweep 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc0 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc1 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll0 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteridxwrker 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteriotwr0 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrssign 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrswrker 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterclogecon 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd0 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd1 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd2 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixjan 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixfsy 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll1 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll2 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixhc Change-Id: Id5f333755c1ba168a2ffaa4fce6e71c375e10703 BUG: 1254002 Updates: #271 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: https://review.gluster.org/11926 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: prevent mem-pool leak in case glfs_new_fs() failsNiels de Vos2017-07-121-12/+21
| | | | | | | | | | | | | | | | | | | | | | | | Commit 7039243e187 adds a call to mem_pools_init() so that the memory pool cleanup thread ("sweeper") is started. However, now it is possible that users of gfapi can not cleanup this thread because glfs_new() can return NULL, but the sweeper is still running. In case glfs_fs_new() fails, mem_pools_fini() needs to be called as well. This seems more correct than calling mem_pools_init() after glfs_fs_new(), and this makes using memory pools possible *really* early in the gfapi initialization. Change-Id: I1f2fb25cc33e227b3c33ce9d1b03f67bc27e981a Fixes: 7039243e187 ("gfapi: add mem_pools_init and mem_pools_fini calls") BUG: 1468863 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17734 Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Reviewed-by: Vijay Bellur <vbellur@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* mgtm/core : use sha hash function for volfile checkMohammed Rafi KC2017-07-101-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are storing the entire volfile and using this to check volfile change. With brick multiplexing there will be lot of graphs per process which will increase the memory foot print of the process. So instead of storing the entire graph we could use sha256 and we can compare the hash to see whether volfile change happened or not. Also with Brick multiplexing, the direct comparison of vol file is not correct. There are two problems. Problem 1: We are currently storing one single graph (the last updated volfile) whereas, what we need is the entire graph with all atttached bricks. If we fix this issue, we have second problem Problem 2: With multiplexing we have a graph that contains multiple bricks. But what we are checking as part of the reconfigure is, comparing the entire graph with one single graph, which will always fail. Solution: We create list in glusterfs_ctx_t that stores sha256 hash of individual brick graphs. When a graph changes happens we compare the stored hash and the current hash. If the hash matches, then no need for reconfigure. Otherwise we first do the reconfigure and then update the hash. For now, gfapi has not changed this way. Meaning when gfapi volfile fetch or reconfigure happens, we still store the entire graph and compare, each memory. This is fine, because libgfapi will not load brick graphs. But changing the libgfapi will make the code similar in both glusterfsd-mgmt and api. Also it helps to reduce some memory. Change-Id: I9df917a771a52b95622ab8f63af34ec390163a77 BUG: 1467986 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17709 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* gfapi+libglusterfs: fix mem_pools_fini without mem_pools_init caseJeff Darcy2017-07-091-6/+6
| | | | | | | | | | | | | | The change consists of two parts: make sure it doesn't happen (in glfs.c), and make it harmless if it does (in mem-pool.c). Change-Id: Icb7dda7a45dd3d1ade2ee3991bb6a22c8ec88424 BUG: 1468863 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17728 Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* gfapi: add mem_pools_init and mem_pools_fini callsKaleb S. KEITHLEY2017-07-051-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | These are both needed because GFAPI clients are a bit unique. They can be long-running, so they can potentially benefit from the memory reclamation that mem_pools_init will enable. On the other hand, they might completely tear down their Gluster environment well before the process exits, so they need mem_pools_fini as well. Our main and auxiliary daemons do need mem_pools_init but don't need to call it themselves because that's handled for them in glusterfsd.c right after they daemonize. They don't need mem_pools_fini, because the only place they could *safely* call it is right before exit and then it won't do them any good. Transient processes (e.g. gf_attach) don't benefit from either because, well, they're transient. They'll be gone before memory reclamation matters. Change-Id: I611cf77d8b408b3ecfc00664e8da294edde9e57a Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17666 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* multiple: fix struct/typedef inconsistenciesJeff Darcy2017-06-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | The most common pattern, both in our code and elsewhere, is this: struct _xyz { ... }; typedef struct _xyz xyz_t; These exceptions - especially call_frame/call_stack - have been slowing down code navigation for years. By converging on a single pattern, navigating from xyz_t in code to the actual definition of struct _xyz (i.e. without having to visit the typedef first) might even be automatable. Change-Id: I0e5dd1f51f98e000173c62ef4ddc5b21d9ec44ed Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17650 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* gfapi : Resolve "." and ".." only for named lookupsJiffin Tony Thottan2017-06-201-8/+64
| | | | | | | | | | | | | | | | | | | | The patch https://review.gluster.org/#/c/17177 resolves "." and ".." to corrosponding inodes and names before sending the request to the backend server. But this will only work if inode and its parent is linked properly. Incase of nameless lookup(applications like ganesha) the inode of parent can be NULL(only gfid is send). So this patch will resolve "." and ".." only if proper parent is available Change-Id: I4c50258b0d896dabf000a547ab180b57df308a0b BUG: 1460514 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: https://review.gluster.org/17502 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* gfapi: change root lookup from nameless to named lookupMohammed Rafi KC2017-06-092-2/+35
| | | | | | | | | | | | | | | | | | | | | | | | Problem: During component resolve we do lookup on root based on root gfid. If we are doing nameless lookup, then md-cache won't be able to serve those lookup. Solution: With this patch, we covert nameless lookup to named lookup on root. Credits: Poornima G <pgurusid@redhat.com> Change-Id: Ie64141936fd294d8969c39e3bd4dbc73ee375c6b BUG: 1458768 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17465 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* gfapi: fix handling of dot and double dot in pathMohammed Rafi KC2017-05-141-24/+17
| | | | | | | | | | | | | | | | This patch is to handle "." and ".." in file path. Which means this special dentry names will be resolved before sending fops on the path. Change-Id: I5e92f6d1ad1412bf432eb2488e53fb7731edb013 BUG: 1447266 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17177 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* gfapi: Fix few severe coverity issuesSoumya Koduri2017-05-053-2/+9
| | | | | | | | | | | | | | | From https://download.gluster.org/pub/gluster/glusterfs/static-analysis/master/glusterfs-coverity/2017-05-03-6484558c/html/ fixes coverity#189, 248, 250, 488, 546 Change-Id: I7c4aa5bf6c367344602748dfd736de27f341cca2 BUG: 789278 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: https://review.gluster.org/17161 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi/handleops: Introducing glfs_xreaddirplus_r() fop for handleopsSoumya Koduri2017-05-0210-4/+303
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Its known that readdirplus operation fetches stat as well for each of the dirents. But often applications may need extra information, like for eg., NFS-Ganesha which operates on handles needs handles for each of those dirents returned. So this would require extra calls to the backend, in this case LOOKUP (which is very expensive operation) resulting in very low readdir performance. To address that introducing this new API using which applications can make request for any extra information to be returned as part of readdirplus response. Currently this new api returns stat and handles as demanded by application. The synopsis of the API is noted in glfs.h. @todo: * Enhance test script using this new API Below were the perf results on single brick volume with and without these changes - Dataset used - 10*100 directories and each directory containing 100 empty files. I used NFS-Ganesha application to test these changes - >for i in {1..5}; do systemctl restart nfs-ganesha; sleep 10; mount -t nfs -o vers=4 localhost:/brick_vol /mnt; cd /mnt; echo "ITERATION$i"; date; find . > tmp-nfs.log; date; cd /; umount /mnt; sleep 2; done; Without these changes - ITERATION1 Mon Mar 20 17:22:26 IST 2017 Mon Mar 20 17:23:18 IST 2017 ITERATION2 Mon Mar 20 17:23:39 IST 2017 Mon Mar 20 17:24:28 IST 2017 ITERATION3 Mon Mar 20 17:24:49 IST 2017 Mon Mar 20 17:25:36 IST 2017 ITERATION4 Mon Mar 20 17:30:57 IST 2017 Mon Mar 20 17:31:37 IST 2017 ITERATION5 Mon Mar 20 17:31:57 IST 2017 Mon Mar 20 17:32:40 IST 2017 [root@dhcp35-197 /]# On an average ~46.2 sec With these changes applied - ITERATION1 Mon Mar 20 17:35:03 IST 2017 Mon Mar 20 17:35:15 IST 2017 ITERATION2 Mon Mar 20 17:35:36 IST 2017 Mon Mar 20 17:35:46 IST 2017 ITERATION3 Mon Mar 20 17:36:06 IST 2017 Mon Mar 20 17:36:17 IST 2017 ITERATION4 Mon Mar 20 17:41:38 IST 2017 Mon Mar 20 17:41:49 IST 2017 ITERATION5 Mon Mar 20 17:42:10 IST 2017 Mon Mar 20 17:42:20 IST 2017 On an average ~10.8 sec Updates #174 BUG: 1442950 Change-Id: I0f74f74dc62085ca4c4a23c38e3edc84bd850876 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: https://review.gluster.org/15663 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* api: fix typo of glfs_upcall_get_eventKinglong Mee2017-05-022-2/+2
| | | | | | | | | | | | | glfs_upcall_inode_get_event is not exist. Change-Id: I27bd7eb9bf99d3289ae12fe31405d3b46855152d BUG: 1447116 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17135 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* api: fix two typos of GFAPI_PUBLIC function defineKinglong Mee2017-04-292-2/+2
| | | | | | | | | | | Change-Id: I2f9915b3572c4cbce6748e99a037daba439a187e BUG: 1446273 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17112 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* api: free the xattr data after using in glfs_volfile_fetchKinglong Mee2017-04-261-0/+2
| | | | | | | | | | | | | | Change-Id: I89a7df93039e367301fd70df722c13f1d8b3484b Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17091 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Poornima G <pgurusid@redhat.com> Tested-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* api: fix bad release of dict causes memory leakKinglong Mee2017-04-251-2/+4
| | | | | | | | | | | | | Change-Id: Ic7169fd05aff7bf46108e8ac7b1f29688a7f2358 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17092 Tested-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Poornima G <pgurusid@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* api: free all dynamic memory returned from xdr_to_genericKinglong Mee2017-04-211-1/+4
| | | | | | | | | | | Change-Id: I01235ffca80683e69e99da09a9fc1a6d3ffdcb65 Signed-off-by: Kinglong Mee <mijinlong@open-fs.com> Reviewed-on: https://review.gluster.org/17090 Tested-by: Poornima G <pgurusid@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Smoke: Gluster Build System <jenkins@build.gluster.org>
* api: Fix incorrect strcmp() during fs initialisationKrutika Dhananjay2017-04-182-4/+4
| | | | | | | | | | | | | | Just chanced upon this while reading graph switch code in libgfapi. Change-Id: Ie62b410c22e6d0768a24187c5d587b6d3fc996bf Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16319 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* gfapi: Fix inode ref leak in anonymous fd I/O APIsSoumya Koduri2017-04-181-0/+6
| | | | | | | | | | | | | | | | | In the APIs to do I/Os using anonymous fd, there is a ref taken for inode which hasn't been unreferenced post the operation. This shall result in the leak. Change-Id: I75ea952a6b2df58c385f4f53398e5562f255248d BUG: 1438738 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: https://review.gluster.org/16989 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: prevent off-by-one buffer overrun in glfs_sysrq()Niels de Vos2017-04-181-5/+5
| | | | | | | | | | | | | | | | | Coverity found a potential buffer overrun in the strncat() usage for logging the help message with glfs_sysrq(). This seems to be an off-by-one mistake and should be addressed by reducing the initial size of the remainder calculation. Change-Id: Ide14add1cb28e5200d2c0df6b3a5154999ef3ca9 BUG: 789278 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17024 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* gfapi: remove unneeded NULL-check in glfs_fsync_async_common()Niels de Vos2017-04-111-2/+1
| | | | | | | | | | | | | | | | Coverity reported that "glfd" has a NULL-check at the end of the function, but is used without checking earlier. Because "glfd" is passed as parameter to the (internal) function, there is no need to have the additional NULL-check. Change-Id: I1390b2ef250fbf624f0bc2b0eda7b7588942aae9 BUG: 789278 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17026 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* nfs: make subdir mounting work for Solaris 10 clientsBipin Kunal2017-04-071-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | This fixes the segfault caused by solaris client in Gluster/NFS. Volname was not being parsed properly, Instead of volume name complete path was being used in nfs_mntpath_to_xlator(). Fixed it by striping volume name from complete path in nfs_mntpath_to_xlator(). Modified function name nfs3_funge_solaris_zerolen_fh() to nfs3_funge_webnfs_zerolen_fh() as zero-filled filehandle is specific to WebNFS. RFC : https://tools.ietf.org/html/rfc2055 Solaris uses WebNFS, the zero-filled FH is defined in the WebNFS spec. Logic was even added in fuction nfs3_funge_webnfs_zerolen_fh() to send subdir path in function glfs_resolve_at() instead of complete path for subdir mount. Change-Id: I19aae3547b8910e7ed4974ee5385424cab3e834a BUG: 1426667 Signed-off-by: Bipin Kunal <bkunal@redhat.com> Reviewed-on: https://review.gluster.org/16770 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: Dump graph upon initilization/switch for better debugging experienceKrutika Dhananjay2017-03-201-0/+2
| | | | | | | | | | | | | | | | | | | Having this information printed the way FUSE/nfs do would helps immensely with debugging experience, especially when a user/customer has added and removed bricks a couple of times. It helps in mapping the brick path to its corresponding protocol/client translator and in turn all the log messages that may be thrown by this protocol/client xlator. Change-Id: I0784e5fe932326fdbdc56f5a12b6951c7541a8cf Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/16919 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* rpc: avoid logging success on failureMilind Changire2017-03-071-5/+10
| | | | | | | | | | | | | | | | Avoid logging Success in the event of failure especially when errno has no meaningful value w.r.t. the failure. In this case the errno is set to zero when there's indeed a failure at the RPC level. Change-Id: If2cc81aa1e590023ed22892dacbef7cac213e591 BUG: 1426032 Signed-off-by: Milind Changire <mchangir@redhat.com> Reviewed-on: https://review.gluster.org/16730 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
* gfapi: OBS build fails in post build analysisKaleb S. KEITHLEY2017-02-231-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | Originally gfapi: create statedump when glusterd requests it When GlusterD sends the STATEDUMP procedure to the libgfapi client, the client checks if it matches the PID that should take the statedump. If so, it will do a statedump for the glfs_t that is connected to this mgmt connection. See https://bugzilla.redhat.com/show_bug.cgi?id=1169302#c25 for the OpenSuSE Build System post build analysis error. See Change-Id: I70d6a1f4f19d525377aebc8fa57f51e513b92d84 https://review.gluster.org/#/c/16415/ Change-Id: I7775f44ce13e20c831e8f1015816a28471d35bb4 BUG: 1169302 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/16722 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* refcount: typecast function for calling on freeNiels de Vos2017-01-311-7/+3
| | | | | | | | | | | | | | | | | | | All of the functions called to free the refcounted structure are doing a typecast from (void*) to their own type taht is being free'd. This really is not needed and the refcount interface is made a little simpler without the requirement of typecasting. With this small improvement in the API, all callers are updated too. Change-Id: I32473b6d1799f62861d4b2d78ea30c09e6c80ab1 BUG: 1416889 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/16471 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* core: run many bricks within one glusterfsd processJeff Darcy2017-01-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | This patch adds support for multiple brick translator stacks running in a single brick server process. This reduces our per-brick memory usage by approximately 3x, and our appetite for TCP ports even more. It also creates potential to avoid process/thread thrashing, and to improve QoS by scheduling more carefully across the bricks, but realizing that potential will require further work. Multiplexing is controlled by the "cluster.brick-multiplex" global option. By default it's off, and bricks are started in separate processes as before. If multiplexing is enabled, then *compatible* bricks (mostly those with the same transport options) will be started in the same process. Change-Id: I45059454e51d6f4cbb29a4953359c09a408695cb BUG: 1385758 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: https://review.gluster.org/14763 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* gfapi: create statedump when glusterd requests itNiels de Vos2017-01-241-0/+61
| | | | | | | | | | | | | | | | | | | | | When GlusterD sends the STATEDUMP procedure to the libgfapi client, the client checks if it matches the PID that should take the statedump. If so, it will do a statedump for the glfs_t that is connected to this mgmt connection. BUG: 1169302 Change-Id: I70d6a1f4f19d525377aebc8fa57f51e513b92d84 See-also: http://review.gluster.org/9228 Signed-off-by: Poornima G <pgurusid@redhat.com> [ndevos: separated patch from 9228] Reviewed-on: https://review.gluster.org/16415 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com>
* gfapi: add API to trigger events for debugging and troubleshootingNiels de Vos2017-01-195-5/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | Introduce glfs_sysrq() as a generic API for triggering debug and troubleshoot events. This interface will be used by the feature to get statedumps for applications using libgfapi. The current events that can be requested through this API are: - 'h'elp: log a mesage with all supported events - 's'tatedump: trigger a statedump for the passed glfs_t In future, this API can be used by a CLI to trigger statedumps from storage servers. At the moment it is limited to take statedumps, but it is extensible to set the log-level, clear caches, force reconnects and much more. BUG: 1169302 Change-Id: I18858359a3957870cea5139c79efe1365a15a992 Original-author: Poornima G <pgurusid@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/16414 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* Make getvolfile.py example pep8 compliantPatrick Uiterwijk2017-01-121-29/+30
| | | | | | | | | | | Change-Id: I7d3c1e871b9fa3859745f4d67acf05f16c7ccf5f BUG: 1412002 Signed-off-by: Patrick Uiterwijk <patrick@puiterwijk.org> Reviewed-on: http://review.gluster.org/16372 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: glfs_subvol_done should NOT wait for graph migration.Rajesh Joseph2016-11-293-14/+28
| | | | | | | | | | | | | | | | | | | | | | | In graph_setup function glfs_subvol_done is called which is executed in an epoll thread. glfs_lock waits on other thread to finish graph migration. This can lead to dead lock if we consume all the epoll threads. In general any call-back function executed in epoll thread should not call any blocking call which waits on a network reply either directly or indirectly, e.g. syncop functions should not be called in these threads. As a fix we should not wait for migration in the call-back path. Change-Id: If96d0689fe1b4d74631e383048cdc30b01690dc2 BUG: 1397754 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/15913 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: Fix memory leak in glfs-mgmtRajesh Joseph2016-11-281-0/+3
| | | | | | | | | | | | | | dictionary was not freed after serialization Change-Id: I495f2f823b0d53a0d858876bde41fde5f0705113 BUG: 1397177 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/15895 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-by: Poornima G <pgurusid@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* libglusterfs:Now mempool is added to ctx pool list under a lockRajesh Joseph2016-11-221-2/+0
| | | | | | | | | | | | | | | | | | | mempool is added to ctx pool list without any lock. This can cause undefined behaviour in case of multithreaded environment. Fix: modify the list only under ctx->lock Change-Id: I7bdbb3db48a899bb0e41427e149b13c0facaedba Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> BUG: 1394719 Reviewed-on: http://review.gluster.org/15842 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> Reviewed-by: Poornima G <pgurusid@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* gfapi: async fops should unref in callbacksRaghavendra Talur2016-11-051-18/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If fd is unref'd at the end of async call then the unref in cbks would lead to double unref and possible crash. Removing duplicate unrefs. Added unref only in failure cases. A simple test case has been added to test async write case. Need to extend the same for other async APIs too. Details: All glfd based calls in libgfapi, except for glfs_open and glfs_close, behave in the same way. At the start of the operation, they take a ref on glfd and fd. At the end of the operation, they unref it. Async calls are a little different as they unref in the cbk function. A successfull open call does not unref either the glfd or fd, thereby functioning as a reference for a OPEN file object. glfs_close makes a syncop_flush call sandwiched between a fd ref and unref(this can be removed, more on this below), followed by a call to glfs_mark_glfd_for_deletion which unrefs glfd and also calls glfs_fd_destroy as a release function thereby doing a unref on fd too. Functionally, there is no problem with how everything works when as described above. However, it is a little non-intuitive that we need to perform a fd_unref as a consequence of a implicit fd_ref that happens within glfs_resolve_fd. As we perform a GF_REF_GET(glfd) at the start of every operation, it would be worthwhile to remove the fd_ref that glfs_resovle_fd takes and do away with explicit fd_unref()s at the end of every operation. This is the same reason why we don't need the fd_ref in glfs_close. This is however not in the scope of this patch. Change-Id: I86b1d3b2ad846b16ea527d541dc82b5e90b0ba85 BUG: 1391086 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/15768 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Prasanna Kumar Kalever <pkalever@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* gfapi/upcall: Fix mismatch in few upcall API SYMVERSoumya Koduri2016-11-031-3/+3
| | | | | | | | | | | | | | | | | There is mismatch in few of the upcall API routine definitions and their corresponding symbol version declarations. Fixed the same. Change-Id: I2edfd9546a4c6a9128757f3b68e3ae4edd2c7a79 BUG: 1344714 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/15760 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Tested-by: Kaleb KEITHLEY <kkeithle@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: add glfs_free() to glfs.hNiels de Vos2016-10-183-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | Commit 4721188a154acd9a0a4c096d8d73e97f3bf1b2a9 introduces glfs_free() but missed adding the function to the header. The symbol is correctly available in the library though. Testcases do not seem to fail when a function is missing for the headers... The glusterfs-3.7.16 packages have been released with the missing declaration in the header and symbol-maps. Still, the function is available for applications: $ objdump -T usr/lib64/libgfapi.so.0 | grep -w glfs_free 0000000000006aa0 g DF .text 0000000000000035 GFAPI_3.7.16 glfs_free Change-Id: Ia707ee957f090dbfca028192fcc81a83dfdf4ae0 BUG: 1344714 Reported-by: Jiffin Tony Thottan <jthottan@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/15653 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* trivial: correct some spelling mistakes in comments and logsNiels de Vos2016-10-181-1/+1
| | | | | | | | | | | | | | BUG: 1385593 Change-Id: Icfae9e557a284182c6c22e9606fdd641528906f0 Reported-by: Patrick Matthäi <pmatthaei@debian.org> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/15656 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* gfapi: correct the gfapi.map for glfs_ipc@GFAPI_4_0_0Niels de Vos2016-10-131-1/+1
| | | | | | | | | | | | | | | | | Commit 85e959052148ec481823d55c8b91cdee36da2b43 introduced an inconsistency in gfapi.map. We need to figure out how to handle the glfs_ipc() function at one point... Change-Id: If53ad904318d5a60c14bd8b80685f7a852bf25e5 BUG: 1370931 Reported-by: Anoop C S <anoopcs@redhat.com> Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/15633 Reviewed-by: Anoop C S <anoopcs@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* gfapi: warn when glfs_realpath() returned malloc'd memoryNiels de Vos2016-10-115-10/+50
| | | | | | | | | | | | | | | | | | | | | | | | glfs_realpath() may return memory allocated with malloc(). Depending on the memory allocator that the application uses, calling free() on the returned string can cause segmentation faults or other problems. Functions that allocate memory, need to match the free'ing of the same memory allocator and memory accounting. glibc/malloc and jemalloc/free do not match together (other allocators could probably trigger these problems as well). Applications need to provide a pre-allocated buffer, or in case glfs_realpath() allocates the memory, glfs_free() should be used to free it. Change-Id: I5d721a7425674aa700db8a7a436cbedb95a5927f BUG: 1370931 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/15332 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* gfapi: redesign the public interface for upcall consumersNiels de Vos2016-09-287-98/+424
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The glfs_callback_arg and glfs_callback_inode_arg were allocated by gfapi, and expected to be free()'d by the application. However it is not reasonable to expect that applications use the same memory allocator to as the compiled libgfapi.so. For instance, it is possible that gfapi uses glibc malloc/free, and an application like NFS-Ganesha the versions from jemalloc. Mismatching of the malloc() and free() functions causes segmentation faults at best. In order to prevent problems like this in the future, the API for applications that consume upcalls has been remodeled. Any of the structures that gfapi allocates, should be free'd with glfs_free(). The members of the structures can not be accessed directly anymore, each has its own function to access now. Correcting the naming of the functions, structures and constants is a continuation of commit 2775dc64101ed37c8d9809bf9852dbf0746ee2b6. These new improvements not only have correct prefixes for the functions and structures, the naming also reflects more to the upcall framework and does not use "callback" anymore. Change-Id: I2b8bd5a0a82036d2abea1a217f5e5975a1d4fe93 BUG: 1344714 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/14701 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
* gfapi: Fix few fd ref leaksSoumya Koduri2016-09-261-0/+21
| | | | | | | | | | | | | | | | From the code inspection, have observed that there are fd ref leaks for few fd-based gfapi fops. 'glfs_resolve_fd' returns a fd (either existing or migrated) with an extra ref taken. This needs to be unref'ed at the end of the operation. Change-Id: Id63394e3e7deafb0c8b06444f2ae847248b126db BUG: 1379285 Signed-off-by: Soumya Koduri <skoduri@redhat.com> Reviewed-on: http://review.gluster.org/15573 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org>
* build: out-of-tree builds generates files in the wrong directoryKaleb S KEITHLEY2016-09-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | And minor cleanup of a few of the Makefile.am files while we're at it. Rewrite the make rules to do what xdrgen does. Now we can get rid of xdrgen. Note 1. netbsd6's sed doesn't do -i. Why are we still running smoke tests on netbsd6 and not netbsd7? We barely support netbsd7 as it is. Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with libglusterfs? A cut-and-paste mistake? It has no references to symbols in libglusterfs. Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_ regex that matches the same lines as the _extended_ regex "/#(ifndef|define|endif)/". To match the extended regex sed needs to be run with -r on Linux; with -E on *BSD. However NetBSD's and FreeBSD's sed helpfully also provide -r for compatibility. Using a basic regex avoids having to use a kludge in order to run sed with the correct option on OS X. Note 4. Not copying the bit of xdrgen that inserts copyright/license boilerplate. AFAIK it's silly to pretend that machine generated files like these can be copyrighted or need license boilerplate. The XDR source files have their own copyright and license; and their copyrights are bound to be more up to date than old boilerplate inserted by a script. From what I've seen of other Open Source projects -- e.g. gcc and its C parser files generated by yacc and lex -- IIRC they don't bother to add copyright/license boilerplate to their generated files. It appears that it's a long-standing feature of make (SysV, BSD, gnu) for out-of-tree builds to helpfully pretend that the source files it can find in the VPATH "exist" as if they are in the $cwd. rpcgen doesn't work well in this situation and generates files with "bad" #include directives. E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`, you get an #include directive in the generated .c file like this: ... #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h" ... which (obviously) results in compile errors on out-of-tree build because the (generated) header file doesn't exist at that location. Compared to `rpcgen ./glusterfs3-xdr.x` where you get: ... #include "glusterfs3-xdr.h" ... Which is what we need. We have to resort to some Stupid Make Tricks like the addition of various .PHONY targets to work around the VPATH "help". Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/... looks exactly like -I$(top_srcdir)/rpc/xdr/... Don't be fooled though. And don't delete the -I$(top_builddir)/rpc/xdr/... bits Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e BUG: 1330604 Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com> Reviewed-on: http://review.gluster.org/14085 Tested-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* md-cache: Register the list of xattrs with cache-invalidationPoornima G2016-08-301-22/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Issue: md-cache caches a specified list of xattrs, and when cache invalidation is enabled, it makes sense to recieve invalidation only when those xattrs are modified by other clients. But the current implementation of upcall is that, it will send invalidation when any of the on-disk xattrs is modified. Solution: md-cache sends a list of xattrs that it is interested in, to upcall by issuing an ipc(). The challenge here is to make sure everytime a brick goes offline and comes back up, the ipc() needs to be issued to the bricks. Hence ipc() is sent from md-cache every time there is a CHILD_UP/CHILD_MODIFIED event. TODO: There will be patches following, in cluster xlators, to implement ipc fop. Change-Id: I6efcf3df474f5ce6eabd3d6694c00c7bd89bc25d BUG: 1211863 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/15002 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* gfapi: SSL connection for mgmt connection is not workingRajesh Joseph2016-08-281-0/+5
| | | | | | | | | | | | | | | | | Problem: libgfapi does not enable SSL on mgmt connection. Fix: Enable SSL when it is enabled on mgmt connection is enabled, i.e. presence of /var/lib/glusterd/secure-access file Change-Id: I1ce4935b04e6140aeab819e42076defd580b0727 BUG: 1362602 Signed-off-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-on: http://review.gluster.org/15073 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Kaushal M <kaushal@redhat.com>
* gfapi: do not cache upcalls if the application is not interestedNiels de Vos2016-08-253-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the volume option 'features.cache-invalidation' is enabled, upcall events are sent from the brick process to the client. Even if the client is not interested in upcall events itself, md-cache or other xlators may benefit from them. By adding a new 'cache_upcalls' boolean in the 'struct glfs', we can enable the caching of upcalls when the application called glfs_h_poll_upcall(). NFS-Ganesha sets up a thread for handling upcalls in the initialization phase, and calls glfs_h_poll_upcall() before any NFS-client accesses the NFS-export. In the future there will be a more flexible registration API for enabling certain kind of upcall events. Until that is available, this should work just fine. Verificatio of this change is not trivial within our current regression test framework. The bug report contains a description on how to reliably reproduce the problem with the glusterfs-coreutils. Change-Id: I818595c92db50e6e48f7bfe287ee05103a4a30a2 BUG: 1368842 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/15191 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* gfapi: add missing glfs_truncateJeff Darcy2016-08-114-4/+52
| | | | | | | | | | | | Change-Id: I80b016090a4d9d86278a0a5144dd58c0cbfe9bb2 BUG: 1365489 Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Signed-off-by: Oleksandr Natalenko <oleksandr@natalenko.name> Reviewed-on: http://review.gluster.org/13927 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
* gfapi: use const qualifier for glfs_*timens()Oleksandr Natalenko2016-08-102-6/+8
| | | | | | | | | | | | | | | | | | | | glfs_*timens() functions have the last argument of struct timespec type that is supposed to be const. Now using glfs_*timens() in fops, implemented on top of FUSE library, leads to compiler-time warning about discarding const qualifier. Introducing const qualifier does not break ABI, so let's just fix it. Change-Id: Iea2a1018de7dce67f67a8229671a5978246de800 BUG: 1365506 Signed-off-by: Oleksandr Natalenko <oleksandr@natalenko.name> Reviewed-on: http://review.gluster.org/15119 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* gfapi: Fix IO error caused when there is consecutive graph switchesPoornima G2016-08-103-6/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is part 2 of the fix, the part 1 can be found at: http://review.gluster.org/#/c/14656/ Problem: ======= Consider a race between, __glfs_active_subvol() and graph_setup(). Lets say @TIME T1: fs->active_subvol = A fs->next_subvol = B __glfs_active_subvol() //under lock fs->mutex { .... new_subvol = fs->next_subvol //which is B .... //Start migration from A to B __glfs_first_lookup(){ .... unlock fs->mutex //@TIME T2 network fop lock fs->mutex .... } .... //migration continue on B fs->active_subvol = fs->next_subvol //which is C (explained below) .... } @Time T2, lets say in another thread, graph_setup() is called with C, note that at T2, fs->mutex is unlocked. graph_stup(C...) { lock fs->mutex .... if (fs->next_subvol) // which is B destroy subvol (fs->next_subvol) .... fs->next_subvol = C .... unlock fs->mutex } Thus at the end of this, fs->old_subvol = A; fs->active_subvol = C; fs->next_subvol = NULL; which is wrong, as B completed migration, but was destroyed by graph_setup, and C never was migrated. Solution: ========= Any new graph can be in one of the 2 states: - Picked for migration, migration in progress (fs->mip_subvol) - Not picked so far for migration (fs->next_subvol) graph_setup() updates fs->next_subvol only, __glfs_active_subvol() moves fs->next_subvol to fs->mip_subvol and fs->next_subvol = NULL atomically, and then once the migration is complete, make that the fs->active_subvol Change-Id: Ib6ff0565105c5eedb912a43da4017cd413243612 BUG: 1343038 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: http://review.gluster.org/14722 NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-by: Rajesh Joseph <rjoseph@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-19 07:15:32 +0000