summaryrefslogtreecommitdiffstats
path: root/libglusterfs
Commit message (Collapse)AuthorAgeFilesLines
* core: FreeBSD has pthread_set_name_np() (versus pthread_setname_np())Kaleb S. KEITHLEY2018-06-111-2/+8
| | | | | | | | | | And has had it since at least FreeBSD 9.0 Reported-by: Roman Serbski <mefystofel@gmail.com> Change-Id: I52cfde7f2f7a82d0e66465ac392ed7e201e1653b BUG: 1576816 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* libglusterfs: fix comparison of a NULL dict with a non-NULL dictXavi Hernandez2018-04-241-8/+8
| | | | | | | | | | | | | Function are_dicts_equal() had a bug when the first argument was NULL and the second one wasn't NULL. In this case it incorrectly returned that the dicts were different when they could be equal. Backport of: > BUG: 1566732 BUG: 1569407 Change-Id: I0fc245c2e7d1395865a76405dbd05e5d34db3273 Signed-off-by: Xavi Hernandez <xhernandez@redhat.com>
* timer: Fix possible race during cleanupSoumya Koduri2018-04-101-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As mentioned in bug1509189, there is a possible race between gf_timer_cancel(), gf_timer_proc() and gf_timer_registry_destroy() leading to use_after_free. Problem: 1) gf_timer_proc() is called, locks reg, and gets an event. It unlocks reg, and calls the callback. 2) Meanwhile gf_timer_registry_destroy() is called, and removes reg from ctx, and joins on gf_timer_proc(). 3) gf_timer_call_cancel() is called on the event being processed. It cannot find reg (since it's been removed from reg), so it frees event. 4) the callback returns into gf_timer_proc(), and it tries to free event, but it's already free, so double free. Solution: The fix is to bail out in gf_timer_cancel() when registry is not found. The logic behind this is that, gf_timer_cancel() is called only on any existing event. That means there was a valid registry earlier while creating that event. And the only reason we cannot find that registry now is that it must have got set to NULL when context cleanup is started. Since gf_timer_proc() takes care of releasing all the remaining events active on that registry, it seems safe to bail out in gf_timer_cancel(). master https://review.gluster.org/18652 master BZ: 1509189 Change-Id: Ia9b088533141c3bb335eff2fe06b52d1575bb34f BUG: 1565590 Reported-by: Daniel Gryniewicz <dang@redhat.com> Signed-off-by: Soumya Koduri <skoduri@redhat.com> Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com>
* rpc: Showing some unusual timer error logs during brick stopMohit Agrawal2018-02-121-15/+3
| | | | | | | | | | | | | | Solution: Update msg condition in gf_timer_call_after function to avoid the message > BUG: 1538427 > Change-Id: I849e8e052a8259cf977fd5e7ff3aeba52f9b5f27 > Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> > (cherry picked from commit c142d26e44436d805e476f2d13ac8726052a59c4) BUG: 1542934 Change-Id: I849e8e052a8259cf977fd5e7ff3aeba52f9b5f27 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
* cluster/dht: Add migration checks to dht_(f)xattropN Balachandran2018-02-061-0/+1
| | | | | | | | | | | | | | | | The dht_(f)xattrop implementation did not implement migration phase1/phase2 checks which could cause issues with rebalance on sharded volumes. This does not solve the issue where fops may reach the target out of order. > Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c > BUG: 1471031 > Signed-off-by: N Balachandran <nbalacha@redhat.com> Change-Id: I2416fc35115e60659e35b4b717fd51f20746586c BUG: 1540224 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* core/memacct: save allocs in mem_acct_rec listN Balachandran2017-12-083-1/+42
| | | | | | | | | | | | | | | | | | | | With configure --enable-debug, add all object allocations to a list in the corresponding mem_acct_rec. This allows us to see all objects of a particular type and allows for additional debugging in case of memory leaks. This is not compiled in by default and must be explicitly enabled. It is intended to be used by developers. > Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8 > BUG: 1522662 > Signed-off-by: N Balachandran <nbalacha@redhat.com> (cherry picked from commit 47d01546a1826dc14a8331ea8700015f1cfdc4db) Change-Id: I7cf2dbeadecf994423d7e7591e85f18d2575cce8 BUG: 1523455 Signed-off-by: N Balachandran <nbalacha@redhat.com>
* glusterd : introduce timer in mgmt_v3_lockGaurav Yadav2017-11-062-1/+2
| | | | | | | | | | | | | | | | | | Problem: In a multinode environment, if two of the op-sm transactions are initiated on one of the receiver nodes at the same time, there might be a possibility that glusterd may end up in stale lock. Solution: During mgmt_v3_lock a registration is made to gf_timer_call_after which release the lock after certain period of time >mainline patch : https://review.gluster.org/#/c/18437/ Change-Id: I16cc2e5186a2e8a5e35eca2468b031811e093843 BUG: 1503239 Signed-off-by: Gaurav Yadav <gyadav@redhat.com>
* build: make it possible to build cleanly 2x in a rowNiels de Vos2017-10-251-1/+2
| | | | | | | | | | | | | | | | | | | | | | | 'make clean' does not cleanup everything, and some of the files get cleaned too eagerly. Several files are being packaged in a 'make dist' tarball, that get rebuild each time anyway. Specifically, this change prevents - libglusterfs/src/generator.pyc from laying around - keeping rpc/xdr/gen/*.x symlinks - modifying tests/basic/{fuse,gfapi}/Makefile each run - including tests/env.rc and events/src/eventtypes.py in the tarball Cherry picked from commit a413c6353dbfff5221ea020ff8e1475d5ee17a81: > Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82 > BUG: 1501317 > Reported-by: Patrick Matthäi <pmatthaei@debian.org> > Signed-off-by: Niels de Vos <ndevos@redhat.com> Change-Id: I774dd1abf3a9d3b6a89b938cf6ee7d7792c59a82 BUG: 1494527 Reported-by: Patrick Matthäi <pmatthaei@debian.org> Signed-off-by: Niels de Vos <ndevos@redhat.com>
* protocol-auth: use the proper validation methodAmar Tumballi2017-10-252-2/+7
| | | | | | | | | | Currently, server protocol's init and glusterd's option validation methods are different, causing an issue. They should be same for having consistent behavior Change-Id: Ibbf9a18c7192b2d77f9b7675ae7da9b8d2fe5de4 BUG: 1501315 Signed-off-by: Amar Tumballi <amarts@redhat.com>
* cluster/ec: Improve performance with xattrop updateSunil Kumar Acharya2017-10-123-56/+126
| | | | | | | | | | | | | | | | Existing EC code updates the xattr on the subvolume in a sequential pattern resulting in very poor performance. With this fix EC now updates the xattr on the subvolume in parallel which improves the xattr update performance. >BUG: 1445663 >Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486 >Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> BUG: 1499150 Change-Id: I3fc40d66db0b88875ca96a9fa01002ba386c0486 Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com>
* glusterd: fix client io-threads option for replicate volumesRavishankar N2017-10-091-1/+3
| | | | | | | | | | | | | | | | | | | | | Backport of https://review.gluster.org/#/c/18430/ Problem: Commit ff075a3d6f9b142911d25c27fd209838782bfff0 disabled loading client-io-threads for replicate volumes (it was set to on by default in commit e068c1997314046658dd502e9118dab32decf879) due to performance issues but in doing so, inadvertently failed to load the xlator even if the user explicitly enabled the option using the volume set command. This was despite returning returning sucess for the volume set. Fix: Modify the check in perfxl_option_handler() and add checks in volume create/add-brick/remove-brick code paths, tying it all to GD_OP_VERSION_3_12_2. Change-Id: Ib612973a999a7da818cc926f5c2601b1f0794fcf BUG: 1499158 Signed-off-by: Ravishankar N <ravishankar@redhat.com>
* mount/fuse: Make event-history feature configurableKrutika Dhananjay2017-10-051-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | ... and disable it by default. Backport of: > Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485 > Reviewed-on: https://review.gluster.org/18242 > BUG: 1467614 > cherry-picked from commit 956d43d6e89d40ee683547003b876f1f456f03b6 This is because having it disabled seems to improve performance. This could be due to the lock contention by the different epoll threads on the circular buff lock in the fop cbks just before writing their response to /dev/fuse. Just to provide some data - wrt ovirt-gluster hyperconverged environment, I saw an increase in IOPs by 12K with event-history disabled for randrom read workload. Usage: mount -t glusterfs -o event-history=on $HOSTNAME:$VOLNAME $MOUNTPOINT OR glusterfs --event-history=on --volfile-server=$HOSTNAME --volfile-id=$VOLNAME $MOUNTPOINT Change-Id: Ia533788d309c78688a315dc8cd04d30fad9e9485 BUG: 1495397 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* event/epoll: don't call handler for events received after a pollerrRaghavendra G2017-09-111-6/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | we register socket with EPOLLONESHOT, which means it has to be explicitly added back through epoll_ctl to receive more events. Normally we do this once the handler completes processing of current event. But event_select_on_epoll is one asynchronous codepath where socket can be added back for polling while an event on the same socket is being processed. event_select_on_epoll has a check whether an event is being processed in the form of slot->in_handler. But this check is not sufficient enough to prevent parallel events as slot->in_handler is not atomically incremented with respect to reception of the event. This means following imaginary sequence of events can happen: * epoll_wait returns with a POLLERR - say POLLERR1 - on a socket (sock1) associated with slot s1. socket_event_handle_pollerr is yet to be invoked. * an event_select_on called from __socket_ioq_churn which was called in request/reply/msg submission codepath (as opposed to __socket_ioq_churn called as part of POLLOUT handling - we cannot receive a POLLOUT due to EPOLLONESHOT) adds back sock1 for polling. * since sock1 was added back for polling in step 2 and our polling is level-triggered, another thread picks up another POLLERR event - say POLLERR2. socket_event_handler is invoked as part of processing POLLERR2 and it completes execution setting priv->sock to -1. * event_unregister_epoll called as part of __socket_reset due to POLLERR1 would receive fd as -1 resulting in assert failure. Also, since the first pollerr event has done rpc_transport_unref, subsequent parallel events (not just pollerr, but other events too) could be acting on a freed up transport too. >Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb >BUG: 1486134 >Signed-off-by: Raghavendra G <rgowdapp@redhat.com> >Reviewed-on: https://review.gluster.org/18129 >Smoke: Gluster Build System <jenkins@build.gluster.org> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: mohammed rafi kc <rkavunga@redhat.com> (cherry picked from commit b1b49997574eeb7c6a42e6e8257c81ac8d2d7578) Change-Id: I5db755068e7890ec755b59f7a35a57da110339eb BUG: 1489296 Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-on: https://review.gluster.org/18223 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
* perf/qr: Use a ref-ed data to extract contentN Balachandran2017-09-072-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | qr_content_extract used dict_get to get the value of the GF_CONTENT_KEY key. dict_get does not ref the data before returning it so QR could be acting on freed memory if another thread deletes the key before then. This patch also fixes a race in dict_get_with_ref. Fix: Use dict_get_with_ref to retrieve the file contents. > BUG: 1484709 > Signed-off-by: N Balachandran <nbalacha@redhat.com> > Reviewed-on: https://review.gluster.org/18115 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Amar Tumballi <amarts@redhat.com> > Tested-by: Raghavendra G <rgowdapp@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: Ib1a7a70bb92eed7e70747ec530e0b3edc53127ec BUG: 1486538 (cherry picked from commit 414d3e92fc56f08e320a3aa65b6b18e65b384551) Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/18145 Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: MOHIT AGRAWAL <moagrawa@redhat.com>
* glusterd: glusterd fails to start if peers file has blank lineGaurav Yadav2017-08-291-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Problem: On start of glusterd service, glusterd fetch data from store, while parsing data from store if peers file consists of blank line glusterd fails to start. Fix: With this fix while parsing peers file glusterd will skip blank lines if it contains any. >Reviewed-on: https://review.gluster.org/18066 >Tested-by: Gaurav Yadav <gyadav@redhat.com> >Smoke: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Prashanth Pai <ppai@redhat.com> >CentOS-regression: Gluster Build System <jenkins@build.gluster.org> >Reviewed-by: Atin Mukherjee <amukherj@redhat.com> >Reviewed-by: Niels de Vos <ndevos@redhat.com> Change-Id: I53cd65a54de5f57baef292b2118b70ffb7f99388 BUG: 1486107 Signed-off-by: Gaurav Yadav <gyadav@redhat.com> Reviewed-on: https://review.gluster.org/18124 Reviewed-by: Prashanth Pai <ppai@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: Gluster should keep PID file in correct locationGaurav Kumar Garg2017-08-121-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently Gluster keeps process pid information of all the daemons and brick processes in Gluster configuration file directory (ie., /var/lib/glusterd/*). These pid files should be seperate from configuration files. Deletion of the configuration file directory might result into serious problems. Also, /var/run/gluster is the default placeholder directory for pid files. So, with this fix Gluster will keep all process pid information of all processes in /var/run/gluster/* directory. > BUG: 1258561 > Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com> > Signed-off-by: Saravanakumar Arumugam <sarumuga@redhat.com> > Reviewed-on: https://review.gluster.org/13580 > Tested-by: MOHIT AGRAWAL <moagrawa@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> > cherry pick from commit 220d406ad13d840e950eef001a2b36f87570058d BUG: 1480459 Change-Id: Idb09e3fccb6a7355fbac1df31082637c8d7ab5b4 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com> Reviewed-on: https://review.gluster.org/18023 Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi: Duplicate the buffer sent in setxattr callsPoornima G2017-08-122-3/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | Issue: The caller of glfs_setxattr sends a buffer to set as the value. We create a dict in which the pointer to the value is set. Underlying layers like md-cache take a ref on this dict to store the value for a longer time. But the moment setxattr is complete, the caller of glfs_setxattr can free the value memory. Solution: memcpy the setxattr value to the gluster buffer. > Reviewed-on: https://review.gluster.org/17967 > Reviewed-by: soumya k <skoduri@redhat.com> > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> > (cherry picked from commit e11296f8e52b7e3b13d21b41d4fa34baea878edf) Change-Id: I58753fe702e8b7d0f6c4f058714c65d0ad5d7a0a BUG: 1479655 Signed-off-by: Poornima G <pgurusid@redhat.com> Reviewed-on: https://review.gluster.org/18001 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterfsd: allow subdir mountAmar Tumballi2017-08-044-8/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changes: 1. Take subdir mount option in client (mount.gluster / glusterfsd) 2. Pass the subdir mount to server-handshake (from client-handshake) 3. Handle subdir-mount dir's lookup in server-first-lookup and handle all fops resolution accordingly with proper gfid of subdir 4. Change the auth/addr module to handle the multiple subdir entries in option, and valid parsing. How to use the feature: `# mount -t glusterfs $hostname:/$volname/$subdir /$mount_point` Or `# mount -t glusterfs $hostname:/$volname -osubdir_mount=$subdir /$mount_point` Option can be set like: `# gluster volume set <volname> auth.allow "/subdir1(192.168.1.*),/(192.168.10.*),/subdir2(192.168.8.*)"` Updates #175 > Reviewed-At: https://review.gluster.org/17141/ Change-Id: I7ea57f76ddbe6c3862cfe02e13f89e8a39719e11 Signed-off-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: https://review.gluster.org/17968 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* glusterd: Add geo-replication session details to get-state outputSamikshan Bairagya2017-08-043-2/+44
| | | | | | | | | | | | | | | | | | | | | | This commit adds support to the get-state CLI to capture details on geo-replication session as obtained in `gluster volume geo-replication status detail` in its output. > Reviewed-on: https://review.gluster.org/17941 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Shubhendu Tripathi <shtripat@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Atin Mukherjee <amukherj@redhat.com> (cherry picked from commit 2e7daeffef05c6100cbcc39f1be62935711db3eb) Fixes: #291 Change-Id: I2fbcba70bfdaf439522637234805545194777ed4 Signed-off-by: Samikshan Bairagya <samikshan@gmail.com> Reviewed-on: https://review.gluster.org/17971 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* libglusterfs: Correct op-version for 3.12ShyamsundarR2017-08-031-3/+1
| | | | | | | | | Change-Id: Ie7f622f255965dee070f149ee48e16ff070502b7 BUG: 1477381 Signed-off-by: ShyamsundarR <srangana@redhat.com> Reviewed-on: https://review.gluster.org/17951 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* logging: localtime logging, cmdline, volume set optionN Balachandran2017-08-034-2/+36
| | | | | | | | | | | | | | | | | | Despite the fact that appliances generally use UTC, some users really want log entries in localtime. fixes gluster/glusterfs#272 feature page: https://review.gluster.org/17807 Backport from master https://review.gluster.org/#/c/16911/ Change-Id: I5fbf2c3eedd9eb128fb3f851dd67b2f4081c8bba Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/17928 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* libglusterfs: the global_xlator should have valid cbksNiels de Vos2017-08-021-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a case where Gluster/NFS needs to resolve a path outside of the nfs-xlator itself. While resolving the path to fetch the GFID for creating the NFS-filehandle, gfapi may set an inode-ctx through glfs_resolve_at(). This inode-ctx is linked with the global_xlator. Because the global_xlator does not have any cbks, loc_wipe() will cause a segfault when it calls inode_unref() and xl->cbks->forget(). It is assumed that all xlators have a cbks symbol, otherwise loading of the xlator will fail. The global_xlator is not loaded in the same way, so there is no failure noticed when the instance is created. By adding an empty `struct xlator_cbks`, the global_xlator behaves similat to other xlators that do not implement all callbacks. I would have preferred to keep the inode-ctx setting through glfs_resolve_at() contained within Gluster/NFS. Unfortunately Gluster/NFS also uses the inode-ctx, and is not prepared to see the values that glfs_resolve_at() stores there. This problem is not easily reproducible because it involves mounting over WebNFS (like Solaris 10 can do). The segfault will also not be immediate, unless the following is done: 1. create a subdir on a volume 2. mount the volume/subdir over WebNFS 3. unmount the volume/subdir 4. mount the root of the volume 5. delete the subdir on the volume -> segfault of Gluster/NFS Cherry picked from commit cec5036f7e99ae265bb5e0e7f3df30166466eb2c: > Change-Id: I2bd71d033e97edc07ba93b2d4ada558f65d68999 > BUG: 1468291 > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: https://review.gluster.org/17897 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Amar Tumballi <amarts@redhat.com> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Change-Id: I2bd71d033e97edc07ba93b2d4ada558f65d68999 BUG: 1477190 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17946 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* mem-pool: track and verify initialization stateNiels de Vos2017-07-311-8/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is possible that pthread_getspecific() returns a non-NULL value in case the pthread_key_t is not initialized. The behaviour for pthread_getspecific() is not defined in this case. This can happen when applications use mem-pools from libglusterfs.so, but did not call mem_pools_init_early(). By tracking the status of the mem-pools initialization, it is now possible to prevent calling pthread_getspecific() in case the pthread_key_t is not initialized. In future, we might want to exend this more to faciliate debugging. Cherry picked from commit b5fa5ae05f73e03023db37e43fb203267b719160) > Reported-by: Kotresh HR <khiremat@redhat.com> > Tested-by: Jiffin Tony Thottan <jthottan@redhat.com> > Change-Id: I6255419fe05792dc78b1eaff55bc008fc5ff3933 > Fixes: 1e8e62640 ("mem-pool: initialize pthread_key_t pool_key in mem_pool_init_early()") > BUG: 1475255 > Signed-off-by: Niels de Vos <ndevos@redhat.com> > Reviewed-on: https://review.gluster.org/17899 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> > Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Change-Id: I6255419fe05792dc78b1eaff55bc008fc5ff3933 BUG: 1475258 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17915 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* storage/posix: Add virtual xattr to fetch path from gfidKotresh HR2017-07-311-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The gfid2path infra stores the "pargfid/bname" as on xattr value for each non directory entry. Hardlinks would have a separate xattr. This xattr key is internal and is not exposed to applications. A virtual xattr is exposed for the applications to fetch the path from gfid. Internal xattr: trusted.gfid2path.<xxhash> Virtual xattr: glusterfs.gfidtopath getfattr -h -n glusterfs.gfidtopath /<aux-mnt>/.gfid/<gfid> If there are hardlinks, it returns all the paths separated by ':'. A volume set option is introduced to change the delimiter to required string of max length 7. gluster vol set gfid2path-separator ":::" > Updates: #139 > Change-Id: Ie3b0c3fd8bd5333c4a27410011e608333918c02a > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: https://review.gluster.org/17785 > Smoke: Gluster Build System <jenkins@build.gluster.org> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com> Updates: #139 Change-Id: Ie3b0c3fd8bd5333c4a27410011e608333918c02a Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17921 CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* posix/gfid2path: Block access to gfid2path xattr via mountKotresh HR2017-07-311-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | gfid2path xattr is an internal xattr and should not be allowed to modify by other applications from gluster mount. This patch blocks the same. > Updates: #139 > Change-Id: Id2cb29797ee1bd77e0e0d2203a47469fd7203355 > Signed-off-by: Kotresh HR <khiremat@redhat.com> > Reviewed-on: https://review.gluster.org/17744 > Smoke: Gluster Build System <jenkins@build.gluster.org> > Reviewed-by: Prashanth Pai <ppai@redhat.com> > Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> > Reviewed-by: Aravinda VK <avishwan@redhat.com> > CentOS-regression: Gluster Build System <jenkins@build.gluster.org> (cherry picked from commit 96eece8abbb9c06f0b91f37e718ac9e337a3f714) Updates: #139 Change-Id: Id2cb29797ee1bd77e0e0d2203a47469fd7203355 Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17869 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Aravinda VK <avishwan@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* mem-pool: free objects from pools on mem_pools_fini()Niels de Vos2017-07-202-2/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using a minimal gfapi application that only initializes a small graph (sink, shard and meta xlators) the following memory leaks are reported by Valgrind: HEAP SUMMARY: in use at exit: 322,976 bytes in 75 blocks total heap usage: 684 allocs, 609 frees, 2,092,116 bytes allocated With this change, the mem-pools are cleaned up on calling of mem_pools_fini() and the objects in the pool are free'd. HEAP SUMMARY: in use at exit: 315,265 bytes in 58 blocks total heap usage: 684 allocs, 626 frees, 2,092,079 bytes allocated This information was gathered with `./run-xlator.sh features/shard` that comes with `gfapi-load-volfile` from gluster-debug-tools. While working on the free'ing of the per_thread_pool_list_t structures, it became apparent that GF_CALLOC() in mem_get_pool_list() gets redirected to a standard calloc() without prepending the Gluster specific memory header. This is because mem_pools_init() gets called before THIS->ctx is valid, so it is not possible to check if memory accounting is enabled or not. Because of this, the GF_CALLOC() call in mem_get_pool_list() has been replaced by CALLOC() to prevent potential mismatches between the allocation/free'ing of per_thread_pool_list_t structures. Change-Id: Id6f558816f399b0c613d74df36deac2300b6dd98 BUG: 1470170 URL: https://github.com/gluster/gluster-debug-tools Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17768 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> Reviewed-by: soumya k <skoduri@redhat.com>
* mem-pool: initialize pthread_key_t pool_key in mem_pool_init_early()Niels de Vos2017-07-192-16/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is not possible to call pthread_key_delete for the pool_key that is intialized in the constructor for the memory pools. This makes it difficult to do a full cleanup of all the resources in mem_pools_fini(). For this, the initialization of pool_key should be moved to mem_pool_init(). However, the glusterfsd binary has a rather complex initialization procedure. The memory pools need to get initialized partially to get mem_get() functionality working. But, the pool_sweeper thread can get killed in case it is started before glusterfsd deamonizes. In order to solve this, mem_pools_init() is split into two pieces: 1. mem_pools_init_early() for initializing the basic structures 2. mem_pools_init_late() to start the pool_sweeper thread With the split of mem_pools_init(), and placing the pthread_key_create() in mem_pools_init_early(), it is now possible to correctly cleanup the pool_key with pthread_key_delete() in mem_pools_fini(). It seems that there was no memory pool initialization in the CLI. This has been added as well now. Without it, the CLI will not be able to call mem_get() successfully which results in a hang of the process. Change-Id: I1de0153dfe600fd79eac7468cc070e4bd35e71dd BUG: 1470170 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17779 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* libglusterfs: Name threads on creationRaghavendra Talur2017-07-198-17/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Set names to threads on creation for easier debugging. Output of top -H -p <PID-OF-GLUSTERFSD> Before: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterfsd 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd After: 19773 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19774 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustertimer 19775 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterfsd 19776 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustermemsweep 19777 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc0 19778 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glustersproc1 19779 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll0 19780 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteridxwrker 19781 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusteriotwr0 19782 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrssign 19783 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterbrswrker 19784 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterclogecon 19785 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd0 19786 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd1 19787 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.01 glusterclogd2 19789 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixjan 19790 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixfsy 25178 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll1 5398 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterepoll2 7881 root 20 0 1301.3m 12.6m 8.4m S 0.0 0.1 0:00.00 glusterposixhc Change-Id: Id5f333755c1ba168a2ffaa4fce6e71c375e10703 BUG: 1254002 Updates: #271 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: https://review.gluster.org/11926 Reviewed-by: Prashanth Pai <ppai@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* common-utils: Remove fop_enum_to_string, get_fop_intPranith Kumar K2017-07-183-66/+15
| | | | | | | | | | | | | Implementation of these two functions becomes easier by using gf_fop_list[] array. So implemented that and removed usage of these functions. BUG: 1472250 Change-Id: I8a592913f9eeb02d965708bcf28a637588ed4988 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17812 Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org>
* storage/posix: Don't allow gfid/volume-id xattr to be removedPranith Kumar K2017-07-182-0/+30
| | | | | | | | | | | | | | | | | | | | | | | Problem: Bulk xattr removal doesn't check if the xattrs that are coming in xdata have gfid/volume-id xattrs, so there is potential for bulkremovexattr removing gfid/volume-id. I also observed that bulkremovexattr is not available for fremovexattr. Fix: Do proper checks in bulk removexattr to remove gfid/volume-id. Refactor [f]removexattr to reduce the differences. BUG: 1470489 Change-Id: Ia845b31846a149500111c0996646e648f72cdce6 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17765 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Anuradha Talur <atalur@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
* mem-pool: remove references to unused ctx->mempool_listNiels de Vos2017-07-142-1/+5
| | | | | | | | | | | | | | | | Only the old (removed) implementation of mem-pools provided access to the ctx->mempool_list to faciliate state-dumps. The usage of the mempool_list has been disabled with "#if defined(OLD_MEM_POOLS)", but a few occurences were missed. Change-Id: I912fb63830efc06247eb0c6551d198271ee57a86 BUG: 1470170 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: https://review.gluster.org/17778 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* fuse: memory leak fixesDanny Couture2017-07-141-0/+2
| | | | | | | | | | | | | | | | | | | Fix fuse ctx memory leak in case an error occurs and the cleanup path is different than usual. Also fix a memory leak in logging if eh_save_history() fails. Change-Id: I7ec967c807b0ed91184e5b958be70702215c46c9 BUG: 1470220 Signed-off-by: Danny Couture <couture.danny@gmail.com> Reviewed-on: https://review.gluster.org/17759 Reviewed-by: Niels de Vos <ndevos@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: N Balachandran <nbalacha@redhat.com> Reviewed-by: Prashanth Pai <ppai@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
* core: miscellaneous cleanupKaleb S. KEITHLEY2017-07-141-145/+158
| | | | | | | | | | | | | | | | | | | | | | | clean up things that I tripped over doing other changes. 1) fix mishmash of random spacing in struct decls in glusterfs.h. Not technically a problem, just ugly to look at. 2) replace open-coded strings constants with existing #define constants. A disaster waiting to happen. 3) Use sys_access() instead of sys_stat() or sys_lstat() to test simple existence of file. Why copy dozens of bytes from kernel to user space that aren't going to be used by anything? There are probably more instances like these. Change-Id: I28089bef4cc93d5e4e4213045fb1a2649d110f82 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/17769 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Prashanth Pai <ppai@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* cluster/ec: Get size of file in EC [f]xattropPranith Kumar K2017-07-131-0/+1
| | | | | | | | | | | | | | | | | | | | Problem: For allowing parallel writes we shouldn't depend on ia_size to be same for all the bricks in each write_cbk(). But we need to make sure backend size is correct on all the bricks and no crashes/manual modifications happened. Fix: At the time of get_size_version() we do 1 check to make sure size of the file is same across the bricks. From then on the FOPs will give the status of the fop, so we rely on this information to keep which bricks are good/bad. Updates #251 Change-Id: I1df645347e2e9f2e09cfa4411b6cc305d7f4e4e5 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: https://review.gluster.org/17741 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
* storage/posix: New gfid2path infraKotresh HR2017-07-102-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | With this infra, a new xattr is stored on each entry creation as below. trusted.gfid2path.<xxhash> = <pargfid>/<basename> If there are hardlinks, multiple xattrs would be present. Fops which are impacted: create, mknod, link, symlink, rename, unlink Option to enable: gluster vol set <VOLNAME> storage.gfid2path on Updates: #139 Change-Id: I369974cd16703c45ee87f82e6c2ff5a987a6cc6a Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17488 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Aravinda VK <avishwan@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* mgtm/core : use sha hash function for volfile checkMohammed Rafi KC2017-07-108-10/+186
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are storing the entire volfile and using this to check volfile change. With brick multiplexing there will be lot of graphs per process which will increase the memory foot print of the process. So instead of storing the entire graph we could use sha256 and we can compare the hash to see whether volfile change happened or not. Also with Brick multiplexing, the direct comparison of vol file is not correct. There are two problems. Problem 1: We are currently storing one single graph (the last updated volfile) whereas, what we need is the entire graph with all atttached bricks. If we fix this issue, we have second problem Problem 2: With multiplexing we have a graph that contains multiple bricks. But what we are checking as part of the reconfigure is, comparing the entire graph with one single graph, which will always fail. Solution: We create list in glusterfs_ctx_t that stores sha256 hash of individual brick graphs. When a graph changes happens we compare the stored hash and the current hash. If the hash matches, then no need for reconfigure. Otherwise we first do the reconfigure and then update the hash. For now, gfapi has not changed this way. Meaning when gfapi volfile fetch or reconfigure happens, we still store the entire graph and compare, each memory. This is fine, because libgfapi will not load brick graphs. But changing the libgfapi will make the code similar in both glusterfsd-mgmt and api. Also it helps to reduce some memory. Change-Id: I9df917a771a52b95622ab8f63af34ec390163a77 BUG: 1467986 Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com> Reviewed-on: https://review.gluster.org/17709 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Atin Mukherjee <amukherj@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* gfapi+libglusterfs: fix mem_pools_fini without mem_pools_init caseJeff Darcy2017-07-091-2/+16
| | | | | | | | | | | | | | The change consists of two parts: make sure it doesn't happen (in glfs.c), and make it harmless if it does (in mem-pool.c). Change-Id: Icb7dda7a45dd3d1ade2ee3991bb6a22c8ec88424 BUG: 1468863 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17728 Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* groups: don't allocate auxiliary gid list on stackCsaba Henk2017-07-064-8/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | When glusterfs wants to retrieve the list of auxiliary gids of a user, it typically allocates a sufficiently big gid_t array on stack and calls getgrouplist(3) with it. However, "sufficiently big" means to be of maximum supported gid list size, which in GlusterFS is GF_MAX_AUX_GROUPS = 64k. That means a 64k * sizeof(gid_t) = 256k allocation, which is big enough to overflow the stack in certain cases. A further observation is that stack allocation of the gid list brings no gain, as in all cases the content of the gid list eventually gets copied over to a heap allocated buffer. So we add a convenience wrapper of getgrouplist to libglusterfs called gf_getgrouplist which calls getgrouplist with a sufficiently big heap allocated buffer (it takes care of the allocation too). We are porting all the getgrouplist invocations to gf_getgrouplist and thus eliminate the huge stack allocation. BUG: 1464327 Change-Id: Icea76d0d74dcf2f87d26cb299acc771ca3b32d2b Signed-off-by: Csaba Henk <csaba@redhat.com> Reviewed-on: https://review.gluster.org/17706 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* core: assorted typos and spelling mistakes from Debian lintianKaleb S. KEITHLEY2017-07-031-12/+12
| | | | | | | | | | | | | | Plus minor readability improvements. Reported-by: pmatthaei@debian.org Change-Id: I5393819a2fc9f240a19811143bb57b127df717cf BUG: 1466785 Signed-off-by: Kaleb S. KEITHLEY <kkeithle@redhat.com> Reviewed-on: https://review.gluster.org/17660 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* libglusterfs: add mem_pools_finiJeff Darcy2017-07-012-5/+29
| | | | | | | | | | | | | | | This also makes mem_pools_init and mem_pools_fini re-callable, so GFAPI can go through infinite init/fini cycles if they want to. Not saying that's a good idea, but at least it's safe. Change-Id: I617913410bcff54568b802cb653f48bdd533bd65 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17662 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Jeff Darcy <jeff@pl.atyp.us> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* multiple: fix struct/typedef inconsistenciesJeff Darcy2017-06-305-18/+18
| | | | | | | | | | | | | | | | | | | | | | | | The most common pattern, both in our code and elsewhere, is this: struct _xyz { ... }; typedef struct _xyz xyz_t; These exceptions - especially call_frame/call_stack - have been slowing down code navigation for years. By converging on a single pattern, navigating from xyz_t in code to the actual definition of struct _xyz (i.e. without having to visit the typedef first) might even be automatable. Change-Id: I0e5dd1f51f98e000173c62ef4ddc5b21d9ec44ed Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17650 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* contrib/xxhash: Add xxhash libraryKotresh HR2017-06-303-1/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | xxhash is a faster non-cryptographic hash. https://github.com/Cyan4973/xxHash Release Taken: "xxHash v0.6.2" -------------- Files added: contrib/xxhash/xxhash.c contrib/xxhash/xxhash.h contrib/xxhash/xxhsum.c Modifications to source: ------------------------ Following functions and data types got 'GF_' prefix as below to avoid any form of name collisions in future. ---- Functions ---- GF_XXH_versionNumber GF_XXH32 GF_XXH32_createState GF_XXH32_freeState GF_XXH32_copyState GF_XXH32_reset GF_XXH32_update GF_XXH32_digest GF_XXH32_canonicalFromHash GF_XXH32_hashFromCanonical GF_XXH64 GF_XXH64_createState GF_XXH64_freeState GF_XXH64_copyState GF_XXH64_reset GF_XXH64_update GF_XXH64_digest GF_XXH64_canonicalFromHash GF_XXH64_hashFromCanonical ---- Data Types ---- GF_XXH_errorcode GF_XXH32_state_t* GF_XXH32_canonical_t* GF_XXH32_hash_t GF_XXH64_state_t* GF_XXH64_canonical_t* GF_XXH64_hash_t It is linked with libglusterfs.so. A wrapper funtion is also added for the easy usage in common-utils.c. xxhash can be used for the all the usecases where a faster non-cryptographic hash is required. gfid to path infra would be using this for now. NOTE: ---- The gluster coding guidelines check is ignored as maintaining it further would be difficult. Updates: #253 Change-Id: Ib143f90d91d4ee99864a10246d5983e92900173b Signed-off-by: Kotresh HR <khiremat@redhat.com> Reviewed-on: https://review.gluster.org/17641 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* logging: Support for Structured logging formatAravinda VK2017-06-292-0/+164
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Wrapper for `gf_log` and `gf_msg` to add support for structured logging format. Two new wrappers available `gf_slog` and `gf_smsg` Example 1: All static details gf_slog ("cli", GF_LOG_INFO, "Volume Set", "name=gv1", "option=changelog.changelog", "value=on", NULL); gf_smsg ("cli", GF_LOG_INFO, 0, MSGID_VOLUME_SET, "Volume Set", "name=gv1", "option=changelog.changelog", "value=on", NULL); Example 2: Using Format chars in key values gf_slog ("cli", GF_LOG_INFO, "Volume Set", "name=%s", volume_name, "option=%s", option_name, "value=%s", option_value, NULL); gf_smsg ("cli", GF_LOG_INFO, 0, MSGID_VOLUME_SET, "Volume Set", "name=%s", volume_name, "option=%s", option_name, "value=%s", option_value, NULL); Formats as, <EVENT><TAB><KEY1=VALUE1><TAB><KEY2=VALUE2>... Example: Volume Set name=gv1 option=changelog.changelog value=on Updates: #240 Change-Id: I871727be16a39f681d41f363daa0029b8066fb52 Signed-off-by: Aravinda VK <avishwan@redhat.com> Reviewed-on: https://review.gluster.org/17543 Reviewed-by: MOHIT AGRAWAL <moagrawa@redhat.com> Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Amar Tumballi <amarts@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
* libglusterfs: fix disable-mempoolJeff Darcy2017-06-281-3/+3
| | | | | | | | | | | | Change-Id: I55f707ae1e7c3ad7fc0545f7aa657584cead58f9 BUG: 1465214 Signed-off-by: Jeff Darcy <jdarcy@fb.com> Reviewed-on: https://review.gluster.org/17636 Smoke: Gluster Build System <jenkins@build.gluster.org> Tested-by: Jeff Darcy <jeff@pl.atyp.us> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Ji-Hyeon Gim Reviewed-by: Amar Tumballi <amarts@redhat.com>
* libglusterfs: build failed with GF_DISABLE_MEMPOOLJi-Hyeon Gim2017-06-271-1/+1
| | | | | | | | | | | | | | | | When we build GlusterFS with GF_DISBLE_MEMPOOL, it is failed due to macro condition in mem-pool.c:mem_get(). Change-Id: I03fe804f93d761ea3bfdc3b20f0253a03350a68f BUG: 1465214 Signed-off-by: Ji-Hyeon Gim <potatogim@potatogim.net> Reviewed-on: https://review.gluster.org/17633 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Vijay Bellur <vbellur@redhat.com> Reviewed-by: jiffin tony Thottan <jthottan@redhat.com> Tested-by: Ji-Hyeon Gim CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com>
* cluster/rebalance: Use GF_XATTR_LIST_NODE_UUIDS_KEY to figure out local subvols.Susant Palai2017-06-261-0/+1
| | | | | | | | | | | | | | | | | | | | | | Afr has introduced a new key GF_XATTR_LIST_NODE_UUIDS_KEY, through which rebalance will figure out its local subvolumes.(Reference bugid=1463250) key: GF_XATTR_NODE_UUID_KEY will continue to serve it's old purpose of returning the first afr chiild. test: prove tests/basic/distribute/rebal-all-nodes-migrate.t Change-Id: I4d602feda2a05b29d2210c712a07a4ac6b8bc112 BUG: 1463648 Signed-off-by: Susant Palai <spalai@redhat.com> Signed-off-by: N Balachandran <nbalacha@redhat.com> Reviewed-on: https://review.gluster.org/17595 Smoke: Gluster Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
* cluster/ec: Node uuid xattr support update for ECSunil Kumar Acharya2017-06-231-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | Problem: The change in EC to return list of node uuids for GF_XATTR_NODE_UUID_KEY was causing problems with geo-rep. Fix: This patch will allow to get the single node uuid as it was doing before with the key "GF_XATTR_NODE_UUID_KEY", and will also allow to get the list of node uuids by using a new key "GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve the problem with geo-rep and any other features which were depending on this. BUG: 1462790 Change-Id: I2d9214a9658d4a41a3d6de08600884d2bda5f3eb Signed-off-by: Sunil Kumar Acharya <sheggodu@redhat.com> Reviewed-on: https://review.gluster.org/17594 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org>
* gfapi : Resolve "." and ".." only for named lookupsJiffin Tony Thottan2017-06-202-116/+19
| | | | | | | | | | | | | | | | | | | | The patch https://review.gluster.org/#/c/17177 resolves "." and ".." to corrosponding inodes and names before sending the request to the backend server. But this will only work if inode and its parent is linked properly. Incase of nameless lookup(applications like ganesha) the inode of parent can be NULL(only gfid is send). So this patch will resolve "." and ".." only if proper parent is available Change-Id: I4c50258b0d896dabf000a547ab180b57df308a0b BUG: 1460514 Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com> Reviewed-on: https://review.gluster.org/17502 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Poornima G <pgurusid@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: soumya k <skoduri@redhat.com> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* cluster/afr: Returning single and list of node uuids from AFRkarthik-us2017-06-201-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Problem: The change in afr to return list of node uuids was causing problems with geo-rep. Fix: This patch will allow to get the single node uuid as it was doing before with the key "GF_XATTR_NODE_UUID_KEY", and will also allow to get the list of node uuids by using a new key "GF_XATTR_LIST_NODE_UUIDS_KEY". This will solve the problem with geo-rep and any other feature which were depending on this. Change-Id: I09885dac6dfca127be94b708470c8c2941356f9a BUG: 1462790 Signed-off-by: karthik-us <ksubrahm@redhat.com> Reviewed-on: https://review.gluster.org/17576 Smoke: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Ravishankar N <ravishankar@redhat.com> Reviewed-by: Kotresh HR <khiremat@redhat.com> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Jeff Darcy <jeff@pl.atyp.us>
* debug/io-stats: Provide option to select stats output formatKrutika Dhananjay2017-06-151-0/+2
| | | | | | | | | | | | | | ... as opposed to hardcoding it to "json" always. Change-Id: I5e79473a514373145ad764f24bb6219a6983a4c6 BUG: 1458197 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com> Reviewed-on: https://review.gluster.org/17451 Smoke: Gluster Build System <jenkins@build.gluster.org> NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org> Reviewed-by: Niels de Vos <ndevos@redhat.com> CentOS-regression: Gluster Build System <jenkins@build.gluster.org> Reviewed-by: Shyamsundar Ranganathan <srangana@redhat.com>
generated by cgit (git 2.34.1) at 2024-11-12 18:34:38 +0000