summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* features/quota: minor fixes to enforcerv3.4quota1Raghavendra G2013-08-092-12/+5
| | | | | | | | | | * send size query to quotad only if limit is set on that inode. * don't check for loc->parent while querying size from quotad, since its a nameless lookup Change-Id: I10dc2f9d1e40875382040b53cb4ee5f6d9a27133 BUG: 969461 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* features/quota: fixes to code reading limits from xattrs.Raghavendra G2013-08-093-46/+34
| | | | | | | | | | It was assumed that hard and soft limits are stored as two different xattrs on disk. However they are stored as two members of a structure which is stored as a value for a single key. Change-Id: I947fa5c375209c31fe1511bda0d5cb0e249af9ba BUG: 969461 Signed-off-by: Raghavendra G <rgowdapp@redhat.com>
* glusterd,cli: changes to quota list <path> ...Krutika Dhananjay2013-08-092-321/+82
| | | | | Change-Id: Ia37020c3aa11af6eed3af09cfe390b848b028b6a Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* glusterd: Clean up and fix glusterd_op_quota()Krutika Dhananjay2013-08-094-130/+85
| | | | | | | | | | | | | | | | | | | | ... and also fix cli logging In glusterd_op_quota(), * do not modify ret after going to 'out' as this causes the failure status (-1) to be overwritten, thereby causing the command to return 0 even on failure. * knock off additional labels like create_vol. * replace 'if' statements with 'switch case' statement. * delete only the 3 timeouts and the defaul-soft-limit, if present, from volinfo->dict, upon disablement of quota. Change-Id: If486a5373b66f2379d6d041a974d9b824fcb8518 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* glusterd: Changes to 'quota remove' subcommand behaviorKrutika Dhananjay2013-08-091-44/+39
| | | | | Change-Id: Ifdc60071146587dc5c60d9a53a92d49b3487fd82 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* glusterd: club limit-usage and soft-limit into a single commandKrutika Dhananjay2013-08-096-221/+183
| | | | | Change-Id: I5f680675576aeec584b497eb25dd804a9dd6d690 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* quota: Fix initialisation of privKrutika Dhananjay2013-08-081-0/+2
| | | | | | | Original-author: Vijay Bellur <vbellur@redhat.com> Change-Id: Iea21ef1cdfb78c79482ad02f81734516b7818714 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* Change branch to quota.Vijay Bellur2013-08-081-1/+1
| | | | Change-Id: I8a6d49cc101da219404633d43b8d211f816094a7
* features/quota: Allow the gluster 'special' processes to supercede the limitsVijay Bellur2013-08-081-4/+18
| | | | | | | | | Don't block the gluster internal processes like rebalance, gsyncd, self heal etc from the disk quotas and the xattrs setting. Solution: Allow all the clients with negative PID. Change-Id: Iaeaa8096e00d48b2a4c3f5df61d103da0b3d6598
* features/quota: Fix compilation warningsVijay Bellur2013-08-082-8/+24
| | | | Change-Id: Ie0b3af8b52f2d909c61094bdcaccfd724ff4ecc0
* features/quota: Introduce quota log helper function.Vijay Bellur2013-08-082-0/+55
| | | | Change-Id: I38077c7adc497b314f4037cbbb116458a26ed589
* glusterd, cli: Provide status of quotad in 'volume status'Krutika Dhananjay2013-08-0811-34/+117
| | | | | | Change-Id: I5e90376ecfe11ae5a3bca936d9d9acdd54c337d7 BUG: 969461 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* features/quota: design changesRaghavendra G2013-08-0813-1387/+1756
| | | | | | | | | | | | | | * hard and soft limits are persisted in xattrs of the inode. Associating limits with inode instead of maintaining as a global list helps us to scale better. * quotad-aggregator acts as a special client to provide cluster view through an rpc program. Quota enforcer loaded on brick uses this to get aggregated directory sizes. Aggregated sizes are cached for a timeout period in in-memory inode contexts. Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: I2ab508d9d4fe224bc1d8cf01cf2b7969dd4200bb BUG: 969461
* Merge "glusterd: Move timeout options and default-soft-limit to quota ↵Krishnan Parthasarathi2013-08-042-80/+6
|\ | | | | | | xlator" into quota-improvements
| * glusterd: Move timeout options and default-soft-limit to quota xlatorKrutika Dhananjay2013-07-312-80/+6
| | | | | | | | | | | | | | | | | | Write the 3 timeout options {soft-timeout, hard-timeout, alert-time} and default-soft-limit, if explicitly set, into brick volfiles. Change-Id: Ie3229a8ab1b081a5936defd4f977afc8a19dad50 BUG: 969461 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* | glusterd: Spawn quotad on every nodeKrutika Dhananjay2013-07-313-31/+46
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... and also trigger a reconfigure when the following quota sub-commands are executed: a. default-soft-limit, b. hard-timeout, c. soft-timeout, and d. alert-time. Also start/restart/stop quotad only when quota is enabled or disabled on any volume. Tests performed in a two node cluster: a. Create and start a volume, enable quota on it and check if quotad is spawned on both the nodes. b. Execute all quota sub-commands on the volume except 'enable' and 'disable' and verify that the pid of quota daemon doesn't change. c. Stop the volume and verify that quotad is stopped. d. Start it again. Quotad must be started now. e. Create, start and enable quota on a second volume, verify that the pid of quotad changes on both the nodes (indicating a restart). f. Disable quota on one of the volumes and verify that quotad's pid changes. g. Disable quota on the second volume too and verify that quotad is stopped on both the nodes. h. Enable quota again on one of the volumes, and verify that quotad is started on both the nodes. i. Add a new node into this cluster and verify that quotad is spawned on this node too. Change-Id: Ie93ab69c685051e196c377cff15078a1cde17fca BUG: 969461 Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
* Change branch to quota-improvementsVijay Bellur2013-07-291-1/+1
| | | | | Change-Id: Idb1ed302a73e645cef66e5096a6cef83290b10ce Signed-off-by: Vijay Bellur <vbellur@redhat.com>
* features/quota: Improvements to quotaVarun Shastry2013-07-2930-1243/+3441
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Old implementation * Client side implementation of quota - Not secure - Increased traffic in updating the ctx New Implementation * 2 stages of quota implementation is done: Soft and hard quota Upon reaching soft quota limit on the directory it logs/alerts in the quota daemon log (ie DEFAULT_LOG_DIR/quotad.log) and no more writes allowed after hard quota limit. After reaching the soft-limit the daemon alerts the user/admin repeatively for every 'alert-time', which is configurable. * Quota is moved to server-side. There will be 2 quota xlators i. Quota Server It takes care of the enforcing the quota and maintains the context specific to the brick. Since this doesn't have the complete picture of the cluster, cluster wide usage is updated from the quota daemon. This updated context is saved and used for the enforcement. It updates its context by searching the QUOTA_UPDATE_KEY from the dict in the setxattr call, and is updated from nowhere else. The quota is always loaded in the server graph and is by passed if the feature is not enabled. Options specific to quota-server: server-quota - Specifies whether the features is on/off. It is used to by pass the quota if turned off. deem-statfs - If set to on, it takes quota limits into consideration while estimating fs size. (df command) ii. Quota Daemon This is the new xlator introduced with this patch. Its the *gluster client* process with no mount point, started upon enabling quota or restarting the volume. This is a single process for all the volumes in the cluster. Its volfile stored in GLUSTERD_DEFAULT_WORKI_DIR/quotad/quotad.vol. It queries for the sizes on all the bricks, aggregates the size and sends back the updated size, periodically. The timeout between successive updation is configurable and typically/by default more for below-soft-quota usage and less for above-soft-quota usage. It maintains the timeout inside the limit structure based on the usage; below soft limit and above soft limit. There will be thread running per volume which iterates through the list and decides whether the size to be queried in the current iteration based on its timeout. It takes the next iteration time taking the least of the timeouts in the list of entries. Maintains a separate inode table for each volume in the quotad. In the first iteration it builds the table for quota-dirs (dirs on which limit is set) and its components. Options specific to quotad: hard-timeout - Timeout for updation of usage to the quota-server when the usage is crosses the soft-limit. soft-timeout - Timeout for the updation of usage to the quota-server when the usage is below soft-limit. alert-time - Frequency of logging after the usage reached soft limit. Options common to both: default-soft-limit - This is used when individual paths are not configured with soft-limit and default value of this option is 90% of the hard-limit. limit-set - String containing all the limits. Thus in the current implementation we'll have 2 quota xlators: one in server graph and one in trusted client (quota daemon) of which the sole purpose will be to aggregate the quota size xattrs from all the bricks and send the same to server quota xlator. * Changes in glusterd and CLI A single volfile is created for all the volumes, similar to nfs volfile. All files related to quota client (volfile, pid etc) are stored in GLUSTERD_DEFAULT_WORK_DIR/quotad/. The new pattern of the quota limit stores in limit-set = <single-dir-limit>[,<single-dir-limit>] single-dir-limit = <abs-path>:<hard-limit>[:<soft-limit-in-percent>] It also introduces new options: volume quota <VOLNAME> {enable|disable|list [<path> ...]|remove <path>| default-soft-limit <percent>} | volume quota <VOLNAME> {limit-usage <path> <size> |soft-limit <path> <percent>} | volume quota <VOLNAME> {alert-time|soft-timeout|hard-timeout} {<time>} Credit: Raghavendra Bhat <rabhat@redhat.com> Varun Shastry <vshastry@redhat.com> Shishir Gowda <sgowda@redhat.com> Kruthika Dhananjay <kdhananj@redhat.com> Brian Foster <bfoster@redhat.com> Krishnan Parthasarathi <kparthas@redhat.com> Change-Id: I16ec5be0c2faaf42b14034b9ccaf17796adef082 BUG: 969461 Signed-off-by: Varun Shastry <vshastry@redhat.com>
* doc: Create a features folder.testVijay Bellur2013-07-253-0/+74
| | | | | | | | | | | | Moved rdma and bd documents to doc/features. Added a new document on rebalance. Change-Id: I04269202adc9605754fc29876433c88480b822a3 BUG: 811311 Signed-off-by: Vijay Bellur <vbellur@redhat.com> Reviewed-on: http://review.gluster.org/5395 Reviewed-by: Anand Avati <avati@redhat.com> Tested-by: Anand Avati <avati@redhat.com>
* features/changelog: fixes when enabling changelogVenky Shankar2013-07-252-29/+41
| | | | | | | | | | | | | Other enhancements being: * ignore fops made by rebalance * ignore internally triggered fops BUG: 987734 Change-Id: I7dd164ae3c209fdb8ec43a27e67b8846f937c93b Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/5380 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* glusterfsd: is_mgmt_rpc_reconnect should not be `char` but gf_boolean_tHarshavardhana2013-07-251-2/+2
| | | | | | | | | Change-Id: I35c4dc040ebb5e926c7f54d823d1c81b1228bd4f BUG: 763046 Signed-off-by: Harshavardhana <harsha@harshavardhana.net> Reviewed-on: http://review.gluster.org/5345 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* write-behind: preserve error returned as-isAmar Tumballi2013-07-241-5/+1
| | | | | | | | | Change-Id: Ib766403774c1323e0bbddafedeaa47e7fa3a59fa Signed-off-by: Amar Tumballi <amarts@redhat.com> BUG: 987415 Reviewed-on: http://review.gluster.org/5296 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* core: increase the auxillary group limit to 65536Anand Avati2013-07-247-12/+75
| | | | | | | | | | | | | Make the allocation of groups dynamic and increase the limit to 65536. Change-Id: I702364ff460e3a982e44ccbcb3e337cac9c2df51 BUG: 953694 Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/5111 Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cli :remove-brick process output leads to ambiguitysusant2013-07-241-3/+6
| | | | | | | | | | | | | The output of remove-brick status as "Not started" leads to ambiguity.We should not show the status of the Server nodes which do not participate in the remove-brick process. Change-Id: I85fea40deb15f3e2dd5487d881f48c9aff7221de BUG: 986896 Signed-off-by: susant <spalai@redhat.com> Reviewed-on: http://review.gluster.org/5383 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/marker: pass xdata in marker_unlink()Venky Shankar2013-07-242-1/+6
| | | | | | | | | | Change-Id: Ia310af96b25f29351f3adc4bbc97aea271df7673 BUG: 987747 Signed-off-by: Venky Shankar <vshankar@redhat.com> Reviewed-on: http://review.gluster.org/5379 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* storage/posix: Fix conditional compiling for syncfsPranith Kumar K2013-07-241-1/+4
| | | | | | | | | Change-Id: Ief22e1c0f2b5074060752d70da41ae93f1028d62 BUG: 927146 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/5381 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cli: gluster volume heal commands are more elaborativeVenkatesh Somyajulu2013-07-241-5/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | 1. "gluster volume heal volume-name" output :Launching heal operation to perform index self heal on volume volume-name has been successful 2. "gluster volume heal volume-name full" Output :Launching heal operation to perform full self heal on volume volume-name has been successful 3. "gluster volume heal volume-name info" Output :Gathering list of entries to be healed on volume volume-name has been successful 4. "gluster volume heal volume-name info healed" Output :Gathering list of healed entries on volume volume-name has been successful 5. "gluster volume heal volume-name info split-brain" Output :Gathering list of split brain entries on volume volume-name has been successful 6. "gluster volume heal volume-name info heal-failed" Output :Gathering list of heal failed entries on volume volume-name has been successful Change-Id: I74c90e8129d23d513ddb7879358a9d21c94a5c0d BUG: 978936 Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com> Reviewed-on: http://review.gluster.org/5286 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/dht: mark linkfile creation/deletion as internal fopshishir gowda2013-07-241-9/+39
| | | | | | | | | | | | | | | | | | | Currently dht creates/deletes linkfiles for various ops like rename/linking and when layout changes. dht_linkfile_create already sends a key GLUSTERFS_INTERNAL_FOP_KEY in dict to identify this as an internal fop and not user based op. Enhancing rename related links/unlinks to send this key too. Marker/changelog or other xlators can now identify these as internal fops and handle them accordingly Change-Id: Ib1ca789e6dbce48703c55ad3f4f88f7cd2df3d06 BUG: 987428 Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.org/5335 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* storage/posix: implement batched fsync in a single threadAnand Avati2013-07-235-1/+288
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Because of the extra fsync()s issued by AFR transaction, they could potentially "clog" all the io-threads denying unrelated operations from making progress. This patch assigns a dedicated thread to issues fsyncs, as an experimental feature to understand performance characteristics with the approach. As a basis, incoming individual fsync requests are grouped into batches, falling in the same @batch-fsync-delay-usec window of time. These windows can extend in practice, as processing of the previous batch can take longer than @batch-fsync-delay-usec while new requests are getting batched. The feature support three modes (similar to the -S modes of fs_mark) - syncfs: In this mode one syncfs() is issued per batch, instead of N fsync()s (one per file.) - syncfs-single-fsync: In this mode one syncfs() is issued per batch (which, on Linux, guarantees the completion of write-out of dirty pages in the filesystem up to that point) and one single fsync() to synchronize or flush the controller/drive cache. This corresponds to -S 2 of fsmark. - syncfs-reverse-fsync: In this mode, one syncfs() is issued per batch, and all the open files in that batch are fsync()'ed in the reverse order of the queue. This corresponds to -S 4 of fsmark. - reverse-fsync: In this mode, no syncfs() is issued and all the files in the batch are fsync()'ed in the reverse order. This corresponds to -S 3 of fsmark. Change-Id: Ia1e170a810c780c8d80e02cf910accc4170c4cd4 BUG: 927146 Signed-off-by: Anand Avati <avati@redhat.com> Reviewed-on: http://review.gluster.org/4746 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* tests: Avoid timing delays caused by fsyncPranith Kumar K2013-07-234-0/+4
| | | | | | | | Change-Id: I2da53dd14d7352bdc878fce5a9019a955e150ce9 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/5377 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* cli: Increased timeout for gluster volume heal commnadsVenkatesh Somyajulu2013-07-232-3/+6
| | | | | | | | | | | | | | | | | Problem: If number of files are very large, then gluster volume heal volumename info commnads take large time. So timeout of 2 minutes seems to be insufficient. Fix: Increased timeout to 10 minutes Change-Id: I5f847163e01c4afbb587b726833ad80183f1a928 BUG: 986945 Signed-off-by: Venkatesh Somyajulu <vsomyaju@redhat.com> Reviewed-on: http://review.gluster.org/5372 Reviewed-by: Kaushal M <kaushal@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* cluster/afr: Handle parallel hardlinks self-healPranith Kumar K2013-07-232-0/+56
| | | | | | | | | Change-Id: Ieda11870c65edae500140b6c061f15a7b3f264f3 BUG: 986905 Signed-off-by: Pranith Kumar K <pkarampu@redhat.com> Reviewed-on: http://review.gluster.org/5370 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* glusterfsd: Use dynamic volfile bufferKaushal M2013-07-231-1/+20
| | | | | | | | | | | | | Glusterfsd used a fixed buffer to store volfiles fetched via getspec. This caused problems with large volfiles. Changining this to a dynamic buffer allows large volfiles to be loaded in memory. Change-Id: I40236dcb1c37b9a0136dfb5231cafabb3d4f00dc BUG: 986100 Signed-off-by: Kaushal M <kaushal@redhat.com> Reviewed-on: http://review.gluster.org/5373 Reviewed-by: Vijay Bellur <vbellur@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* move 'xlators/marker/utils/' to 'geo-replication/' directoryAvra Sengupta2013-07-2225-159/+9
| | | | | | | | | | Change-Id: Ibd0faefecc15b6713eda28bc96794ae58aff45aa BUG: 847839 Original Author: Amar Tumballi <amarts@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5133 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* features/changelog: changelog translatorAvra Sengupta2013-07-2235-16/+5053
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the initial version of the Changelog Translator. What is it ----------- Goal is to capture changes performed on a GlusterFS volume. The translator needs to be loaded on the server (bricks) and captures changes in a plain text file inside a configured directory path (controlled by "changelog-dir", should be somewhere in <export>/.glusterfs/changelog by default). Changes are classified into 3 types: - Data: : TYPE-I - Metadata : TYPE-II - Entry : TYPE-III Changelog file is rolled over after a certain time interval (defauls to 60 seconds) after which a changelog is started. The thing to be noted here is that for a time interval (time slice) multiple changes for an inode are recorded only once (ie. say for 100+ writes on an inode that happens within the time slice has only a single corresponding entry in the changelog file). That way we do not bloat up the changelog and also save lots of writes. Changelog Format ----------------- TYPE-I and TYPE-II changes have the gfid on the entity on which the operation happened. TYPE-III being a entry op requires the parent gfid and the basename. Changelog format has been kept to a minimal and it's upto the consumers to do the heavy loading of figuring out deletes, renames etc.. A single changelog file records all three types of changes, with each change starting with an identifier ("D": DATA, "M": METADATA and "E": ENTRY). Option is provided for the encoding type (See TUNABLES). Consumers ---------- The only consumer as of today would be geo-replication, although backup utilities, self-heal, bit-rot detection could be possible consumers in the future. CLI ---- By default, change-logging is disabled (the translator is present in the server graph but does nothing). When enabled (via cli) each brick starts to log the changes. There are a set of tunable that can be used to change the translators behaviour: - enable/disable changelog (disabled by default) gluster volume set <volume> changelog {on|off} - set the logging directory (<brick>/.glusterfs/changelogs is the default) gluster volume set <volume> changelog-dir /path/to/dir - select encoding type (binary (default) or ascii) gluster volume set <volume> encoding {binary|ascii} - change the rollover time for the logs (60 secs by default) gluster volume set <volume> rollover-time <secs> - when secs > 0, changelog file is not open()'d with O_SYNC flag - and fsync is trigerred periodically every <secs> seconds. gluster volume set <volume> fsync-interval <secs> features/changelog: changelog consumer library (libgfchangelog) A shared library is provided for the consumer of the changelogs for easy acess via APIs. Application can link against this library and request for changelog updates. Conversion of binary logs to human-readable ascii format is also taken care by the library which keeps a copy of the changelog in application provided working directory. Change-Id: I75575fb7f1c53d2bec3dba1a329ea7bb3c628497 BUG: 847839 Original Author: Venky Shankar <vshankar@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5127 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* log: send current gf_log to syslog conditionallyBala.FA2013-07-193-19/+170
| | | | | | | | | | | | | | | | When compile time option GF_USE_SYSLOG is enabled (which is default), generated logs are sent to syslog with error code ERR_DEV. User can opt to use traditional log at run time by creating /var/log/glusterd/logger.conf file and restarting respective gluster services. Change-Id: I9837d0f99da1afc2189d7ecd214c4293ec53715a BUG: 928648 Signed-off-by: Bala.FA <barumuga@redhat.com> Reviewed-on: http://review.gluster.org/5002 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* log: enhance syslog logging using CEE formatBala.FA2013-07-193-1/+231
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch enables to use syslog as log target in addition to the default. The logs are sent in CEE format (http://cee.mitre.org/). This logging can be disabled using compile time option by ./configure --disable-syslog (or) rpmbuild glusterfs.tar.gz --without syslog The framework provides two api void gf_openlog (const char *ident, int option, int facility); void gf_syslog (int error_code, int facility_priority, char *format, ...); consumers need to call gf_openlog() prior to gf_syslog() like the way traditional syslog function calls. error_code is mandatory when using gf_syslog(). For example, gf_openlog (NULL, -1, -1); gf_syslog (GF_ERR_DEV, LOG_ERR, "error reading configuration file"); Using syslog, admin is free to configure logger to * reduce repeated log messages * forward logs to remote logger * execute a command on certain log pattern * alert people for certain log pattern by email, snmp etc * and many more Change-Id: Ibacbcbbc547192893fc4a46b387496b622e4811f BUG: 928648 Signed-off-by: Bala.FA <barumuga@redhat.com> Reviewed-on: http://review.gluster.org/4915 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* log: error code generation supportBala.FA2013-07-197-2/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | error code and message are generated at compile time by reading a json file which contains information of elements for each error code. This framework provides error handling and ability to do more cleaner log messages to users. error-codes.json file contains error description is below format { "ERR_NAME": {"code": ERR_NUM, "message": {"LOCALE": "ERR_MESSAGE"}} } At compile time autogen.sh calls gen-headers.py which produces C header file libglusterfs/src/gf-error-codes.h. This header has a function const char *_gf_get_message (int code); which returns respective ERR_MESSAGE for given ERR_NUM. Change-Id: Ieefbf4c470e19a0175c28942e56cec98a3c94ff0 BUG: 928648 Signed-off-by: Bala.FA <barumuga@redhat.com> Reviewed-on: http://review.gluster.org/4977 Reviewed-by: Niels de Vos <ndevos@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* fuse: auxiliary gfid mount supportRaghavendra G2013-07-1913-136/+1317
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * files can be accessed directly through their gfid and not just through their paths. For eg., if the gfid of a file is f3142503-c75e-45b1-b92a-463cf4c01f99, that file can be accessed using <gluster-mount>/.gfid/f3142503-c75e-45b1-b92a-463cf4c01f99 .gfid is a virtual directory used to seperate out the namespace for accessing files through gfid. This way, we do not conflict with filenames which can be qualified as uuids. * A new file/directory/symlink can be created with a pre-specified gfid. A setxattr done on parent directory with fuse_auxgfid_newfile_args_t initialized with appropriate fields as value to key "glusterfs.gfid.newfile" results in the entry <parent>/bname whose gfid is set to args.gfid. The contents of the structure should be in network byte order. struct auxfuse_symlink_in { char linkpath[]; /* linkpath is a null terminated string */ } __attribute__ ((__packed__)); struct auxfuse_mknod_in { unsigned int mode; unsigned int rdev; unsigned int umask; } __attribute__ ((__packed__)); struct auxfuse_mkdir_in { unsigned int mode; unsigned int umask; } __attribute__ ((__packed__)); typedef struct { unsigned int uid; unsigned int gid; char gfid[UUID_CANONICAL_FORM_LEN + 1]; /* a null terminated gfid string * in canonical form. */ unsigned int st_mode; char bname[]; /* bname is a null terminated string */ union { struct auxfuse_mkdir_in mkdir; struct auxfuse_mknod_in mknod; struct auxfuse_symlink_in symlink; } __attribute__ ((__packed__)) args; } __attribute__ ((__packed__)) fuse_auxgfid_newfile_args_t; An initial consumer of this feature would be geo-replication to create files on slave mount with same gfids as that on master. It will also help gsyncd to access files directly through their gfids. gsyncd in its newer version will be consuming a changelog (of master) containing operations on gfids and sync corresponding files to slave. * Also, bring in support to heal gfids with a specific value. fuse-bridge sends across a gfid during a lookup, which storage translators assign to an inode (file/directory etc) if there is no gfid associated it. This patch brings in support to specify that gfid value from an application, instead of relying on random gfid generated by fuse-bridge. gfids can be healed through setxattr interface. setxattr should be done on parent directory. The key used is "glusterfs.gfid.heal" and the value should be the following structure whose contents should be in network byte order. typedef struct { char gfid[UUID_CANONICAL_FORM_LEN + 1]; /* a null terminated gfid * string in canonical form */ char bname[]; /* a null terminated basename */ } __attribute__((__packed__)) fuse_auxgfid_heal_args_t; This feature can be used for upgrading older geo-rep setups where gfids of files are different on master and slave to newer setups where they should be same. One can delete gfids on slave using setxattr -x and .glusterfs and issue stat on all the files with gfids from master. Thanks to "Amar Tumballi" <amarts@redhat.com> and "Csaba Henk" <csaba@redhat.com> for their inputs. Signed-off-by: Raghavendra G <rgowdapp@redhat.com> Change-Id: Ie8ddc0fb3732732315c7ec49eab850c16d905e4e BUG: 952029 Reviewed-on: http://review.gluster.com/#/c/4702 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Amar Tumballi <amarts@redhat.com> Reviewed-on: http://review.gluster.org/4702 Reviewed-by: Xavier Hernandez <xhernandez@datalab.es> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* core: avoid infinite mutual recursion between THIS and gf_logJeff Darcy2013-07-181-2/+0
| | | | | | | | | | | | | | | | | This caused a stack overflow when (for some unknown reason) pthread_setspecific was failing. Therefore __glusterfs_this_location called gf_log which used THIS which wraps __glusterfs_this_location which . . . you get the idea. We have to break the loop somewhere, and we can't reasonably make _gf_log stop using THIS, so we make __glusterfs_this_location stop using _gf_log. Change-Id: I79c3ea40dd7980bb8ac76a52cdbf5c057b2e1c3c Signed-off-by: Jeff Darcy <jdarcy@redhat.com> Reviewed-on: http://review.gluster.org/5341 Reviewed-by: Krishnan Parthasarathi <kparthas@redhat.com> Reviewed-by: Raghavendra G <rgowdapp@redhat.com> Reviewed-by: Raghavendra Bhat <raghavendra@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* mount/fuse: unlink the inode on revalidate if entry not foundBrian Foster2013-07-182-0/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | If an inode/dentry is linked via a client and removed via a separate client, the inode/dentry mapping in the initial client remains. A lookup of the removed name on the initial client typically returns ENOENT once the associated caches expire. If the initial client has multiple dentries linked to the same inode, however, lookups on the non-removed dentry create windows of time where lookups on the stale/removed name return successfully. This occurs because the stale mapping resolves to the still valid inode and tricks md-cache into returning valid lookup data. To correct this situation, unlink the stale inode mapping on a failed (ENOENT) revalidation lookup (i.e., when fuse has resolved the inode but a lookup returns ENOENT). Note that with this change, the state still occurs until an md-cache window has expired, allowed a lookup to pass through to the server and given the fuse translator an opportunity to clean up. Change-Id: I47dde2f11e2ef5b8dd51e9ac8be0f36cdb5081a3 BUG: 985074 Signed-off-by: Brian Foster <bfoster@redhat.com> Reviewed-on: http://review.gluster.org/5337 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* include <limits.h> for PATH_MAXEmmanuel Dreyfus2013-07-181-0/+1
| | | | | | | | | | | | | | I need to include <limits.h> in order to use PATH_MAX, Otherwise it will not build at mine. I believe it is standard compliant to do so: http://pubs.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html BUG: 764655 Change-Id: I3f124466f7f7742e94a9d1256bc9239ec16aab04 Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org> Reviewed-on: http://review.gluster.org/5340 Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Jeff Darcy <jdarcy@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com>
* locks: Added an xdata-based 'cmd' for inodelk count in a given domainshishir gowda2013-07-185-20/+68
| | | | | | | | | | | | | | | | | | | | Following is the semantics of the 'cmd': 1) If @domain is NULL - returns no. of locks blocked/granted in all domains 2) If @domain is non-NULL- returns no. of locks blocked/granted in that domain 3) If @domain is non-existent - returns '0'; This is important since locks xlator creates a domain in a lazy manner. where @domain - a string representing the domain. Change-Id: I5e609772343acc157ca650300618c1161efbe72d BUG: 951195 Original-author: Krishnan Parthasarathi <kparthas@redhat.com> Signed-off-by: Krishnan Parthasarathi <kparthas@redhat.com> Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.org/4889 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com>
* dht: fix dht_discover_cbk doing a wrong layout set.shishir gowda2013-07-171-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | with the sequence of operations are like below, we have issues with current code (MP == mountpoint): T0,MP1# mkdir /abcd (Succeeds on hash_subvol) T1,MP2# mkdir /abcd (Gets EEXIST as dir exists in hash_subvol) T2,MP2# mkdir /.gfid/<abcd's gfid>/xyz (lookup happens on abcd's gfid, calls dht_discover) T3,MP1# (Completes mkdir(), goes to dir_selfheal to set the layouts). T4,MP2# (dht_discover_cbk gets success for lookup as the entry existed, as layout is not yet written, it says normalize done, found holes). T5,MP2# (as layout anomaly is not considered an issue in this patch, dht_layout_set happens on inode, with all xlators pointing to 0s) T6,MP1# (completes mkdir call, inode has proper layouts) T7,MP2# mkdir /.gfid/<abcd's gfid>/xyz fails with ENOENT (with log saying no subvol found for hash value of xyz. Porting Amar's fix from down-stream beta branch. Change-Id: Ibdc37ee614c96158a1330af19cad81a39bee2651 BUG: 982913 Original-author: Amar Tumballi <amarts@redhat.com> Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.org/5302 Reviewed-by: Amar Tumballi <amarts@redhat.com> Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* gfapi: fix for not able to rename foldersRaghavendra Talur2013-07-161-8/+11
| | | | | | | | | | | | If oldname is being renamed to newname, we need to check for type of newname only if newname exists. Change-Id: I068a283f9ffe67fcd5e8754d6bf052a2339efbf2 BUG: 953694 Signed-off-by: Raghavendra Talur <rtalur@redhat.com> Reviewed-on: http://review.gluster.org/5333 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* cluster/dht: Prevent dht_access from going into a loop.shishir gowda2013-07-153-1/+40
| | | | | | | | | | | | | | | | | If access fails with ENOTCONN, do not wind to same subvol. We wind to first-up-subvol if access fails with ENOTCONN. In few cases, if dht has only 1 subvolume, and access fails with ENOTCONN, we go into a infinite loop of winding to same subvol The fix is to check if we previously wound to same subvol, and fail if first-up-subvol is same. Change-Id: Ib5d3ce7d33e8ea09147905a7df1ed280874fa549 BUG: 983431 Signed-off-by: shishir gowda <sgowda@redhat.com> Reviewed-on: http://review.gluster.org/5319 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Anand Avati <avati@redhat.com>
* tests: call 'cleanup' at the end of each testNiels de Vos2013-07-158-0/+14
| | | | | | | | | | | | | | Some tests do not cleanup after themselves. That is bad behaviour and makes it difficult to run single tests and verify the state of the system afterwards. Change-Id: I4ac5401d790d6bc81e6975fd1384874b21d6cf8a BUG: 983975 Signed-off-by: Niels de Vos <ndevos@redhat.com> Reviewed-on: http://review.gluster.org/5328 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com> Reviewed-by: Anand Avati <avati@redhat.com>
* afr: customize client-pid=-1 xtime aggregation to tolerate a replica downAvra Sengupta2013-07-152-2/+50
| | | | | | | | | | | | Using the new 'pluggable policies' API of libxlator. Change-Id: Ie7528182dff8fb42c6e8287a106d3057944df775 BUG: 847839 Original Author: Csaba Henk <csaba@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/4904 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* libxlator: implement pluggable aggregation policiesAvra Sengupta2013-07-155-47/+158
| | | | | | | | | | | | | | | | | The API is described in libxlator.h. Behavior remains the same for this commit; this is a preparatory step for per-translator customization of aggregation. Change-Id: I5d42923af59b2fd78e1ff59c12763875b57c5190 BUG: 847839 Original Author: Csaba Henk <csaba@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/4903 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
* cluster/dht: node-uuid for directories winds to all subvolumesAvra Sengupta2013-07-151-2/+6
| | | | | | | | | | | | | | | this works similar to pathinfo now except that the request is sent to all subvolumes of dht. Underlying replica selects it's subvolume in a round-robin fashion till one of them returns successfully. Change-Id: Ie46c5f7090d04d8c2e487b209916ae6791e94624 BUG: 847839 Original Author: Venky Shankar <vshankar@redhat.com> Signed-off-by: Avra Sengupta <asengupt@redhat.com> Reviewed-on: http://review.gluster.org/5225 Tested-by: Gluster Build System <jenkins@build.gluster.com> Reviewed-by: Amar Tumballi <amarts@redhat.com> Reviewed-by: Vijay Bellur <vbellur@redhat.com>
generated by cgit (git 2.34.1) at 2024-04-19 17:38:37 +0000