From 9ba8963999bca431ec14a25961a163810cfe1e5b Mon Sep 17 00:00:00 2001 From: Venky Shankar Date: Mon, 27 Apr 2015 21:34:34 +0530 Subject: features/bitrot: Throttle filesystem scrubber This patch introduces multithreaded filesystem scrubber based on throttling option configured for a particular volume. The implementation "logically" breaks scanning and scrubbing with the number of scrubber threads auto-configured depending upon the throttle configuration. Scanning (crawling) is left single threaded (per brick) with entries scrubbed in bulk. On reaching this "bulk" watermark, scanner waits until entries are scrubbed. Bricks for a particular volume have a set of thread(s) assigned for scrubbing, with entries for each brick scrubbed in a round robin fashion to avoid scrub "stalls" when a brick (out of N bricks) is under active scrubbing. This mechanism helps us implement "pause/resume" with ease: all one need to do is to cleanup scrubber threads and let the main scanner thread "wait" untill scrubbing is resumed (where the scrubber thread(s) are spawned again), therefore continuing where we left off (unless we restart the deamons, where crawl initiates from root directory again, but I guess that's OK). [ NOTE: Throttling is optional for the signer daemon, without which it runs full throttle. However, passing "-DBR_RATE_LIMIT_SIGNER" predefined in CFLAGS enables CPU throttling (during checksum calculation) thereby avoiding high CPU usage. ] Subsequent patches would introduce CPU throttling during hash calculation for scrubber. Change-Id: I5701dd6cd4dff27ca3144ac5e3798a2216b39d4f BUG: 1207020 Signed-off-by: Venky Shankar Reviewed-on: http://review.gluster.org/10511 Tested-by: Gluster Build System Reviewed-by: Vijay Bellur --- libglusterfs/src/syncop.c | 17 ----------------- 1 file changed, 17 deletions(-) (limited to 'libglusterfs/src/syncop.c') diff --git a/libglusterfs/src/syncop.c b/libglusterfs/src/syncop.c index cb08b03d44b..81eae5a9105 100644 --- a/libglusterfs/src/syncop.c +++ b/libglusterfs/src/syncop.c @@ -1217,23 +1217,6 @@ syncop_lookup (xlator_t *subvol, loc_t *loc, struct iatt *iatt, return args.op_ret; } -static gf_dirent_t * -entry_copy (gf_dirent_t *source) -{ - gf_dirent_t *sink = NULL; - - sink = gf_dirent_for_name (source->d_name); - - sink->d_off = source->d_off; - sink->d_ino = source->d_ino; - sink->d_type = source->d_type; - sink->d_stat = source->d_stat; - - if (source->inode) - sink->inode = inode_ref (source->inode); - return sink; -} - int32_t syncop_readdirp_cbk (call_frame_t *frame, void *cookie, -- cgit