summaryrefslogtreecommitdiffstats
path: root/xlators/performance/io-threads
Commit message (Collapse)AuthorAgeFilesLines
* Memory accounting changesVijay Bellur2010-04-233-3/+54
| | | | | | | | | | | Memory accounting Changes. Thanks to Vinayak Hegde and Csaba Henk for their contributions. Signed-off-by: Vijay Bellur <vijay@gluster.com> Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 329 (Replacing memory allocation functions with mem-type functions) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=329
* iatt: changes across the codebaseAnand V. Avati2010-03-161-36/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - libglusterfs -- call-stub -- inode -- protocol - libglusterfsclient - cluster/replicate - cluster/{dht,nufa,switch} - cluster/unify - cluster/HA - cluster/map - cluster/stripe - debug/error-gen - debug/trace - debug/io-stats - encryption/rot-13 - features/filter - features/locks - features/path-converter - features/quota - features/trash - mount/fuse - performance/io-threads - performance/io-cache - performance/quick-read - performance/read-ahead - performance/stat-prefetch - performance/symlink-cache - performance/write-behind - protocol/client - protocol/server - storage-posix Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com> Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 361 (GlusterFS 3.0 should work on Mac OS/X) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=361
* io-threads: more fixesAnand Avati2010-02-272-7/+14
| | | | | | | | | | | | * conditional for scaling up threads was wrong * ETIMEDOUT check was performed wrongly Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com> Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 583 (filesystem access hangs while deleting large files) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=583
* io-threads: single queue/multi-thread modelAnand Avati2010-02-222-956/+154
| | | | | | | | | | | | | | | | | | This patch lets io-threads work with a single queue and multiple threads work on picking the next request from the queue and process it. Whenever the number of pending requests in the queue double, a new worker thread is spawned. Workers expire after a (configurable) timeout of inactivity Signed-off-by: Anand V. Avati <avati@amp.gluster.com> Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com> Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 583 (filesystem access hangs while deleting large files) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=583
* performance/io-threads: use STACK_UNWIND_STRICT for unwinding.Raghavendra G2009-11-241-90/+102
| | | | | | | | Signed-off-by: Raghavendra G <raghavendra@gluster.com> Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 269 (Add a specialized STACK_UNWIND macro for each FOP) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=269
* Changed occurrences of Z Research to Gluster.Vijay Bellur2009-10-072-2/+2
| | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: Support readdirp fopShehjar Tikoo2009-10-061-0/+51
| | | | | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 292 (Separate readdirp functionality from readdir fop) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=292
* io-threads: NFS-friendly logic changesShehjar Tikoo2009-10-011-25/+31
| | | | | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 145 (NFSv3 related additions to 2.1 task list) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=145
* Global: NFS-friendly prototype changesShehjar Tikoo2009-10-011-22/+36
| | | | | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 145 (NFSv3 related additions to 2.1 task list) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=145
* Global: Introduce setattr and fsetattr fopsShehjar Tikoo2009-10-011-207/+33
| | | | | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 146 (Add setattr FOP) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=146
* Add iot_fxattrop to io-threadsPavan Sondur2009-09-171-0/+1
| | | | | | | | | It was already implemented but not set to .fxattrop Signed-off-by: Anand V. Avati <avati@dev.gluster.com> BUG: 267 (Add fxattrop to iothreads) URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=267
* io-threads: Fix missed notification in sem_timedwaitShehjar Tikoo2009-06-112-5/+4
| | | | | | | | | | | | | | | We're performing a calculation for skewing idle time that resulted in a timespec.tv_nsec value becoming larger than 1000 million or less than 0, forcing sem_timedwait to return with an EINVAL instead of waiting for a request notification from sem_post in iot_notify_worker(). This resulted in a missed notification that resulted in a hang followed by a timeout on the protocol/client side. This commit avoids the over- and under-flow in tv_nsec by skewing the tv_sec value instead. Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* mem-pool,io-threads: Destroy mem-pool on deallocationShehjar Tikoo2009-05-211-0/+1
| | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: Use boolean enums instead of integer 1Shehjar Tikoo2009-05-201-2/+2
| | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: Change mutexes/condvars to spinlocks/semaphoresShehjar Tikoo2009-05-202-36/+80
| | | | | | | | | | | | | | | | | It seems that use of mutexes is resulting in pretty high thread sleep and wake-up cost. What is worse, if a worker thread has acquired a lock, there is a possibility of the main glusterfs thread being put to sleep. We change the use of mutexes into spinlock. At the same time, we cannot anymore use condvars for notification since the condvar interface depends on mutexes itself. Semaphores come to out rescue. Luckily, even the pthread semaphores have a timedwait interface to allow our idle worker threads to make an exit decision. Further, it is possible that spinlocks are not available on all systems so all this is curtained behind #defines so we can fall back to mutexes and condvars implementation. Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: Reduce default max for threads to 64Shehjar Tikoo2009-05-201-1/+1
| | | | | | | | | We've had complaints from users who've used autoscaling option with default settings for min and max threads, about high memory consumption because of the large default value for max-threads. Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: Support mem-pool allocator for iot_request_tShehjar Tikoo2009-05-202-23/+44
| | | | | | | | | | | | | | | | This commit brings in support for allocation of iot_request_t's in io-threads through the use of the mem-pool. We're hoping that the overheads of hundreds and thousands of small allocations can be avoided through this. The important point to note is that the memory pool is not for the translator as a whole but there is one small memory pool for each worker thread. Not only does that help us avoid malloc overheads for small allocations like iot_request_t but also avoid contention on the heap data structures when multiple threads want an iot_request_t from the pool. Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: Add graceful shutdown of worker threadsRaghavendra G2009-05-202-25/+79
| | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: handle memory allocation failuresRaghavendra G2009-05-201-197/+760
| | | | Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
* io-threads: Clarify thread count rangeShehjar Tikoo2009-05-052-16/+23
| | | | | | | | | | | | | | | This patch cleans up io-threads behaviour regarding the range values that can be specified for min-threads and max-threads. THe major change is that the min threads have been reduced to 2 to signify that io-threads needs minimum two threads for its operation, while keeping the default number of threads at 16. The idea is to decouple the default thread count from the minimum thread count. Note to Avati: This applies over Raghu's indentation and logging take-3 patch. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: indentation and logging fixesRaghavendra G2009-05-052-199/+250
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add fxattrop fopShehjar Tikoo2009-04-221-666/+565
| | | | | | -- added some indentation fixes Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add xattrop fopShehjar Tikoo2009-04-221-0/+44
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add readdir fopShehjar Tikoo2009-04-221-0/+36
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add removexattr fopShehjar Tikoo2009-04-221-0/+45
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add fsetxattr fopShehjar Tikoo2009-04-221-0/+35
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add fgetxattr fopShehjar Tikoo2009-04-221-0/+34
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add getxattr fopShehjar Tikoo2009-04-221-0/+44
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add setxattr fopShehjar Tikoo2009-04-221-0/+45
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add statfs fopShehjar Tikoo2009-04-221-1/+34
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add fsyncdir fopShehjar Tikoo2009-04-221-0/+34
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add opendir fopShehjar Tikoo2009-04-221-0/+34
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add link fopShehjar Tikoo2009-04-221-0/+37
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Remove unnecessary call to pthread_attr_setstacksizeShehjar Tikoo2009-04-121-4/+0
| | | | | | | We do not need to explicitly set the stack size to its default value. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* updated performance/io-threads with new readv writev prototypesAnand V. Avati2009-04-121-6/+10
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Reduce default min and max threadsShehjar Tikoo2009-04-101-2/+2
| | | | | | | | Going by the memory usage for each threads, it is prudent to have lower number of threads by default and let users who understand the memory consequences increase the thread count for themselves. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Use non-default thread stack sizeShehjar Tikoo2009-04-102-1/+27
| | | | | | | | The default stack size on Linux is around 8 MiB for each thread. This is clearly too high for our purpose. This commit reduces the stack size down to 1 MiB. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Refactor un-ordered worker exit pathShehjar Tikoo2009-04-081-33/+17
| | | | | | | This is being done for the same reason as the previous refactoring for ordered threads. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Refactor ordered worker exit pathShehjar Tikoo2009-04-081-34/+25
| | | | | | | | | | | | | | This patch re-factors the exit path of an ordered worker on a time-out. Earlier we're checking for exit permission in such a way that required us to release and acquire the worker lock a second time in the worker loop opening a window wherein a new request could've been appended to the request queue. This patch makes the decision to exit while still holding on to the worker lock. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Cleaner stub destructionShehjar Tikoo2009-04-061-0/+3
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Sanity check worker thread idxShehjar Tikoo2009-04-061-20/+62
| | | | | | | | This patch adds sanity check for the ordered worker thread index returned from the inode's context. If the index is corrupted we STACK_UNWIND with ECANCELED. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Support autoscaling volfile optionsShehjar Tikoo2009-04-022-24/+104
| | | | | | | | | | | | | | | | | This commit finally makes the autoscaling feature visible to the user. Know that we're now using two separate thread-pools, one for data requests, called ordered thread-pool in io-threads, and the other for meta-data requests, called un-ordered thread-pool. We do not expose this information to the user to keep io-threads simple. Consequently, when the user specifies a min-threads and max-threads value, the number of threads assigned to each pool is equal, i.e. both pools start with their min threads set to half of the option "min-threads" and both scale up their threads at most up to half of option "max-threads". Volfile options will be added to the wiki and user-guide. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Cleanup request queueing,thread firingShehjar Tikoo2009-04-021-30/+38
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Allow scaling to be switched offShehjar Tikoo2009-04-022-11/+73
| | | | | | | | The default is also to provide no scaling. For both, ordered and unordered request pools, when scaling is off, we maintain atleast the minimum number of threads specified in the volfile. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add ordered threadpool state and codeShehjar Tikoo2009-04-022-58/+182
| | | | | | | | | | | Now we have the remaining fops going through the ordered thread-pool. To route a request through ordered thread, we use iot_schedule_ordered(..) and the worker thread for ordered requests is iot_worker_ordered(..) Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Add un-ordered thread-pool.Shehjar Tikoo2009-04-022-98/+334
| | | | | | | | | | | | | | | | | | This commit adds everything needed to: a. Get un-ordered request going through the un-ordered thread-pool. This happens through, the iot_schedule_unordered(..). The unordered thread-pool consists of thread running the iot_worker_unordered(..) function. b. Make threads in the un-ordered thread pool start-up and exit depending on the thread state. Note that at this point the requests that need ordering are still going through iot_schedule(..). Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Classify requests for threadpool typeShehjar Tikoo2009-04-021-26/+33
| | | | | | | | | | | | | | | | | New io-threads will serve requests through two separate threadpools. One thread pool for requests that must be ordered on a file that is open. so that the server can process the requests in the order they were entered in the requests queue, and not in the order the io-thread is able to send a request, which in turn is determined by how the thread gets scheduled. This can also be called the data-intensive ops thread pool. Second thread-pool for requests that dont care about ordering, i.e. requests like lookup, open, create, mkdir, etc. Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Wire in support for rename fopShehjar Tikoo2009-04-021-0/+55
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Wire in support for symlink fopShehjar Tikoo2009-04-021-0/+53
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
* io-threads: Wire in support for rmdir fopShehjar Tikoo2009-04-021-0/+40
| | | | Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
generated by cgit (git 2.34.1) at 2024-11-19 19:07:37 +0000