| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
| |
Signed-off-by: Amar Tumballi <amar@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 734 (keep only the working/usable code in build tree to focus more on development)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=734
|
|
|
|
|
|
|
|
|
|
|
| |
Uninitialized value was causing pthread_create() to fail on solaris
resulting in hung calls
Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 1001 (unclean termination upon getting signalled)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=1001
|
|
|
|
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 329 (Replacing memory allocation functions with mem-type functions)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=329
|
|
|
|
|
|
|
|
| |
Signed-off-by: Amar Tumballi <amar@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 875 (Implement a new protocol to provide proper backward/forward compatibility)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=875
|
|
|
|
|
|
|
|
|
|
|
| |
Memory accounting Changes. Thanks to Vinayak Hegde and Csaba Henk for their
contributions.
Signed-off-by: Vijay Bellur <vijay@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 329 (Replacing memory allocation functions with mem-type functions)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=329
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- libglusterfs
-- call-stub
-- inode
-- protocol
- libglusterfsclient
- cluster/replicate
- cluster/{dht,nufa,switch}
- cluster/unify
- cluster/HA
- cluster/map
- cluster/stripe
- debug/error-gen
- debug/trace
- debug/io-stats
- encryption/rot-13
- features/filter
- features/locks
- features/path-converter
- features/quota
- features/trash
- mount/fuse
- performance/io-threads
- performance/io-cache
- performance/quick-read
- performance/read-ahead
- performance/stat-prefetch
- performance/symlink-cache
- performance/write-behind
- protocol/client
- protocol/server
- storage-posix
Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 361 (GlusterFS 3.0 should work on Mac OS/X)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=361
|
|
|
|
|
|
|
|
|
|
|
|
| |
* conditional for scaling up threads was wrong
* ETIMEDOUT check was performed wrongly
Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 583 (filesystem access hangs while deleting large files)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=583
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch lets io-threads work with a single queue and multiple
threads work on picking the next request from the queue and process
it.
Whenever the number of pending requests in the queue double, a new
worker thread is spawned.
Workers expire after a (configurable) timeout of inactivity
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
Signed-off-by: Anand V. Avati <avati@blackhole.gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 583 (filesystem access hangs while deleting large files)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=583
|
|
|
|
|
|
|
|
| |
Signed-off-by: Raghavendra G <raghavendra@gluster.com>
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 269 (Add a specialized STACK_UNWIND macro for each FOP)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=269
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 292 (Separate readdirp functionality from readdir fop)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=292
|
|
|
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 145 (NFSv3 related additions to 2.1 task list)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=145
|
|
|
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 145 (NFSv3 related additions to 2.1 task list)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=145
|
|
|
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 146 (Add setattr FOP)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=146
|
|
|
|
|
|
|
|
|
| |
It was already implemented but not set to .fxattrop
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
BUG: 267 (Add fxattrop to iothreads)
URL: http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=267
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We're performing a calculation for skewing idle time
that resulted in a timespec.tv_nsec value becoming larger than
1000 million or less than 0, forcing sem_timedwait to return
with an EINVAL instead of waiting for a request notification
from sem_post in iot_notify_worker(). This resulted in a missed
notification that resulted in a hang followed by a timeout
on the protocol/client side.
This commit avoids the over- and under-flow in tv_nsec by
skewing the tv_sec value instead.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It seems that use of mutexes is resulting in pretty high thread
sleep and wake-up cost. What is worse, if a worker thread has
acquired a lock, there is a possibility of the main glusterfs thread
being put to sleep. We change the use of mutexes into spinlock.
At the same time, we cannot anymore use condvars for notification since
the condvar interface depends on mutexes itself. Semaphores come to
out rescue. Luckily, even the pthread semaphores have a timedwait
interface to allow our idle worker threads to make an exit decision.
Further, it is possible that spinlocks are not available on all systems
so all this is curtained behind #defines so we can fall back to
mutexes and condvars implementation.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
| |
We've had complaints from users who've used
autoscaling option with default settings for min and
max threads, about high memory consumption because of
the large default value for max-threads.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit brings in support for allocation of iot_request_t's
in io-threads through the use of the mem-pool. We're hoping
that the overheads of hundreds and thousands of small allocations
can be avoided through this.
The important point to note is that the memory pool is not
for the translator as a whole but there is one small memory
pool for each worker thread. Not only does that help us
avoid malloc overheads for small allocations like iot_request_t
but also avoid contention on the heap data structures when multiple
threads want an iot_request_t from the pool.
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@dev.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch cleans up io-threads behaviour regarding the
range values that can be specified for min-threads
and max-threads. THe major change is that the min threads
have been reduced to 2 to signify that io-threads needs minimum
two threads for its operation, while keeping the default number of
threads at 16. The idea is to decouple the default thread count
from the minimum thread count.
Note to Avati:
This applies over Raghu's indentation and logging take-3 patch.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
| |
-- added some indentation fixes
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
| |
We do not need to explicitly set the stack size to its default
value.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
Going by the memory usage for each threads, it is prudent to
have lower number of threads by default and let users who understand
the memory consequences increase the thread count for themselves.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
The default stack size on Linux is around 8 MiB for each
thread. This is clearly too high for our purpose. This commit reduces
the stack size down to 1 MiB.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
| |
This is being done for the same reason as the
previous refactoring for ordered threads.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch re-factors the exit path of an ordered worker
on a time-out. Earlier we're checking for exit permission
in such a way that required us to release and acquire
the worker lock a second time in the worker loop opening
a window wherein a new request could've been appended to the
request queue.
This patch makes the decision to exit while still holding on to the
worker lock.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
This patch adds sanity check for the ordered worker thread index
returned from the inode's context. If the index is corrupted we
STACK_UNWIND with ECANCELED.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit finally makes the autoscaling feature visible to the user.
Know that we're now using two separate thread-pools, one for data
requests, called ordered thread-pool in io-threads, and the other
for meta-data requests, called un-ordered thread-pool.
We do not expose this information to the user to keep io-threads
simple. Consequently, when the user specifies a min-threads and
max-threads value, the number of threads assigned to each pool
is equal, i.e. both pools start with their min threads set to half of
the option "min-threads" and both scale up their threads at most up to
half of option "max-threads".
Volfile options will be added to the wiki and user-guide.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
| |
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
| |
The default is also to provide no scaling. For both, ordered and
unordered request pools, when scaling is off, we maintain atleast the
minimum number of threads specified in the volfile.
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
| |
Now we have the remaining fops going through the ordered
thread-pool.
To route a request through ordered thread, we use
iot_schedule_ordered(..) and the worker thread for
ordered requests is iot_worker_ordered(..)
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This commit adds everything needed to:
a. Get un-ordered request going through the un-ordered
thread-pool. This happens through, the
iot_schedule_unordered(..). The unordered thread-pool
consists of thread running the iot_worker_unordered(..)
function.
b. Make threads in the un-ordered thread pool start-up
and exit depending on the thread state.
Note that at this point the requests that need
ordering are still going through iot_schedule(..).
Signed-off-by: Anand V. Avati <avati@amp.gluster.com>
|