| Commit message (Collapse) | Author | Age | Files | Lines |
... | |
|
|
|
|
|
|
|
|
|
| |
Change-Id: I4e51961c158c3b5c78791846ca7f0f6cf7fb5c4a
BUG: 1313293
Signed-off-by: Krutika Dhananjay <kdhananj@redhat.com>
Reviewed-on: http://review.gluster.org/13562
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Afr does dict_ref of the xattr_req that comes to it and deletes "gfid-req" key.
Dht uses same dict to send lookup to other subvolumes. So in case of
directories and more than 1 dht subvolumes, second subvolume till the last
subvolume won't get a lookup request with "gfid-req". So gfid reset never
happens on the directories in distributed replicate subvolume for 2nd till last
subvolumes.
Fix:
Make a copy of lookup xattr request.
Also fixed replies_wipe possibly resetting gfid to NULL gfid
BUG: 1312816
Change-Id: Ic16260e5a4664837d069c1dc05b9e96ca05bda88
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13545
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Krutika Dhananjay <kdhananj@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a heal is needed after inode refresh (lookup, read_txn), launch it in
the background instead of blocking the fop (that triggered refresh) until the
heal happens.
afr_replies_interpret() is modified such that the heal is
launched only if atleast one sink brick is up.
Max. no of heals that can happen in parallel is configurable via the
'background-self-heal-count' volume option. Any number greater than that
is put in a wait queue whose length is configurable via
'heal-wait-queue-leng' volume option. If the wait queue is also full,
further heals will be ignored.
Default values: background-self-heal-count=8, heal-wait-queue-leng=128
Change-Id: I1d4a52814cdfd43d90591b6d2ad7b6219937ce70
BUG: 1297172
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/13207
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The trusted.ec.config xattr is not defined for directories. However
sometimes it could be requested because the inode type of a directory
can temporarily be IA_INVAL.
Requesting such xattr using the xattrop fop when it doesn't exist,
returns a config value full of 0's, which is invalid and caused some
fops to fail.
This patch filters out this case by ignoring config xattr == 0.
Change-Id: Ied51c35b313ea8c3eeae27812f9bae61d3808e92
BUG: 1293223
Signed-off-by: Xavier Hernandez <xhernandez@datalab.es>
Reviewed-on: http://review.gluster.org/13446
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Ashish Pandey <aspandey@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Consider the situation where directory path for a truncated
file does not exists under trash directory. In this scenario
after creating the required path we failed to create the
orginal file with proper permissions. Eventhough we try to
fetch permissions from local->origpath, it was never filled
with required value in truncate and ftruncate call paths.
This change will copy original location to local->origpath
inside both fop handling functions.
Change-Id: If5930b6d368d08e58f04db999f3f9edb9250bcb9
BUG: 1309342
Signed-off-by: Anoop C S <anoopcs@redhat.com>
Reviewed-on: http://review.gluster.org/13461
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: jiffin tony Thottan <jthottan@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If an object is marked as bad, then an entry is corresponding to the
bad object is created in the .glusterfs/quarantine directory to help
scrub status. The entry name is the gfid of the corrupted object.
The quarantine handle is removed in below 2 cases.
1) When protocol/server revceives the -ve lookup on an entry whose inode
is there in the inode table (it can happen when the corrupted object
is deleted directly from the backend for recovery purpose) it sends a
forget on the inode and bit-rot-stub removes the quarantine handle in
upon getting the forget.
refer to the below commit
f853ed9c61bf65cb39f859470a8ffe8973818868:
http://review.gluster.org/12743)
2) When bit-rot-stub itself realizes that lookup on a corrupted object
has failed with ENOENT.
But with step1, there is a problem when the bit-rot-stub receives forget
due to lru limit exceeding in the inode table. In such cases, though the
corrupted object is not deleted (either from the mount point or from the
backend), the handle in the quarantine directory is removed and that object
is not shown in the bad objects list in the scrub status command.
So it is better to follow only 2nd step (i.e. bit-rot-stub removing the handle
from the quarantine directory in -ve lookups). Also the handle has to be removed
when a corrupted object is unlinked from the mount point itself.
Change-Id: Ibc3bbaf4bc8a5f8986085e87b729ab912cbf8cf9
BUG: 1308961
Original author: Raghavendra Bhat <raghavendra@redhat.com>
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/13472
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
glusterd_is_brickpath_available
glusterd_is_brickpath_available () used to call realpath() for checking the
whether the new brick path matches with the existing ones. The problem with this
is if the underlying file system is bad for any one of the existing bricks then
realpath() would fail and we wouldn't allow to create the new brick even if it
should be allowed.
Fix is to use string comparison with having a new field real_path in brickinfo
to store the absolute path
Change-Id: I1250ea5345f00fca0f6128056ebd08750d604f0a
BUG: 1299710
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/13258
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ia69cc420ad7b5766d513ea2715bbca50d8d57132
BUG: 1312226
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13530
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I3f5f29378c8c55cc07a83f5fc1506c4d7e9ac09d
BUG: 1309659
Signed-off-by: hari <hgowtham@redhat.com>
Reviewed-on: http://review.gluster.org/13470
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Tested-by: hari gowtham <hari.gowtham005@gmail.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove parent_volinfo irrespective of success or failure
of glusterd_lvm_snapshot_remove(). This prevents the
duplication of the volinfo after the execution of
glusterd_snapshot_restore().
When commit fails on originator node, we don't perform
commit on the other nodes, as a result we don't have a
backup of /var/lib/glusterd/vols/<volname> in the
GLUSTERD_TRASH in other nodes. But in the postvalidate
we try to restore from GLUSTED_TRASH and end up cleaning
up /var/lib/glusterd/vols/<volname>. Hence moved
glusterd_snapshot_backup_vol() from commit to prevalidate,
so that the backup is always available when a cleanup is
needed.
Change-Id: Icd471b23faf02bad680b9a1aadc4a0175f7cce8b
BUG: 1300979
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/13282
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently a main loop in tiering spawns promotion and demotion threads,
and does a join to wait for them to complete. When one of the two
threads takes a long time, the main thread waits for it
before exiting the join. It may wait so long the scheduled time
for the other thread is skipped. In the case of demotion, it may be
a long time before another attempt. This patch fixes that by
making the promotion and demotion activities independant. A side
effect of this change is the logic is significantly simplified.
Change-Id: I1196bd4bbfc95e8aa326a9bd4ebf395032369d1c
BUG: 1306852
Signed-off-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-on: http://review.gluster.org/13433
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Joseph Fernandes
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bricks are started before performing rebalance
Change-Id: I458ea9cd86cf35bdb7d758be55f951ae9f3e66f0
BUG: 1224857
Signed-off-by: Sakshi <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/10906
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As of now volume get framework doesn't consider user.* xattrs to be displayed.
This patch is to include them in volume get output. Please note these options will be
only shown for a given volume name, 'all' as a volume name wouldn't consider them displaying.
Change-Id: Ifc19e89c612e9254d760deaaef50bc1b4bfe02ce
BUG: 1297638
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/13222
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra Talur <rtalur@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently fops like mkdir used the the ctime it gets
after creating the directory entry. But setting layout
also updates the ctime of a directory. Hence DHT must
get the ctime after the setxattr call and unwind with
the latest ctime to avoid mismatch in time seen by
applications like tar.
Change-Id: Iecbbe3aac5244af5da9788b48ccf299ca56b4bae
BUG: 1302948
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/13352
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Allow access to xtime and stime xattrs only
to gsyncd client and mask them for the rest.
This is to prevent afr from performing self
healing on marker xtime and geo-rep stime
xattr which is not expected as each of which
gets updated them from backend brick and
should not be healed.
Change-Id: I24c30f3cfac636a55fd55be989f8db9f8ca10856
BUG: 1296496
Signed-off-by: Kotresh HR <khiremat@redhat.com>
Reviewed-on: http://review.gluster.org/13242
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I1a7b23db8f196616be338baeb8f7f56572a05c90
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/12908
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As per 'man 3 fcntl',
"If l_len is positive, the area affected shall start at
l_start and end at l_start+l_len−1. If l_len is negative,
the area affected shall start at l_start+l_len and end at
l_start−1. Locks may start and extend beyond the current
end of a file, but shall not extend before the beginning
of the file."
Currently we return EINVAL if l_len is found to be negative.
Fixed the same as mentioned in the man page.
Change-Id: I493ce202c543185fc4ae7266d1aaf9d7e2a66991
BUG: 1241104
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-on: http://review.gluster.org/11613
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I2458e18197bdf7565563a85e9021b5b2850c1825
BUG: 1303945
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/13392
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Xavier Hernandez <xhernandez@datalab.es>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Current glusterd code base having memory leak. This is because of
memory allocate by dict_allocate_and_serialize function in
"gd_syncop_mgmt_v3_lock" and "gd_syncop_mgmt_v3_unlock"
function is not freeing up meory upon exit.
Fix is to free the memory after exit of the above function.
Thanx Carlos and Roman for finding out the issue and fix.
Change-Id: Id67aa794c84969830ca7ea8c2374f80c64d7a639
BUG: 1287517
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Signed-off-by: Carlos Chinea <carlos.chinea@nokia.com>
Signed-off-by: Roman Tereshonkov <roman.tereshonkov@nokia.com>
Reviewed-on: http://review.gluster.org/12927
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fix unused return value from snprintf.
Also minor cleanup, including safe handling of truncated output by
snprintf.
Change-Id: Ibec8c7e6cc4ef07b6535d01a0719359fa3ef8030
BUG: 1310755
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/13486
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
op on glusterd restart
While remove brick is in progress, if glusterd is restarted since decommission
flag is not persisted in the store the same value is not retained back resulting
in glusterd not blocking remove brick commit when rebalance is already in
progress.
Change-Id: Ibbf12f3792d65ab1293fad1e368568be141a1cd6
BUG: 1303269
Signed-off-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-on: http://review.gluster.org/13323
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Gaurav Kumar Garg <ggarg@redhat.com>
Reviewed-by: mohammed rafi kc <rkavunga@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
During reblance restart after glusterd restarted, we are not
connecting to rebalance process from glusterd, because the
defrag variable in volinfo will be null.
Initializing the variable will connect the rpc
Change-Id: Id820cad6a3634a9fc976427fbe1c45844d3d4b9b
BUG: 1303028
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/13319
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Change-Id: I56e5f5696fbbb4290f95adb9b68f9e43ed037ed3
BUG: 1189473
Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
Reviewed-on: http://review.gluster.org/9861
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently when server quorum is not met then upon executing
# gluster volume start [force] command its starting the volume.
With this patch if server side quorum is not met then it will
prevent starting of the volume.
Change-Id: I39734b2dcf8e90c3c68bf2762d8350aecc82cc38
BUG: 1308402
Signed-off-by: Gaurav Kumar Garg <ggarg@redhat.com>
Reviewed-on: http://review.gluster.org/13442
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently DHT takes non-blocking locks for file rename.
Due to this during parallel renames some clients fail
with EBUSY or ESTALE errors. Hence to avoid application
discontinuity file rename must take blocking inode locks.
Change-Id: I986e9d08b3be359f20b1a3e1564e049b0f3dffd3
BUG: 1304966
Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
Reviewed-on: http://review.gluster.org/13366
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
Glusterd not working using ipv6 transport. The idea is with proper glusterd.vol configuration,
1. glusterd needs to listen on default port (240007) as IPv6 TCP listner.
2. Volume creation/deletion/mounting/add-bricks/delete-bricks/peer-probe
needs to work using ipv6 addresses.
3. Bricks needs to listen on ipv6 addresses.
All the above functionality is needed to say that glusterd supports ipv6 transport and this is broken.
Fix:
When "option transport.address-family inet6" option is present in glusterd.vol
file, it is made sure that glusterd creates listeners using ipv6 sockets only and also the same information is saved
inside brick volume files used by glusterfsd brick process when they are starting.
Tests Run:
Regression tests using ./run-tests.sh
IPv4: Ran manually till tests/basic/rpm.t .
IPv6: (Need to add the above mentioned config and also add an entry for "hostname ::1" in /etc/hosts)
Started failing at ./tests/basic/glusterd/arbiter-volume-probe.t and ran successfully till here
Unit Tests using Ipv6
peer probe
add-bricks
remove-bricks
create volume
replace-bricks
start volume
stop volume
delete volume
Change-Id: Iebc96e6cce748b5924ce5da17b0114600ec70a6e
BUG: 1117886
Signed-off-by: Nithin D <nithind1988@yahoo.in>
Reviewed-on: http://review.gluster.org/11988
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Atin Mukherjee <amukherj@redhat.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Perform a pmap_registry_remove on a stale registry
entry from the previous run of snapd, which makes the
query callback always return the older port number.
Change-Id: I7fc504d2b09d32a9c945ec2c33e0e092a0d186ec
BUG: 1306193
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/13420
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For *create* fops (CREATE, MKDIR, MKNOD), we invalidate the parent
entry. Hence send parent attributes in the stat field.
Also "UP_PARENT_DENTRY_FLAGS" has to be set only for the fops which shall
result in two invalidations requests - one for the inode on which fop is
being performed and another on parent entry.
In case of CREATE/MKDIR/MKNOD fops, there shall be only one invalidation
request sent, that too on parent inode. We send invalidation directly on
parent inode's gfid. So there is no necessity to set these flags which
when set shall endup invalidating the parent's parent entry.
Change-Id: I7514ee08382081e3e060818ede497dbca26987dc
BUG: 1291259
Signed-off-by: Soumya Koduri <skoduri@redhat.com>
Reviewed-on: http://review.gluster.org/12962
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Niels de Vos <ndevos@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
cli/src/cli-cmd-parser.c (chenk)
cli/src/cli-xml-output.c (spandit)
cli/src/cli.c (chenk)
libglusterfs/src/common-utils.c (vmallika)
libglusterfs/src/gfdb/gfdb_sqlite3.c (jfernand +1)
rpc/rpc-transport/socket/src/socket.c (?)
xlators/cluster/afr/src/afr-transaction.c (?)
xlators/cluster/dht/src/dht-common.h (srangana +2)
xlators/cluster/dht/src/dht-selfheal.c (srangana +2)
xlators/debug/io-stats/src/io-stats.c (R. Wareing)
xlators/features/barrier/src/barrier.c (vshastry)
xlators/features/bit-rot/src/bitd/bit-rot-scrub.h (vshankar +1)
xlators/features/shard/src/shard.c (kdhananj +1)
xlators/mgmt/glusterd/src/glusterd-ganesha.c (skoduri)
xlators/mgmt/glusterd/src/glusterd-handler.c (atinmu)
xlators/mgmt/glusterd/src/glusterd-op-sm.h (atinmu)
xlators/mgmt/glusterd/src/glusterd-snapshot.c (spandit)
xlators/mgmt/glusterd/src/glusterd-syncop.c (atinmu)
xlators/mgmt/glusterd/src/glusterd-volgen.c (atinmu)
xlators/protocol/client/src/client-messages.h (mselvaga +1)
xlators/storage/bd/src/bd-helper.c (M. Mohan Kumar)
xlators/storage/bd/src/bd.c (M. Mohan Kumar)
xlators/storage/posix/src/posix.c (nbalacha +1)
Change-Id: I85934fbcaf485932136ef3acd206f6ebecde61dd
BUG: 1293133
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/13031
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
fixes for various warnings reported by cppcheck
N.B. cppcheck output is in the bugzilla
Change-Id: I33acec127bc4536935fdd8d52a0c490ec54d50b2
BUG: 1292954
Signed-off-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Reviewed-on: http://review.gluster.org/13006
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If there is a .snaps directory pre-exists and when USS is enabled,
it should not be listed with 'ls -a'
Change-Id: I1c43e2decc0bbbd3213b190b675e3a32d04b22d3
BUG: 1303828
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/13330
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Running "rm -rf" on a tiered volume sometimes caused
the client to crash because dht_readdirp_cbk referenced
a NULL layout for the hot tier subvol.
Now, entries are skipped if the layout is NULL. This
can cause "rm -rf" to fail with ENOTEMPTY rmdir failures.
Change-Id: Idd71a9d0f7ee712899cc7113bbf2cd3dcb25808b
BUG: 1307208
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/13440
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
NSR needs logging that is different than our existing changelog in
several ways:
* Full data, not just metadata
* Pre-op, not post-op
* High performance
* Supports the concept of time-bounded "terms"
Others (for example EC) might need the same thing. This patch adds such
a translator. It also adds code to dump the resulting journals, and to replay
them using syncops, plus (very rudimentary) tests for all of the above.
Change-Id: I29680a1b4e0a9e7d5a8497fef302c46434b86636
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/12450
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
If index heal is launched when some of the bricks are down, glustershd of that
node sends a -1 op_ret to glusterd which eventually propagates it to the CLI.
Also, glusterd sometimes sends an err_str and sometimes not (depending on the
failure happening in the brick-op phase or commit-op phase). So the message that
gets displayed varies in each case:
"Launching heal operation to perform index self heal on volume testvol has been
unsuccessful"
(OR)
"Commit failed on <host>. Please check log file for details."
Fix:
1. Modify afr_xl_op() to return -1 even if index healing of atleast one brick
fails.
2. Ignore glusterd's error string in gf_cli_heal_volume_cbk and print a more
meaningful message.
The patch also fixes a bug in glusterfs_handle_translator_op() where if we
encounter an error in notify of one xlator, we break out of the loop instead of
sending the notify to other xlators.
Change-Id: I957f6c4b4d0a45453ffd5488e425cab5a3e0acca
BUG: 1302291
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Reviewed-on: http://review.gluster.org/13303
Reviewed-by: Anuradha Talur <atalur@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The NSR-server with this patch, appoints the first node
on every replica subvolume, as the leader for that subvolume.
On receiving a 'write' fop, the leader first checks if there is
quorum in the replica subvolume to proceeed. In case there isn't
it fails with EROFS.
If there is quorum, the leader forwards the fop to the
followers. The followers on receiving the fop, perform the
operation, and based on the success or failure of the outcome
send a +ve or a -ve ack to the leader.
The leader after receiving acks from the followers performs a
quorum check of the acks, to see if it should even try to perform
the fop. If quorum is not being met, and the leader's outcome
wouldn't affect quorum, then it would send -ve ack to the client
without even performing the fop. If quorum is being met, the leader
will then try the fop on itself, and based on it's outcome perform
a quorum check of all the acks received (this time, including it's
own). Based on the result of the quorum check (irrespective of the
outcome on the leader), a +ve or -ve ack is send back to the client.
Change-Id: I860654b74c53e9b139b37dba43848e5504df6dce
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/12705
Smoke: Gluster Build System <jenkins@build.gluster.com>
Tested-by: Jeff Darcy <jdarcy@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The wrong variable was being checked to determine
the watermark value.
Change-Id: If4c97fa70b772187f1fcbdf5193e077cb356a8b1
BUG: 1303895
Signed-off-by: N Balachandran <nbalacha@redhat.com>
Reviewed-on: http://review.gluster.org/13357
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Dan Lambright <dlambrig@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Linux FUSE kernel module has gained support for passing SEEK_HOLE
and SEEK_DATA on through lseek(). This can greatly improve performance
when working with sparse files.
Linux FUSE introduced support for lseek() with version 4.5. The commit
in mainline Linux is 0b5da8db145bfd44266ac964a2636a0cf8d7c286.
URL: http://thread.gmane.org/gmane.comp.file-systems.fuse.devel/14752
Change-Id: I12496d788e59461a3023ddd30e0ea3179007f77e
BUG: 1220173
Signed-off-by: Ravishankar N <ravishankar@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/11474
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When quota is enabled on a tiered volume, incorrect data usage is
shown, it is because, during the process of migrating files in tiering,
we are accounting both for the src file and dst file at some point. By the
time we make the srcfile as a T file, marker has already accounted the
contri and has updated it's parent and also we are not accounting for the
truncate operation done, which accounts to incorrect data usage even after
unlinking the file. The size can increase drastically with multiple promotes
and demotes since the contri keeps changing and the parent is being updated.
Change-Id: Ie567228786713d7dc257ff374a69ad3be40f9e82
BUG: 1304970
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-on: http://review.gluster.org/13363
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Whenever a new brick is added, quota related xattr's
should be healed but currently, the xattr
"quota.limit-objects.<suffix>" needed for inode-quota is
not being healed.
The patch fixes this issue.
Change-Id: I1e7b229126f7b058642bbc3fb5c109bfd8925325
BUG: 1302257
Signed-off-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-on: http://review.gluster.org/13299
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Vijaikumar Mallikarjuna <vmallika@redhat.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The only lseek() options we need are SEEK_HOLE and SEEK_DATA.
Change-Id: I5d15533c53fd710497f97c3cb4a8ea29fba47271
BUG: 1220173
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Reviewed-on: http://review.gluster.org/11484
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We use python-config to get recommended CFLAGS. It provides
-D_FORTIFY_SOURCE=2 by default that conflicts with our
--enable-debug option or a developer provided no-optimization option.
Hence, filter it out from default CFLAGS.
Change-Id: Id80196baeb55415b1ea334e7b17143e56dfbadb3
BUG: 1283948
Co-authored-by: Kaleb S KEITHLEY <kkeithle@redhat.com>
Signed-off-by: Raghavendra Talur <rtalur@redhat.com>
Reviewed-on: http://review.gluster.org/12707
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
A check for dictionary is required before calling handling_other_acl_related_xattr()
in posix_setxattr_cbk(). Otherwise following message will be poped up on every settxattr
call.
[2016-01-31 17:38:48.027792] I [dict.c:473:dict_get]
(-->/usr/lib/x86_64-linux-gnu/glusterfs/3.7.6/xlator/system/posix-acl.so
(posix_acl_setxattr_cbk+0x26) [0x7f2334c5c166] -->
/usr/lib/x86_64-linux-gnu/glusterfs/3.7.6/xlator/system/posix-acl.so
(handling_other_acl_related_xattr+0xb0) [0x7f2334c5c0f0]
-->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(dict_get+0x93)
[0x7f233c04b0c3] ) 0-dict: !this ||
key=system.posix_acl_default [Invalid argument]
Change-Id: I93f7e0f764c94dce38bdde95a68341f69a52122d
BUG: 1303501
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
Reviewed-on: http://review.gluster.org/13325
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This version of the client checks if the error returned
is EREMOTE or ENOTCONN, and if not unwnds the error back
In case of a EREMOTE or ENOTCONN error, it retries on
all the bricks in the replica subgroup, and if the error
still persists, it waits for a sec before going through the
same exercise again.
Change-Id: I916bed32f0820f381dd60fdde3d05b71c69a34dc
Signed-off-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-on: http://review.gluster.org/12388
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Jeff Darcy <jdarcy@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
USS option features.snapshot-directory
contains only 'alphanum, -, _, .'
starts with dot (.)
value cannot exceed 255 characters
and throws error for any other argument.
Change-Id: Iad64635206ddf5599351020d99aafb3dd9d17bc1
BUG: 1168819
Signed-off-by: vmallika <vmallika@redhat.com>
Reviewed-on: http://review.gluster.org/9209
Smoke: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
Reviewed-by: Manikandan Selvaganesh <mselvaga@redhat.com>
Reviewed-by: Rajesh Joseph <rjoseph@redhat.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem:
After a replace brick command, newly added
brick does not contain data which existed
on old brick.
Solution:
Do getxattr after initialization of all the
bricks. This will trigger heal for brick root
as soon as it finds the version mismatch on
newly added brick.
Removing tests from ec-new-entry.t which were
required to simulate automation of heal after
replace brick.
Change-Id: I08e3dfa565374097f6c08856325ea77727437e11
BUG: 1304686
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/13353
Reviewed-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Tested-by: Pranith Kumar Karampuri <pkarampu@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also printing flags correctly in statedump now
Change-Id: Ibfdd74aab5643ecc47d0a88f109d5d1050685f5a
BUG: 1294051
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13076
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Allows the user to convert an afr-volume to a nsr-volume
by using cluster.nsr option in the volume set command
gluster volume set <volname> cluster.nsr <on/off>
Change-Id: Ia1c5aa89d27535f7275d474cf312dc5efb8e222f
BUG: 1158654
Signed-off-by: Jeff Darcy <jdarcy@redhat.com>
Reviewed-on: http://review.gluster.org/12943
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Avra Sengupta <asengupt@redhat.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Debian Wheezy, which has urcu-0.6 doesn't ship the pkg-config files for
urcu. So when I was checking support, I didn't bother testing Wheezy or
urcu-0.6 as I didn't want to spend time trying to build without
pkg-config support.
But it turns out that urcu-0.6 supports the APIs being used by GlusterD.
This patch improves the configure script to manually search and set URCU
build flags if pkg-config search fails.
Change-Id: I8c2ae748b58e94d89aef61da6fb3c660710a5f3b
BUG: 1304348
Signed-off-by: Kaushal M <kaushal@redhat.com>
Reviewed-on: http://review.gluster.org/13345
Reviewed-by: Kaleb KEITHLEY <kkeithle@redhat.com>
Smoke: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Ec manager shouldn't return -ve states, but it is, fixed that.
Change-Id: I3f97c6ba2dbf9da724e8e1ee9b2c9da73f40013d
BUG: 1300929
Signed-off-by: Pranith Kumar K <pkarampu@redhat.com>
Reviewed-on: http://review.gluster.org/13278
Tested-by: Xavier Hernandez <xhernandez@datalab.es>
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Problem: Immediately after starting a disperse volume (2+1)
kill one brick and just after that try to mount it
through fuse. This lead to crash.
Our test scripts use process statedumps to determine various things
like whether they are up, connected to bricks etc. It takes some time
for an active_subvol to be be associated with fuse even after mount
process is daemonized. This time is normally a function of completion
of handshake with bricks. So, if we try to take statedump in this time
window, fuse wouldn't have an active_subvol associated with it leading
to this crash.
This happened while executing ec-notify.t, which contains above steps.
Solution: Check priv and priv->active_subvol for NULL before
inode_table_dump. If priv->active_subvol is null its perfectly fine to
skip dumping of inode table as inode table is associated with an
active_subvol. A Null active_subvol indicates initialization in
progress and fuse wouldn't even have started reading requests from
/dev/fuse and hence there wouldn't be any inodes or file system
activity.
Change-Id: I323a154789edf8182dbd1ac5ec7ae07bf59b2060
BUG: 1299410
Signed-off-by: Ashish Pandey <aspandey@redhat.com>
Reviewed-on: http://review.gluster.org/13253
Smoke: Gluster Build System <jenkins@build.gluster.com>
CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
|