glusterfs.git/rpc/xdr/src/Makefile.am, branch v3.10.10

build: libgfxdr.so calls GF_FREE(), needs to link with -lglusterfs

2017-03-21T17:12:15+00:00

build: libgfxdr.so calls GF_FREE(), needs to link with -lglusterfs

The previous change to remove the xdrgen script exposed (or
created) a recursive build dependency: libglusterfs needs the
generated headers, and libgfxdr should be linked with libglusterfs
for GF_FREE/__gf_free.

(Much grumbling about libglusterfs being the kitchen sink of gluster
elided. This would not be necessary if there were two more more libs,
a gluster "runtime" library with common gluster code shared by the
xlators and daemons, and a utility library with things like the
rbtree, memory allocation, and whatnot.)

So. Link at build time or link at runtime? For truth-and-beauty, link
with libglusterfs.so at build time. Without truth-and-beauty, don't
link with libglusterfs and rely on other the libs that link with
libglusterfs to provide resolution of __gf_free().

Truth-and-beauty it is. But how to generate the headers first, then
build libglusterfs, then come back and build libgfxdr? Autotools is a
maze of twisty passages, all different. Things that work with gnu
make on linux don't work with the BSD make. Finally I hit on this
solution. Add a shadow directory where make only generates the headers,
then build libglusterfs using the generated headers, and finally build
libgfxdr and link with libglusterfs.

See original BZ 1330604
change http://review.gluster.org/14085

master BZ 1429696
master change: https://review.gluster.org/#/c/16873/

Change-Id: Iede8a30e3103176cb8f0b054885f30fcb352492b
BUG: 1430512
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: https://review.gluster.org/16874
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System

build: rpc/xdr .c and .h files are regenerated unnecessarily

2016-09-20T11:45:45+00:00

.c and .h files are blindly (re)generated. If you run `make`
followed by a second `make` or `make install` the subsequent
`make` will unnecessarily recompile everything because of the
redundant regeneration of the .c and .h files are now newer
than the prior builds .o files

Change-Id: I7e477bcdcc20869e144ada7e6d91e7221b8ee71f
BUG: 1377341
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/15530
Smoke: Gluster Build System 
Reviewed-by: Niels de Vos 
Tested-by: Niels de Vos 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Anoop C S

build: out-of-tree builds generates files in the wrong directory

2016-09-18T16:34:37+00:00

And minor cleanup of a few of the Makefile.am files while we're
at it.

Rewrite the make rules to do what xdrgen does. Now we can get rid
of xdrgen.

Note 1. netbsd6's sed doesn't do -i. Why are we still running
smoke tests on netbsd6 and not netbsd7? We barely support netbsd7
as it is.

Note 2. Why is/was libgfxdr.so (.../rpc/xdr/src/...) linked with
libglusterfs? A cut-and-paste mistake? It has no references to
symbols in libglusterfs.

Note3. "/#ifndef\|#define\|#endif/" (note the '\'s) is a _basic_
regex that matches the same lines as the _extended_ regex
"/#(ifndef|define|endif)/". To match the extended regex sed needs to
be run with -r on Linux; with -E on *BSD. However NetBSD's and
FreeBSD's sed helpfully also provide -r for compatibility. Using a
basic regex avoids having to use a kludge in order to run sed with
the correct option on OS X.

Note 4. Not copying the bit of xdrgen that inserts copyright/license
boilerplate. AFAIK it's silly to pretend that machine generated
files like these can be copyrighted or need license boilerplate.
The XDR source files have their own copyright and license; and
their copyrights are bound to be more up to date than old
boilerplate inserted by a script. From what I've seen of other
Open Source projects -- e.g. gcc and its C parser files generated
by yacc and lex -- IIRC they don't bother to add copyright/license
boilerplate to their generated files.

It appears that it's a long-standing feature of make (SysV, BSD,
gnu) for out-of-tree builds to helpfully pretend that the source
files it can find in the VPATH "exist" as if they are in the $cwd.
rpcgen doesn't work well in this situation and generates files
with "bad" #include directives.

E.g. if you `rpcgen ../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.x`,
you get an #include directive in the generated .c file like this:

  ...
  #include "../../../../$srcdir/rpc/xdr/src/glusterfs3-xdr.h"
  ...

which (obviously) results in compile errors on out-of-tree build
because the (generated) header file doesn't exist at that location.
Compared to `rpcgen ./glusterfs3-xdr.x` where you get:

  ...
  #include "glusterfs3-xdr.h"
  ...

Which is what we need. We have to resort to some Stupid Make Tricks
like the addition of various .PHONY targets to work around the VPATH
"help".

Warning: When doing an in-tree build, -I$(top_builddir)/rpc/xdr/...
looks exactly like -I$(top_srcdir)/rpc/xdr/...  Don't be fooled though.
And don't delete the -I$(top_builddir)/rpc/xdr/... bits

Change-Id: Iba6ab96b2d0a17c5a7e9f92233993b318858b62e
BUG: 1330604
Signed-off-by: Kaleb S KEITHLEY 
Reviewed-on: http://review.gluster.org/14085
Tested-by: Niels de Vos 
Smoke: Gluster Build System 
NetBSD-regression: NetBSD Build System 
CentOS-regression: Gluster Build System 
Reviewed-by: Niels de Vos

rpc: split FOPs enum from glusterfs.h

2016-04-28T20:51:12+00:00

Moving the enumeration of FOPs and some of the other parts that are
defining the network protocol to the rpc/xdr/ section. These structures
need some care when modifications are made, moving them out of the
common glusterfs.h header helps with that.

The protocol definition structures are generated in a new glusterfs-fops
header. This file is present in rpc/xdr/src/ and libglusterfs/src/, it
is a little ugly, but prevents the need to update all Makefile.am files
with the additional -I option for finding the new header file.

The generation of the .c and .h files from the .x descriptions needed
small modifications to accommodate these changes. The build/xdrgen
script was improved slightly for this. The .c and .h files are
incorrectly in the $(top_srcdir), instead of $(top_builddir). This is
an existing issue, and bug 1330604 has been filed to get that addressed.

Change-Id: I98fc8cf7e4b631082c7b203b5a0a77111bec1fb9
BUG: 1328502
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/14032
Smoke: Gluster Build System 
CentOS-regression: Gluster Build System 
NetBSD-regression: NetBSD Build System 
Reviewed-by: Jeff Darcy

build: install and package header files more conventionally

2015-11-16T09:15:04+00:00

The current way we install and package header files for the -devel
package is a hack. This patch uses more conventional autoconf, libtool,
and rpmbuild idioms to package -devel headers and libraries.

Change-Id: I63ffb3460f5c12b6b355493bd00824ac9e5354c5
BUG: 1271907
Signed-off-by: Kaleb S. KEITHLEY 
Reviewed-on: http://review.gluster.org/12360
Tested-by: NetBSD Build System 
Tested-by: Gluster Build System 
Reviewed-by: Niels de Vos

build: do not include rpcgen generated files in "make dist"

2015-05-28T09:33:37+00:00

The .c and .h files are generated by rpcgen, there is no need to include
them in the "make dist" tarball.

BUG: 1198849
Change-Id: Ie6281720e4d485d2b84d98efccfc7053594d1542
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/10897
Tested-by: NetBSD Build System
Reviewed-by: Kaleb KEITHLEY 
Tested-by: Gluster Build System

features/changelog: RPC'fy {libgf}changelog

2015-03-19T01:22:36+00:00

This patch introduces RPC based communication between the changelog
translator and libgfchangelog. It replaces the old pathetic stream
based interaction that existed earlier (due to time constraints :-/).

Changelog, upon initialization starts a RPC server (rpcsvc) allowing
clients to invoke a probe API as a bootup mechanism to request for
event notifications. During probe, clients can choose an event
filter specifying the type(s) of events they are interested in. As
of now there is no way to change the event notification set once
the probe RPC call is made, but that is easier to implement.

The actual event notifications is done on a separate RPC session.
The client (libgfchangelog) itself starts and RPC server which the
changelog translator "connects back" during probe. Notifications
are dispatched by a bunch of threads from the server (translator)
and the client optionally orders them if ordered notifications
are requried. FOPs fill in their respective event details in a
buffer (rot-buffs to be particular) and a bunch of threads
(consumers) swap the buffers out of roatation and dispatch them
via RPC. To avoid writer starvation, then number of dispatcher
threads is one less than the number of buffer list in rot-buffs.x

libgfchangelog becomes purely callback based -- upon event
notification from the server (and re-ordering them if required)
invoke a callback routine specified by consumer(s).

A major part of the patch is also aimed at providing backward
compatibility for geo-replication, which was one of the main
consumer of the stream based API. Also, this patch does not\
"turn on" event notifications for all fops, just a bunch which
is currently in requirement. Another pain point is that the
server does not filter events before dispatching it to the
clients. That load is taken up by the client itself (although
it's done at the library layer rather than making it hard on
the callback implementor). This needs improvement and care
needs to be taken to not load the server up with expensive
filtering mechanisms.

Change-Id: Ibf60a432b68f2dfa60c6f9add2bcfd37a9c41395
BUG: 1170075
Signed-off-by: Venky Shankar 
Reviewed-on: http://review.gluster.org/9708
Reviewed-by: Jeff Darcy 
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Build fix: xdrgen

2014-11-14T06:16:17+00:00

As discovered in https://review.gluster.org/8762, BSD systems fail to
run xdrgen during glusterfs build. This seems to be caused by a
difference between BSD make and GNU make whith implcit targets. The
former seems to use absolute path here, which means we should not
prepend it with the current directory path, otherwise we have the
directory path twice and the files cannot be found my make.

This is a second attempt after I178123bf6f3d9e963ff5b78839d498f530c05a97
which was broken and reverted in I3c8966288f66d0eafa2e94490e3b64a057b4f2c0

BUG: 1157839
Change-Id: I797c536c319a156b71a42c82cbaf80bbf17b7234
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/9046
Tested-by: Gluster Build System 
Reviewed-by: Kaleb KEITHLEY 
Reviewed-by: Vijay Bellur

Revert "Build fix: xdrgen"

2014-11-04T13:41:32+00:00

This reverts commit 12bc39c144aa41a097435f2aab304ddfbbb9b625

This causes all regression tests to fail. Reverting for now, we can include a
correct fix later.

Change-Id: I3c8966288f66d0eafa2e94490e3b64a057b4f2c0
URL: http://supercolony.gluster.org/pipermail/gluster-devel/2014-November/042759.html
BUG: 1157839
Reported-by: Emmanuel Dreyfus 
Signed-off-by: Niels de Vos 
Reviewed-on: http://review.gluster.org/9040
Tested-by: Gluster Build System 
Reviewed-by: Vijay Bellur

Build fix: xdrgen

2014-11-02T16:41:34+00:00

As discovered in https://review.gluster.org/8762, BSD systems fail to
run xdrgen during glusterfs build. This seems to be caused by a
difference between BSD make and GNU make whith implcit targets. The
former seems to use absolute path here, which means we should not
prepend it with the current directory path, otherwise we have the
directory path twice and the files cannot be found my make.

BUG: 1157839
Change-Id: I178123bf6f3d9e963ff5b78839d498f530c05a97
Signed-off-by: Emmanuel Dreyfus 
Reviewed-on: http://review.gluster.org/9016
Reviewed-by: Harshavardhana 
Tested-by: Harshavardhana 
Reviewed-by: Humble Devassy Chirammal