Source build w/memcached and/or fastcgi support fails

Basics

Technical

Logistics

Basics

Technical

Logistics

Description

I'm building Shibboleth SP from source on RHEL 5.9 with support for the memcached storage provider, which is installed in a custom directory. I.e., I'm invoking configure like this:

./configure --enable-apache-22
            --with-apxs22=/opt/apache/bin/apxs2
            ...
            --with-memcached=/home/ct-dist/mylibmemcached

When adding the --with-memcached line the builds fails; otherwise it's fine.
I found two problems:

The memcached header presence check ran by ./configure fails because the necessary -I flag is not on the command line. I've verified this in config.log:
configure:20037: checking for Memcached support configure:20047: result: /home/ct-dist/mylibmemcached configure:20057: checking libmemcached/memcached.h usability configure:20057: g++ -c -Wall -I/home/ct-dist/mycurl/include -g -D_DEBUG -I/home/ct-dist/myopensaml/include -I/home/ct-dist/myxmltooling/include -I/home/ct-dist/myxmlsecurityc/include -I/home/ct-dist/myxerces/include -I/home/ct-dist/mylog4shib/include -pthread -Wall -O0 -g -D_DEBUG -I/home/ct-dist/myboost/include -I/home/ct-dist/myopenssl/include conftest.cpp >&5 conftest.cpp:88:36: error: libmemcached/memcached.h: No such file or directory
The build inside memcache-store fails for the same reason: missing -I flag:
make[2]: Entering directory `/home/ct-dist/shibboleth-sp-2.5.1/memcache-store' if /bin/sh ../libtool --silent --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I. -I.. -I../shibsp -I/home/ct-dist/myopensaml/include -I/home/ct-dist/myxmltooling/include -I/home/ct-dist/myxmlsecurityc/include -I/home/ct-dist/myxerces/include -I/home/ct-dist/mylog4sh ib/include -pthread -Wall -O0 -g -D_DEBUG -I/home/ct-dist/myboost/include -I/home/ct-dist/myopenssl/include -Wall -I/home/ct-dist/mycurl/include -g -D_DEBUG -MT memcache-store.lo -MD -MP -MF ".deps/memcache-store.Tpo" -c -o memcache-store.lo memcache-store.cpp; \ then mv -f ".deps/memcache-store.Tpo" ".deps/memcache-store.Plo"; else rm -f ".deps/memcache-store.Tpo"; exit 1; fi memcache-store.cpp:45:36: error: libmemcached/memcached.h: No such file or directory memcache-store.cpp:102: error: ISO C++ forbids declaration of 'memcached_st' with no type memcache-store.cpp:102: error: expected ';' before '*' token memcache-store.cpp:107: error: 'memcached_return' has not been declared // etc

I'm relatively new to autotools, but it seems to me like this is due to two separate issues with configure.ac and memcache-store/Makefile.am:

configure.ac

// ...
if test "$WANT_MEMCACHED" != "no"; then 
    if test "$WANT_MEMCACHED" != "yes"; then 
        if test x_$WANT_MEMCACHED != x_/usr; then 
            MEMCACHED_INCLUDE="-I$WANT_MEMCACHED/include"
            MEMCACHED_LDFLAGS="-L$WANT_MEMCACHED/lib"
        fi   
    fi   
    AC_CHECK_HEADER([libmemcached/memcached.h],,
        AC_MSG_ERROR([unable to find Memcached header files]))
    AC_CHECK_DECL([memcached_last_error_message],
        [AC_DEFINE([HAVE_MEMCACHED_LAST_ERROR_MESSAGE],[1],[Define to 1 if libmemcached supports error handling function.])],,
        [#include <libmemcached/memcached.h>])
    MEMCACHED_LIBS="-lmemcached"
fi

AC_SUBST(MEMCACHED_INCLUDE)
AC_SUBST(MEMCACHED_LDFLAGS)
AC_SUBST(MEMCACHED_LIBS)

For the first issue, notice that in configure.ac MEMCACHED_INCLUDE is not added to any variable contributing to the final g++ flags, thus leading to the necessary -I flag not being there at the time AC_CHECK_HEADER does its thing. Google tells me that common practice seems to be to temporarily add it to the CPPFLAGS; something like this:

configure.ac'

save_CPPFLAGS="$CPPFLAGS"
    save_LDFLAGS="$LDFLAGS"
    CPPFLAGS="$CPPFLAGS $MEMCACHED_INCLUDE"
    LDFLAGS="$LDFLAGS $MEMCACHED_LDFLAGS"
    AC_CHECK_HEADER([libmemcached/memcached.h],,
        AC_MSG_ERROR([unable to find Memcached header files]))
    AC_CHECK_DECL([memcached_last_error_message],
        [AC_DEFINE([HAVE_MEMCACHED_LAST_ERROR_MESSAGE],[1],[Define to 1 if libmemcached supports error handling function.])],,
        [#include <libmemcached/memcached.h>])
    CPPFLAGS="$save_CPPFLAGS"
    LDFLAGS="$save_LDFLAGS"

This solved the missing header check for me, but I will leave its correctness up to your judgement.

For the second issue, the problem seems to be with memcache-store/Makefile.am:

memcache-store/Makefile.am

AM_CFLAGS = $(MEMCACHED_CFLAGS)
AM_CXXFLAGS = $(MEMCACHED_CFLAGS)
//...
memcache_store_la_LDFLAGS = -module -avoid-version

MEMCACHED_CFLAGS appears nowhere else in the distribution tree, so it seems like that should've been MEMCACHED_INCLUDE. Also, MEMCACHED_LDFLAGS needs to be added to memcache_store_la_LDFLAGS as it contains the -L flag to allow the linker to find the libmemcached .so file:

memcache-store/Makefile.am

AM_CFLAGS = $(MEMCACHED_INCLUDE)
AM_CXXFLAGS = $(MEMCACHED_INCLUDE)
//...
memcache_store_la_LDFLAGS = -module -avoid-version $(MEMCACHED_LDFLAGS)

Indeed, changing this and running autoreconf allowed the entire build to complete successfully again.

I've only looked at memcache-store, but looking at configure.ac it seems like the FastCGI support has the exact same logic as the memcached support in configure.ac, and may therefore exhibit the same issue of the required includes not being there during the header presence check. Additionally, fastcgi/Makefile.am includes the following lines:

// ...
shibauthorizer_CXXFLAGS = -I$(FASTCGI_INCLUDE)
// ...
shibresponder_CXXFLAGS = -I$(FASTCGI_INCLUDE)

Since FASTCGI_INCLUDE already includes the -I prefix though, I suspect that the final command line will end up with something like this (unverified):

-I-I/path/to/fastcgi

As a final issue unrelated to the above, while pouring over configure.ac I found the following bit:

AC_ARG_WITH(saml,
    AS_HELP_STRING([--with-saml=PATH],[where opensaml is installed]),,
    [with_saml=/usr])
if test x_$with_xmltooling != x_/usr; then 
    CPPFLAGS="-I${with_saml}/include $CPPFLAGS"
// ...

I think that should read:

AC_ARG_WITH(...
if test x_$with_saml != x_/usr; then 
    CPPFLAGS="-I${with_saml}/include $CPPFLAGS"

As I said, I'm new to autotools, so feel free to point out if and where I'm wrong, or suggest better ways of doing things.

Environment

Red Hat Enterprise Linux Server release 5.9 (Tikanga)
autoconf (GNU Autoconf) 2.59
automake (GNU automake) 1.9.6

Attachments

15 Jun 2013, 10:53 PM
15 Jun 2013, 10:48 PM
15 Jun 2013, 10:48 PM

Activity

Scott Cantor June 18, 2013 at 2:24 AM

Closing on release.

myhandisadolphin@mailinator.com June 16, 2013 at 10:19 PM
Edited

Figured I might as well build and install FastCGI in a custom directory and try it. Extra configure flag:

./configure --prefix=/home/ct-dist/myshibboleth \
            // ...
            --with-fastcgi=/home/ct-dist/myfastcgi

Same thing as for memcached: ./configure can't find the header because the -I flag is added to CFLAGS and not CPPFLAGS, but once corrected, the build succeeds.

*EDIT: Slightly too late Thanks for your support Scott!

Scott Cantor June 16, 2013 at 10:03 PM

Corrected in r3866. Most of the C library tests are in one spot with gcc active, forgot to make that change. Easier to just use CPPFLAGS, which the ODBC check already did. Corrected same error in fastcgi test.

myhandisadolphin@mailinator.com June 16, 2013 at 9:51 PM
Edited

Hi Scott,

Excellent response time!
I checked out the REL_2 branch and tested it, but unfortunately the libmemcached.h header presence check in ./configure still fails:

./configure --prefix=/home/ct-dist/myshibboleth \
            --disable-doxygen-doc \
            --enable-apache-22 \
            --disable-odbc \
            --enable-debug \
            --with-openssl=/home/ct-dist/myopenssl \
            --with-log4shib=/home/ct-dist/mylog4shib \
            --with-xerces=/home/ct-dist/myxerces \
            --with-xmlsec=/home/ct-dist/myxmlsecurityc \
            --with-xmltooling=/home/ct-dist/myxmltooling \
            --with-saml=/home/ct-dist/myopensaml \
            --with-memcached=/home/ct-dist/mylibmemcached \
            --with-boost=/home/ct-dist/myboost \
            --with-apxs22=/opt/apache/bin/apxs

configure:27563: checking for Memcached support
configure:27573: result: /home/ct-dist/mylibmemcached
configure:27599: checking libmemcached/memcached.h usability
configure:27611: g++ -c -pthread -Wall  -g -D_DEBUG -I/home/ct-dist/myopensaml/include -I/home/ct-dist/myxmltooling/include -I/home/ct-dist/myxmlsecurityc/include -I/home/ct-dist/myxerces/include -I/home/ct-dist/mylog4shib/include -pthread -Wall -O0 -g -D_DEBUG -I/home/ct-dist/myboost/include -I/home/ct-dist/myopenssl/include  conftest.cc >&5
conftest.cc:91:36: error: libmemcached/memcached.h: No such file or directory
configure:27617: $? = 1

In your patch you added MEMCACHED_INCLUDE to CFLAGS; however, since g++ is being invoked, I think it needs to be added to CPPFLAGS instead. Once I change that, ./configure runs OK, and the build succeeds.

I didn't test the FastCGI part as I don't actually have it available to build against.

Scott Cantor June 16, 2013 at 6:53 PM

http://svn.shibboleth.net/view/cpp-sp?rev=3865&view=rev

Thank you for catching all that, literally last minute before tomorrow night's release.

I didn't do exactly what you suggested in every case, but there's no right or wrong, there are just a lot of inconsistent idioms in the code, same as every non-trivial configure script that's been around 10+ years.

If you have a chance to test before tomorrow night using the branch or by testing the patch rev above, it would be great.

Resize issue view side panel

Fixed

Details

Assignee

Scott Cantor

Reporter

myhandisadolphin@mailinator.com

Original estimate

Created June 15, 2013 at 10:21 PM

Updated June 18, 2013 at 2:24 AM

Resolved June 16, 2013 at 6:53 PM