TransportOption is not being passed correctly to curl
Basics
Technical
Logistics
Basics
Technical
Logistics
Description
When using the TransportOption as described in the documentation, the option 10004 (OBJECTSTRING, PROXYNAME) do not appear to being set or passed to curl. There is a current workaround in that I am now using a cron job to use wget and download the Metadata once per hour and this appears to be satisfactory for the time being. Please see the email below that has been sent to the shibboleth-users list.
The environment for shibboleth/shibd is as follows :
2009-10-12 11:39:23 INFO XMLTooling.XMLToolingConfig : xmltooling 1.2.2 library shutdown complete2009-10-12 11:39:29 INFO Shibboleth.Config : Library versions: Xerces-C 3.0.1, X ML-Security-C 1.5.1, XMLTooling-C 1.2.1, OpenSAML-C 2.2.1, Shibboleth 1.2.1 2009-10-12 11:39:23 INFO OpenSAML.SAMLConfig : opensaml 2.2.1 library shutdown c omplete 2009-10-12 11:39:23 INFO Shibboleth.Config : shibboleth 2.2.1 library shutdown c omplete 2009-10-12 11:39:29 INFO Shibboleth.Config : Library versions: Xerces-C 3.0.1, X ML-Security-C 1.5.1, XMLTooling-C 1.2.1, OpenSAML-C 2.2.1, Shibboleth 1.2.1
This shows the following :
Calling of the appropriate routine in libcurl by shibboleth (shibd?), the XML from the configuration file showing the TransportOption settings, a verbose output from libcurl showing the verbose option "41" had indeed been set showing that Integer options are being set correctly. Please notice proxy name of "nil" showing this had not been set correctly
2009-10-09 11:32:33 DEBUG XMLTooling.libcurl.InputStream : libcurl trying to fetch http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml 2009-10-09 11:32:33 INFO XMLTooling.StorageService : cleanup thread started...running every 900 seconds 2009-10-09 11:33:03 ERROR XMLTooling.libcurl.InputStream : error while fetching http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml: (28) Connection time-out after 30035 ms 2009-10-09 11:33:03 ERROR OpenSAML.MetadataProvider.XML : error while loading configuration from (http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml): internal error in NetAccessor 2009-10-09 11:33:03 WARN OpenSAML.MetadataProvider.XML : using local backup of remote resource
The timeout is due to shibd being unable to access the Metadataprovider via a proxy. To workaround this I've followed the documentation that I could find and have put the following options into the MetadataProvider section
I have also set the verbose option on for curl (option 41) and I know that this is working because I receive the following on the console after restarting the shibd service Stopping shibd: [ OK ] Starting shibd: * About to connect() to proxy (nil) port 80
Expire at 1255049178 / 924947 (29998ms)
Trying 131.181.188.197... [FAILED] [root@authsvc shibboleth]# * Connection time-out after 30051 ms
Expire cleared
Closing connection #0
leaving me somewhat confused as to why the option "10004" and "59" have not been set (I presume as shown by proxy (nil) in the output above. The hostname has been resolved to the correct IP but the proxy does not appear to have been set.
I've searched through the various archives and googled to death but can't seem to find anybody having this particular issue. Any assistance or pointers would be most appreciated.
As a workaround, I have a cron job downloading the metadata from the exact same url via wget with no issues thus allowing shibd to fall back to a backup file. Curl can also be used on the command line successfully with both these commands linking correctly through the proxy. I believe however this is, for some reason, not being set when using the TransportOption as shown above. This is also identical to that shown in the NativeSPTransportOption examples. I know that both curl (command line version) and wget will look at the http_proxy variable but this does not appear to be checked by the shibd daemon.
Actually, I misread the code...can you indicate what the curl version is? I think the code is flawed for < 7.17.0, which would be typical for Linux at this point.
Scott Cantor October 12, 2009 at 8:45 AM
OBJECTPOINT option types are being handled in a way that won't preserve string option values, so currently only LONG and offset option types will work.
Fixed
Pinned fields
Click on the next to a field label to start pinning.
When using the TransportOption as described in the documentation, the option 10004 (OBJECTSTRING, PROXYNAME) do not appear to being set or passed to curl. There is a current workaround in that I am now using a cron job to use wget and download the Metadata once per hour and this appears to be satisfactory for the time being. Please see the email below that has been sent to the shibboleth-users list.
The environment for shibboleth/shibd is as follows :
2009-10-12 11:39:23 INFO XMLTooling.XMLToolingConfig : xmltooling 1.2.2 library
shutdown complete2009-10-12 11:39:29 INFO Shibboleth.Config : Library versions: Xerces-C 3.0.1, X
ML-Security-C 1.5.1, XMLTooling-C 1.2.1, OpenSAML-C 2.2.1, Shibboleth 1.2.1
2009-10-12 11:39:23 INFO OpenSAML.SAMLConfig : opensaml 2.2.1 library shutdown c
omplete
2009-10-12 11:39:23 INFO Shibboleth.Config : shibboleth 2.2.1 library shutdown c
omplete
2009-10-12 11:39:29 INFO Shibboleth.Config : Library versions: Xerces-C 3.0.1, X
ML-Security-C 1.5.1, XMLTooling-C 1.2.1, OpenSAML-C 2.2.1, Shibboleth 1.2.1
This shows the following :
Calling of the appropriate routine in libcurl by shibboleth (shibd?), the XML from the configuration file showing the TransportOption settings, a verbose output from libcurl showing the verbose option "41" had indeed been set showing that Integer options are being set correctly. Please notice proxy name of "nil" showing this had not been set correctly
2009-10-09 11:32:33 DEBUG XMLTooling.libcurl.InputStream : libcurl
trying to fetch http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml
2009-10-09 11:32:33 INFO XMLTooling.StorageService : cleanup thread
started...running every 900 seconds
2009-10-09 11:33:03 ERROR XMLTooling.libcurl.InputStream : error while
fetching http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml:
(28) Connection time-out after 30035 ms
2009-10-09 11:33:03 ERROR OpenSAML.MetadataProvider.XML : error while
loading configuration from
(http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml): internal
error in NetAccessor
2009-10-09 11:33:03 WARN OpenSAML.MetadataProvider.XML : using local
backup of remote resource
The timeout is due to shibd being unable to access the Metadataprovider
via a proxy. To workaround this I've followed the documentation that I
could find and have put the following options into the MetadataProvider
section
<MetadataProvider type="XML"
url="http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml"
backingFilePath="/etc/shibboleth/AAF-metadata.xml"
reloadInterval="7200">
<SignatureMetadataFilter
certificate="AAF-metadata.crt.pem"/>
<TransportOption provider="CURL"
option="41">1</TransportOption>
<TransportOption provider="CURL"
option="10004">www-proxy.ersa.edu.au</TransportOption>
<TransportOption provider="CURL"
option="59">8080</TransportOption>
I have also set the verbose option on for curl (option 41) and I know
that this is working because I receive the following on the console
after restarting the shibd service
Stopping shibd: [ OK ]
Starting shibd: * About to connect() to proxy (nil) port 80
Expire at 1255049178 / 924947 (29998ms)
Trying 131.181.188.197... [FAILED]
[root@authsvc shibboleth]# * Connection time-out after 30051 ms
Expire cleared
Closing connection #0
leaving me somewhat confused as to why the option "10004" and "59" have
not been set (I presume as shown by proxy (nil) in the output above. The
hostname has been resolved to the correct IP but the proxy does not
appear to have been set.
I've searched through the various archives and googled to death but
can't seem to find anybody having this particular issue. Any assistance
or pointers would be most appreciated.
As a workaround, I have a cron job downloading the metadata from the
exact same url via wget with no issues thus allowing shibd to fall back
to a backup file. Curl can also be used on the command line successfully
with both these commands linking correctly through the proxy. I believe
however this is, for some reason, not being set when using the
TransportOption as shown above. This is also identical to that shown in
the NativeSPTransportOption examples. I know that both curl (command
line version) and wget will look at the http_proxy variable but this
does not appear to be checked by the shibd daemon.
[root@authsvc shibboleth]# wget
http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml
--11:54:55-- http://manager.aaf.edu.au/metadata/metadata.aaf.signed.xml
Resolving www-proxy.sapac.edu.au... 129.127.96.82
Connecting to www-proxy.sapac.edu.au|129.127.96.82|:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 545724 (533K) [text/xml]
Saving to: `metadata.aaf.signed.xml.3'
100%[=======================================>] 545,724 1.19M/s in
0.4s
11:54:55 (1.19 MB/s) - `metadata.aaf.signed.xml.3' saved [545724/545724]
If any more information is needed or if I have put this in the wrong category, please feel free to contact me via email
Thanks and regards
David Logan