Uploaded image for project: 'Shibboleth SP - C++'
  1. Shibboleth SP - C++
  2. SSPCPP-904

Unhandled Exception in iis7_shib.dll

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, 3.1.0
    • Fix Version/s: 3.1.0.2
    • Labels:
      None
    • Environment:

      Windows Server 2016 on VMWare

      Our website is running mostly Classic ASP, I have set useHeaders="true" useVariables="false" 

    • Operating System:
      Windows
    • CPU Type:
      x86_64
    • Web Server:
      IIS 9 (Windows 2016)

      Description

      A few months ago we started using the Shibboleth Service Provider v3.1.0.1 on IIS on Windows Server 2016 (with all recent Windows updates)

      It can be running fine for days but then sometimes w3wp.exe crashes (a few times on a day) with no apparent cause.

      Windows Error Reporting shows:

      Faulting application name: w3wp.exe, version: 10.0.14393.0, time stamp: 0x57899b8a 
       Faulting module name: KERNELBASE.dll, version: 10.0.14393.3659, time stamp: 0x5e9140ed 
       Exception code: 0xe06d7363 
       Fault offset: 0x0000000000034f38 
       Faulting process id: 0xc44c 
       Faulting application start time: 0x01d65efa426943f8 
       Faulting application path: c:\windows\system32\inetsrv\w3wp.exe 
       Faulting module path: C:\Windows\System32\KERNELBASE.dll

       

       

      The 0xe06d7363 translates to EXCEPTION_UNCAUGHT_CXX_EXCEPTION

      Next step is to catch a minidump. The Call Stack is shown below
      Iis7_shib.dll is very high in the call-stack with IIS7Request::throwError at the top.
      It is called by IIS7Request::getRequestBody because a call to m_request->ReadEntityBody fails.

      First of all: I am not sure Shibboleth is *causing* the crash, maybe it is just collateral damage. But it may help in finding the root cause

      My questions:

      • A message goes into the IOException. Is there any way to retrieve that message (some logging perhaps? I could not find it in shibd.log or any other log)
        --> In the minidump I can see the error is ERROR_NETNAME_DELETED
      • ReadEntityBody sound like it is receiving data. Is it fair to generate an (unhandled) exception for ‘just’ a network error?
      • If it is only collateral damage: is there a way to catch the exception?
        I do see the <InProcess> element supports a (disadvised) catchAll but if I read the sourcecode correctly it may not apply to the IOException
        -->Currently testing

       (callstack and shibd.log follows)

       KERNELBASE.dll!RaiseException-() Unknown
      [External Code]
      iis7_shib.dll!IIS7Request::throwError(const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & operation, HRESULT hr) Line 452 C++
      iis7_shib.dll!IIS7Request::getRequestBody() Line 332 C++
      shibsp-lite3_1.dll!std::_Tree<std::_Tmap_traits<std::basic_string<char,std::char_traits<char>,std::allocator<char> >,boost::shared_ptr<shibsp::DOMPropertySet>,std::less<std::basic_string<char,std::char_traits<char>,std::allocator<char> > >,std::allocator<std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const ,boost::shared_ptr<shibsp::DOMPropertySet> > >,0> >::_Insert_hint<std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const ,boost::shared_ptr<shibsp::DOMPropertySet> > &,std::_Tree_node<std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const ,boost::shared_ptr<shibsp::DOMPropertySet> >,void *> *>(std::_Tree_const_iterator<std::_Tree_val<std::_Tree_simple_types<std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const ,boost::shared_ptr<shibsp::DOMPropertySet> > > > > _Where, std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const ,boost::shared_ptr<shibsp::DOMPropertySet> > & _Val, std::_Tree_node<std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const ,boost::shared_ptr<shibsp::DOMPropertySet> >,void *> * _Newnode) Line 1703 C++
      shibsp-lite3_1.dll!shibsp::AbstractSPRequest::getHandlerURL(const char * resource) Line 247 C++
      xmltooling-lite3_1.dll!xmltooling::TemplateEngine::TemplateParameters::getParameter(const char * name) Line 61 C++
      shibsp-lite3_1.dll!shibsp::ServerThread::job() Line 597 C++
      xmltooling-lite3_1.dll!xmltooling::TemplateEngine::process(bool visible, const std::basic_string<char,std::char_traits<char>,std::allocator<char> > & buf, const char * & lastpos, std::basic_ostream<char,std::char_traits<char> > & os, const xmltooling::TemplateEngine::TemplateParameters & parameters, const std::pair<std::basic_string<char,std::char_traits<char>,std::allocator<char> > const ,std::basic_string<char,std::char_traits<char>,std::allocator<char> > > & loopentry, const xmltooling::XMLToolingException * e) Line 157 C++
      xmltooling-lite3_1.dll!xmltooling::TemplateEngine::run(std::basic_istream<char,std::char_traits<char> > & is, std::basic_ostream<char,std::char_traits<char> > & os, const xmltooling::TemplateEngine::TemplateParameters & parameters, const xmltooling::XMLToolingException * e) Line 286 C++
      shibsp-lite3_1.dll!shibsp::ServiceProvider::doHandler(shibsp::SPRequest & request) Line 684 C++
      [External Code]
      shibsp-lite3_1.dll!shibsp::DOMPropertySet::load(const xercesc_3_2::DOMElement * e, log4shib::Category * log, xercesc_3_2::DOMNodeFilter * filter, const shibsp::DOMPropertySet::Remapper * remapper, const xmltooling::QName * unsetter) Line 151 C++
      iis7_shib.dll!ShibHttpModule::DoHandler(IHttpContext * pHttpContext, IHttpEventProvider * pProvider) Line 62 C++
      [External Code]

       

      Also: this is what shibd.log shows during the crash:

      
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [45]: error reading size of input message
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [45]: failed socket call (unknown), result (0): No error
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [45]: I/O failure processing request on socket (636)
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [47]: error reading size of input message
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [47]: failed socket call (unknown), result (0): No error
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [47]: I/O failure processing request on socket (440)
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [46]: error reading size of input message
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [46]: failed socket call (unknown), result (0): No error
      2020-08-19 17:13:17 ERROR Shibboleth.Listener [46]: I/O failure processing request on socket (648)

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              rdw@iay.org.uk Rod Widdowson
              Reporter:
              57be911d30be4c376c7fb4d4693c20b45d246f18b6540ad4f1f45c20f7a83c2d@https://login.idp.eduid.se/idp.xml 57be911d30be4c376c7fb4d4693c20b45d246f18b6540ad4f1f45c20f7a83c2d@https://login.idp.eduid.se/idp.xml
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - Not Specified
                  Not Specified
                  Logged:
                  Time Spent - 4 hours
                  4h