Fixed
Details
Details
Assignee
Scott Cantor
Scott CantorReporter
jlawson@weaveonline.com
jlawson@weaveonline.comOriginal estimate
1d 4h
Components
Fix versions
Affects versions
Created October 28, 2011 at 4:52 PM
Updated August 7, 2012 at 1:11 AM
Resolved August 7, 2012 at 1:10 AM
Our IIS web service keeps crashing with the following error:
A process serving application pool 'app.Production' suffered a fatal communication error with the Windows Process Activation Service. The process id was '2576'. The data field contains the error number.
A debug of the crash shows an access violation that appears to be caused by Shibboleth as referenced from a support request we opened with Microsoft (below):
//From Microsoft PSS
I have finished analyzing the dumps you sent Friday and the crash is being caused by the Shibboleth ISAPI filter. In the crashing call stack below you can see that isapi_shib is calling into our ServerSupportFunction. When I look at the instruction that is causing the AV in our code, I can see that the Shibboleth component is passing in a bad pointer for the ul1 parameter, see http://msdn.microsoft.com/en-us/library/aa503395.aspx.
0:030> kL
Child-SP RetAddr Call Site
00000000`0b23b970 00000001`8000174e filter!W3_FILTER_CONTEXT::ServerSupportFunction+0x174
00000000`0b23bb80 00000001`8000c566 isapi_shib!TerminateFilter+0x45e
00000000`0b23bbc0 00000000`745e6f60 isapi_shib!GetFilterVersion+0x2896
00000000`0b23bc00 00000000`745b3b3c msvcr90!_CallSettingFrame+0x20
00000000`0b23bc30 00000000`77990c21 msvcr90!__CxxCallCatchBlock+0xfc
00000000`0b23bd00 00000001`80007ff2 ntdll!RcFrameConsolidation+0x3
00000000`0b23ea20 000007fe`f67e17e4 isapi_shib!HttpFilterProc+0x2d2
00000000`0b23eec0 000007fe`f67e1e01 filter!W3_FILTER_CONTEXT::NotifyFilters+0x178
00000000`0b23f0e0 000007fe`f8f6a185 filter!GlobalDoWork+0x351
00000000`0b23f310 000007fe`f8f6ab24 iiscore!W3_CONTEXT::SetupStateMachine+0x685
00000000`0b23f820 000007fe`fb4310d2 iiscore!W3_MAIN_CONTEXT::OnNewRequest+0x1b0
00000000`0b23f850 000007fe`fb43109c w3dt!UL_NATIVE_REQUEST::DoWork+0x126
00000000`0b23f8b0 000007fe`f8b01fba w3dt!OverlappedCompletionRoutine+0x1c
00000000`0b23f8e0 000007fe`f8b02024 w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x7a
00000000`0b23f930 000007fe`f8b020a1 w3tp!THREAD_POOL_DATA::ThreadPoolThread+0x34
00000000`0b23f960 00000000`7783652d w3tp!THREAD_MANAGER::ThreadManagerThread+0x61
00000000`0b23f990 00000000`7796c521 kernel32!BaseThreadInitThunk+0xd
00000000`0b23f9c0 00000000`00000000 ntdll!RtlUserThreadStart+0x1d
As you can see from below the AV is caused by an attempted read from 8000e6f0.
0:030> .exr -1
ExceptionAddress: 000007fef67ebc14 (filter!W3_FILTER_CONTEXT::ServerSupportFunction+0x0000000000000174)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: 000000008000e6f0
Attempt to read from address 000000008000e6f0
The memory at the referenced address is free and is Page_Protect
0:030> !address 000000008000e6f0
Usage: Free
Base Address: 00000000`7fff0000
End Address: 00000000`ffb00000
Region Size: 00000000`7fb10000
Type: 00000000
State: 00010000 MEM_FREE
Protect: 00000001 PAGE_NOACCESS
0:030> lmvm isapi_shib
start end module name
00000001`80000000 00000001`80020000 isapi_shib (export symbols) isapi_shib.dll
Loaded symbol image file: isapi_shib.dll
Image path: D:\opt\shibboleth-sp\lib\shibboleth\isapi_shib.dll
Image name: isapi_shib.dll
Timestamp: Sun Jul 03 17:00:27 2011 (4E10D86B)
CheckSum: 00026390
ImageSize: 00020000
File version: 2.4.3.0
Product version: 2.4.3.0
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: UCAID
ProductName: Shibboleth 2.4.3
InternalName: isapi_shib
OriginalFilename: isapi_shib.dll
ProductVersion: 2, 4, 3, 0
FileVersion: 2, 4, 3, 0
PrivateBuild: 2, 4, 3, 0
SpecialBuild: 2, 4, 3, 0
FileDescription: Shibboleth ISAPI Filter / Extension
LegalCopyright: Copyright © 2011 UCAID
LegalTrademarks: Copyright © 2011 UCAID
Comments: Copyright © 2011 UCAID