In the IdP configuration, credentials are configured in two parts, a certificate and the corresponding private key. If someone points one of these at the wrong file, or in updating their credentials replaces only one of these files, the SP doesn't notice and signs with a private key which then doesn't allow messages to be validated against the public key provided with the certificate. This is very hard to debug.

The SP could verify that the public key in the certificate and in the key file were the same, and throw an error if not. This would make the error obvious in the SP logs without needing the co-operation of an IdP to debug the issue.

(equivalent issue for the IdP is: https://bugs.internet2.edu/jira/browse/SIDP-230)