BasicX509Credential::extract incorrectly frees x509 serial number causes crash in debug builds
Basics
Technical
Logistics
Basics
Technical
Logistics
Description
When my application and all 3rd party tools are built in debug mode I get a crash in xmltooling\security\impl\BasicX509Crednetial.cpp in the method BasicX509Credential::extract.
When built release mode the crash does not occur.
The problem seems to stem from the wrong function being used to free a pointer returned from the OpenSSL library. The code around line 340 reads:
ASN1_INTEGER* serialASN = X509_get_serialNumber(cert); BIGNUM* serialBN = ASN1_INTEGER_to_BN(serialASN, NULL); if (serialBN) { char* serial = BN_bn2dec(serialBN); if (serial) { m_serial = serial; free(serial); } BN_free(serialBN); }
But serial was returned from one of the BN functions from the OpenSSL library so should be freed using BN_free not using free. If I change this and rebuild xmltooling then the crash goes away. I changed the code to:
ASN1_INTEGER* serialASN = X509_get_serialNumber(cert); BIGNUM* serialBN = ASN1_INTEGER_to_BN(serialASN, NULL); if (serialBN) { char* serial = BN_bn2dec(serialBN); if (serial) { m_serial = serial; BN_free(serial); } BN_free(serialBN); }
Environment
Using OpenSSL version 0.9.8l. Application and all 3rd party tools built debug.
If it was failing for you, there's a pretty good chance you're mixing runtime library models. My code does not support that. You need to use the DLL C/C++ runtime, which is the defalt, and ensure that all dependencies are using it.
Scott Cantor February 16, 2010 at 1:30 PM
Any chance you meant OPENSSL_free? That appears to be the intended method, based on reading the documentation (i.e. the source code).
Scott Cantor February 16, 2010 at 1:27 PM
Well, there's two problems, one fairly significant.
I'm using the same compiler and always run with debug builds, and it's not crashing on me. I just traced it to confirm.
Secondly, and the bigger point, BN_free in my build doesn't take a char*, so your patch won't compile.
It isn't typical of them to change the API between letter releases, so it's hard to believe the signature or the intended functions to use would change between k and l, but I am using k at the moment. Perhaps that's the problem, but I'd have to conditionally code it if that's true.
Fixed
Pinned fields
Click on the next to a field label to start pinning.
When my application and all 3rd party tools are built in debug mode I get a crash in xmltooling\security\impl\BasicX509Crednetial.cpp in the method BasicX509Credential::extract.
When built release mode the crash does not occur.
The problem seems to stem from the wrong function being used to free a pointer returned from the OpenSSL library.
The code around line 340 reads:
ASN1_INTEGER* serialASN = X509_get_serialNumber(cert);
BIGNUM* serialBN = ASN1_INTEGER_to_BN(serialASN, NULL);
if (serialBN) {
char* serial = BN_bn2dec(serialBN);
if (serial) {
m_serial = serial;
free(serial);
}
BN_free(serialBN);
}
But serial was returned from one of the BN functions from the OpenSSL library so should be freed using BN_free not using free.
If I change this and rebuild xmltooling then the crash goes away. I changed the code to:
ASN1_INTEGER* serialASN = X509_get_serialNumber(cert);
BIGNUM* serialBN = ASN1_INTEGER_to_BN(serialASN, NULL);
if (serialBN) {
char* serial = BN_bn2dec(serialBN);
if (serial) {
m_serial = serial;
BN_free(serial);
}
BN_free(serialBN);
}