OpenVPN and remote-cert-tls server

This required a bit of digging into OpenVPN's and OpenSSL's code to figure out.

The problem

This error:

Thu Sep 11 00:12:05 2014 Validating certificate key usage
Thu Sep 11 00:12:05 2014 ++ Certificate has key usage  00f8, expects 00a0
Thu Sep 11 00:12:05 2014 ++ Certificate has key usage  00f8, expects 0088

The condition

Using openvpn with the following option:

remote-cert-tls server

The solution

(for me) to add this to openvpn's config file:

remote-cert-ku f8

The explanation

Background

remote-cert-tls attempts to solve one problem: Lets say you run a CA and you distribute the certificates to 2 people including me and you. Then you setup a VPN server for us to use and you generate another certificate for the VPN server.

As always, the problem that certificates attempt to solve is "how do you know you're connecting to the remote end you assume you are. In normal SSL pages you trust a CA to verify that the CN of the certificate matches the owner of the domain. If you want to achieve the same thing with openvpn then you need to verify the CN of the remote end against either the hostname or a predefined string. If not then you need to use the "remote-cert-tls server" option.

If you don't use any of the above methods then I can fire up an openvpn server using the certificate you provided me with and since both my certificate and the actual VPN server's certificate are signed by the same CA you would be verifying both and be equally willing to connect to both, thus allowing me to spy on you.

To solve this kind of problems X509 has some properties for certificates that designate them for certain purposes. E.g. one of them is to run a VPN endpoint (TLS Web Client Authentication)

To be precise there are 2+1 such designations in X509:

X509v3 Key Usage: 
    Digital Signature, Non Repudiation, Key Encipherment, Data Encipherment, Key Agreement
 
X509v3 Extended Key Usage: 
    TLS Web Server Authentication, TLS Web Client Authentication, IPSec End System, IPSec Tunnel, Time Stamping
 
Netscape Cert Type: 
    SSL Client, SSL Server, S/MIME, Object Signing

"Netscape Cert Type" is kind of old. "Key Usage" is the main one and "Extended Key Usage" is the final addition. Ignoring NS Cert Type, "Key Usage" is a bitmap and thus has limited space for expansion. "Extended Key Usage" on the other hand is a list of object identifiers which allows for unlimited expansion.

The certificate

The certificate I was using for the server-side of the OpenVPN had the above attributes. Ignoring NS Cert Type once more, the other two correspond to the following data:

  494:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Key Usage
  499:d=5  hl=2 l=   4 prim: OCTET STRING      [HEX DUMP]:030203F8
 
  433:d=5  hl=2 l=   3 prim: OBJECT            :X509v3 Extended Key Usage
  438:d=5  hl=2 l=  52 prim: OCTET STRING      [HEX DUMP]:303206082B0601050507030106082B0601050507030206082B0601050507030506082B0601050507030606082B06010505070308

Starting with "Key Usage", the actual value is "F8". The meaning of each bit can be found in OpenSSL's code:

#define KU_DIGITAL_SIGNATURE    0x0080
#define KU_NON_REPUDIATION      0x0040
#define KU_KEY_ENCIPHERMENT     0x0020
#define KU_DATA_ENCIPHERMENT    0x0010
#define KU_KEY_AGREEMENT        0x0008
#define KU_KEY_CERT_SIGN        0x0004
#define KU_CRL_SIGN             0x0002
#define KU_ENCIPHER_ONLY        0x0001
#define KU_DECIPHER_ONLY        0x8000

On the other hand, the "Extended Key Usage" part contains the following Object IDs:

06082B 06010505070301 -> serverAuth (TLS Web Server Authentication)
06082B 06010505070302 -> clientAuth (TLS Web Client Authentication)
06082B 06010505070305 -> ipsecEndSystem (IPSec End System)
06082B 06010505070306 -> ipsecTunnel (IPSec Tunnel)
06082B 06010505070308 -> timeStamping (Time Stamping)

The "bug"

The first thing to notice is that the failure is for "Key Usage" and not for "Extended Key Usage" (took me some time to figure out).

After that, a bit of digging into the code confirms that OpenVPN attempts to verify a bitmap with equality. I.e. it gets the certificates' value and compares it against a predefined list of allowed values, which according to OpenVPN's documentation defaults to "a0 88" (which means one of them). However the actual certificates bitmap value is 0xf8 as mentioned above. And thus the comparison fails with the error:

Thu Sep 11 00:12:05 2014 Validating certificate key usage
Thu Sep 11 00:12:05 2014 ++ Certificate has key usage  00f8, expects 00a0
Thu Sep 11 00:12:05 2014 ++ Certificate has key usage  00f8, expects 0088

The reason I'm calling this a bug is because it's not sensible to use equality to compare against a bitmap. Instead one can use AND, in which case there would be:

( <certificate's value> & <desired value> ) == <desired value>
or:
( 0xf8 & 0xa0) == 0xa0  -> True

In order for the validation to succeed with the defaults the certificate should have one of the following designations:

0xa0: Digital Signature, Key Encipherment
0x88: Digital Signature, Key Agreement

The solution

So there, since the comparison is done with equality you can do one of the following:

• Use the above Key Usage on the certificate (inconvenient)


• Don't use "remote-cert-tls server" (bad)


• Use "remote-cert-ku XX" where XX is the value of your certificate which can be seen in OpenVPN's messages (the last octet). In my case it's f8.

 

pyOpenSSL and invalid certificates

I was trying to import some X509v3 certificates that were created with pyOpenSSL to a MikroTik router (RouterOS 6.1) but they were always being imported with an invalid validity period (not before 1970 and not after 1970).

Eventually I found out that this is because pyOpenSSL stores the validity field in an invalid format. Here's the story:

cert1.pem is the pyOpenSSL certificate and cert2.pem is a certificate created with openssl. Both have mostly the same information. Decoding the certificates with openssl shows that cert1.pem actually has an older notAfter date so it's not an issue of overflow.

[code language="shell" gutter="false"]
$ openssl x509 -noout -startdate -enddate < cert1.pem
notBefore=Jul 20 18:35:58 2013 GMT
notAfter=Jan  1 00:00:00 2032 GMT

$ openssl x509 -noout -startdate -enddate < cert2.pem
notBefore=Aug 31 21:44:54 2012 GMT
notAfter=Aug 26 21:44:54 2032 GMT
[/code]

I examined the certificates in python by decoding their DER structures and looking for the validity field (copy-paste the following in a python shell).

[code language="python"]
import Crypto.Util.asn1 as asn1
import OpenSSL.crypto as c

fn1="cert1.pem"
fn2="cert2.pem"

st1=open(fn1, 'r').read()
st2=open(fn2, 'r').read()

cert1=c.load_certificate(c.FILETYPE_PEM, st1)
cert2=c.load_certificate(c.FILETYPE_PEM, st2)

dump1=c.dump_certificate(c.FILETYPE_ASN1, cert1)
dump2=c.dump_certificate(c.FILETYPE_ASN1, cert2)

der1=asn1.DerSequence()
der2=asn1.DerSequence()

der1.decode(dump1)
der2.decode(dump2)

dcert1=der1[0]
dcert2=der2[0]

t1=asn1.DerSequence()
t2=asn1.DerSequence()

t1.decode(dcert1)
t2.decode(dcert2)

tt1=asn1.DerSequence()
tt2=asn1.DerSequence()

tt1.decode(t1[4])
tt2.decode(t2[4])
[/code]

at this point tt1 and tt2 are sequences of the validity field (notBefore, notafter) for the two certificates . Here's what they contain:

[code language="python" gutter="false"]
>>> tt1[0]
'\x18\x0f20130720183558Z'
>>> tt1[1]
'\x18\x0f20320101000000Z'

>>> tt2[0]
'\x17\r120831214454Z'
>>> tt2[1]
'\x17\r320826214454Z'
[/code]

SoaB! They differ!

Reading the X509 spec [1], section 4.1.2.5 indicates that there are two possible formats for the validity period: both notBefore and notAfter may be encoded as UTCTime or  GeneralizedTime.

• UTCTime is defined as YYMMDDHHMMSSZ


• GeneralizedTime is defined as YYYYMMDDHHMMSSZ

So pyOpenSSL uses GeneralizedTime while openssl uses UTCTime. So both are valid.

However the RFC also says:

CAs conforming to this profile MUST always encode certificate
validity dates through the year 2049 as UTCTime; certificate validity
dates in 2050 or later MUST be encoded as GeneralizedTime.

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARG!

So it seems that pyOpenSSL uses GeneralizedTime unconditionally which is not RFC compliant and thus rejected by RouterOS.

A quick look at pyOpenSSL's code unfortunately proves that...

[1]   http://www.ietf.org/rfc/rfc3280.txt

Verify that a private key matches a certificate with PyOpenSSL

Verify that a private key matches a certificate using PyOpenSSL and PyCrypto:

[code language="python"]
import OpenSSL.crypto
from Crypto.Util import asn1

c=OpenSSL.crypto

# The certificate - an X509 object
cert=...

# The private key - a PKey object
priv=...

pub=cert.get_pubkey()

# Only works for RSA (I think)
if pub.type()!=c.TYPE_RSA or priv.type()!=c.TYPE_RSA:
raise Exception('Can only handle RSA keys')

# This seems to work with public as well
pub_asn1=c.dump_privatekey(c.FILETYPE_ASN1, pub)
priv_asn1=c.dump_privatekey(c.FILETYPE_ASN1, priv)

# Decode DER
pub_der=asn1.DerSequence()
pub_der.decode(pub_asn1)
priv_der=asn1.DerSequence()
priv_der.decode(priv_asn1)

# Get the modulus
pub_modulus=pub_der[1]
priv_modulus=priv_der[1]

if pub_modulus==priv_modulus:
print('Match')
else:
print('Oops')
[/code]

The idea is to get the modulus from the two DER structures and compare them. They should be the same.

Note: You can use the above under the MIT license. If it doesn’t fit your needs let me know. My intention is to make this usable by anyone for any kind of use with no obligation.

Verifying an SSL certificate with python

This one took me a considerable amount of time and had to figure some parts from scratch.

Unfortunately there doesn't seem to exist an easy (out-of-the-box) way for checking whether a certificate is signed by another certificate in python.

After days of searching and despair, here is a solution without using M2Crypto:

[code language="python"]
import OpenSSL
from Crypto.Util import asn1

c=OpenSSL.crypto

# This is the certificate to validate
# an OpenSSL.crypto.X509 object
cert=...

# This is the CA certificate to use for validation
# again an OpenSSL.crypto.X509 object
cacert=...

# Get the signing algorithm
algo=cert.get_signature_algorithm()

# Get the ASN1 format of the certificate
cert_asn1=c.dump_certificate(c.FILETYPE_ASN1, cert)

# Decode the certificate
der=asn1.DerSequence()
der.decode(cert_asn1)

# The certificate has three parts:
# - certificate
# - signature algorithm
# - signature
# http://usefulfor.com/nothing/2009/06/10/x509-certificate-basics/
der_cert=der[0]
der_algo=der[1]
der_sig=der[2]

# The signature is a BIT STRING (Type 3)
# Decode that as well
der_sig_in=asn1.DerObject()
der_sig_in.decode(der_sig)

# Get the payload
sig0=der_sig_in.payload

# Do the following to see a validation error for tests
# der_cert=der_cert[:20]+'1'+der_cert[21:]

# First byte is the number of unused bits. This should be 0
# http://msdn.microsoft.com/en-us/library/windows/desktop/bb540792(v=vs.85).aspx
if sig0[0]!='\x00':
raise Exception('Number of unused bits is strange')

# Now get the signature itself
sig=sig0[1:]

# And verify the certificate
try:
c.verify(cacert, sig, der_cert, algo)
print "Certificate looks good"
except OpenSSL.crypto.Error, e:
print "Sorry. Nope."
[/code]

Note: You can use the above under the MIT license. If it doesn’t fit your needs let me know. My intention is to make this usable by anyone for any kind of use with no obligation.

X509v3 Authority Key Identifier pains (authorityKeyIdentifier)

"X509v3 Authority Key Identifier" or "authorityKeyIdentifier" is an X509v3 extension that's added to X509 certificates and identifies the CA that signed the Certificate. I suppose that this speeds up the certificate validation process by eliminating multiple checks.

Short version

Edit openssl.cnf and make sure that authorityKeyIdentifier does not include "issuer"

Long version

There's an issue when using the default OpenSSL configuration or when basing a config on that: the default OpenSSL configuration has the following:

authorityKeyIdentifier=keyid,issuer

In the section that lists options for user certificates (i.e. not the CA section). The above results in new certificates using the extension and include two identifiers for the signing CA:

• The Key ID of the CA's cert (because if "keyid")


• The subject and the serial number of the CA's cert (because of issuer)

For example:

X509v3 Authority Key Identifier: 
    keyid:7E:E5:82:FF:FF:FF:15:96:9B:40:FF:C9:5E:51:FF:69:67:4D:BF:FF
    DirName:/C=UK/O=V13/OU=V13/CN=V13 Certificate Authority
    serial:8E:FF:A2:1B:74:DD:54:FF

And this is where the pain and the suffering happens: If you ever decide that you want to re-create the CA's certificate using the same private key then you won't be able to do so because all certificates that are already signed dictate  the subject and the serial number of the old certificate as the CA certificate identifier. Thus your new CA certificate will not be able to verify the existing certificates.

Thus the only way to replace your certificate would be:

• To start from scratch recreating all certificates, or


• to create another CA certificate with the same subject and serial number (not tested)

Recreating a certificate with the same details (like serial number) will make it impossible to have both certificates available and will most probably cause a mess.

The best approach is to completely remove the "issuer" from authorityKeyIdentifier from the configuration file. Then only the Key ID will be used to identify the CA which should be more than enough.

So use the following and live a happy life:

authorityKeyIdentifier=keyid