Installing package build dependencies from a .dsc file (Debian)

There are cases where one needs to install build-dependencies of a .dsc file in Debian.

Apparently this is not as trivial as:

[code gutter="false"]
# apt-get build-dep package
[/code]

The easiest way I've found so far is to use mk-build-deps (from the devscripts package):

[code gutter="false"]
# mk-build-deps -i vadm_1.0.4ci+r16.dsc -t apt-get --no-install-recommends -y
dh_testdir
dh_testroot
dh_prep
dh_testdir
dh_testroot
dh_install
dh_installdocs
dh_installchangelogs
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb
dpkg-deb: building package `vadm-build-deps' in `../vadm-build-deps_1.0.4ci+r16_all.deb'.

The package has been created.
Attention, the package has been created in the current directory,
not in ".." as indicated by the message above!
(Reading database ... 41052 files and directories currently installed.)
Preparing to replace vadm-build-deps 1.0.4ci+r16 (using vadm-build-deps_1.0.4ci+r16_all.deb) ...
Unpacking replacement vadm-build-deps ...

Reading package lists...
Building dependency tree...
Reading state information...

0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up vadm-build-deps (1.0.4ci+r16) ...
[/code]

This godly script:

• Creates a psudo package that depends on the build-depends of the .dsc file


• Does a dpkg -i on the generated deb file (which may fail because of missing build depends)


• Does apt-get -f install

Extra points for using -r which will remove the generated package after it's done.

pyOpenSSL and invalid certificates

I was trying to import some X509v3 certificates that were created with pyOpenSSL to a MikroTik router (RouterOS 6.1) but they were always being imported with an invalid validity period (not before 1970 and not after 1970).

Eventually I found out that this is because pyOpenSSL stores the validity field in an invalid format. Here's the story:

cert1.pem is the pyOpenSSL certificate and cert2.pem is a certificate created with openssl. Both have mostly the same information. Decoding the certificates with openssl shows that cert1.pem actually has an older notAfter date so it's not an issue of overflow.

[code language="shell" gutter="false"]
$ openssl x509 -noout -startdate -enddate < cert1.pem
notBefore=Jul 20 18:35:58 2013 GMT
notAfter=Jan  1 00:00:00 2032 GMT

$ openssl x509 -noout -startdate -enddate < cert2.pem
notBefore=Aug 31 21:44:54 2012 GMT
notAfter=Aug 26 21:44:54 2032 GMT
[/code]

I examined the certificates in python by decoding their DER structures and looking for the validity field (copy-paste the following in a python shell).

[code language="python"]
import Crypto.Util.asn1 as asn1
import OpenSSL.crypto as c

fn1="cert1.pem"
fn2="cert2.pem"

st1=open(fn1, 'r').read()
st2=open(fn2, 'r').read()

cert1=c.load_certificate(c.FILETYPE_PEM, st1)
cert2=c.load_certificate(c.FILETYPE_PEM, st2)

dump1=c.dump_certificate(c.FILETYPE_ASN1, cert1)
dump2=c.dump_certificate(c.FILETYPE_ASN1, cert2)

der1=asn1.DerSequence()
der2=asn1.DerSequence()

der1.decode(dump1)
der2.decode(dump2)

dcert1=der1[0]
dcert2=der2[0]

t1=asn1.DerSequence()
t2=asn1.DerSequence()

t1.decode(dcert1)
t2.decode(dcert2)

tt1=asn1.DerSequence()
tt2=asn1.DerSequence()

tt1.decode(t1[4])
tt2.decode(t2[4])
[/code]

at this point tt1 and tt2 are sequences of the validity field (notBefore, notafter) for the two certificates . Here's what they contain:

[code language="python" gutter="false"]
>>> tt1[0]
'\x18\x0f20130720183558Z'
>>> tt1[1]
'\x18\x0f20320101000000Z'

>>> tt2[0]
'\x17\r120831214454Z'
>>> tt2[1]
'\x17\r320826214454Z'
[/code]

SoaB! They differ!

Reading the X509 spec [1], section 4.1.2.5 indicates that there are two possible formats for the validity period: both notBefore and notAfter may be encoded as UTCTime or  GeneralizedTime.

• UTCTime is defined as YYMMDDHHMMSSZ


• GeneralizedTime is defined as YYYYMMDDHHMMSSZ

So pyOpenSSL uses GeneralizedTime while openssl uses UTCTime. So both are valid.

However the RFC also says:

CAs conforming to this profile MUST always encode certificate
validity dates through the year 2049 as UTCTime; certificate validity
dates in 2050 or later MUST be encoded as GeneralizedTime.

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAARG!

So it seems that pyOpenSSL uses GeneralizedTime unconditionally which is not RFC compliant and thus rejected by RouterOS.

A quick look at pyOpenSSL's code unfortunately proves that...

[1]   http://www.ietf.org/rfc/rfc3280.txt

Verify that a private key matches a certificate with PyOpenSSL

Verify that a private key matches a certificate using PyOpenSSL and PyCrypto:

[code language="python"]
import OpenSSL.crypto
from Crypto.Util import asn1

c=OpenSSL.crypto

# The certificate - an X509 object
cert=...

# The private key - a PKey object
priv=...

pub=cert.get_pubkey()

# Only works for RSA (I think)
if pub.type()!=c.TYPE_RSA or priv.type()!=c.TYPE_RSA:
raise Exception('Can only handle RSA keys')

# This seems to work with public as well
pub_asn1=c.dump_privatekey(c.FILETYPE_ASN1, pub)
priv_asn1=c.dump_privatekey(c.FILETYPE_ASN1, priv)

# Decode DER
pub_der=asn1.DerSequence()
pub_der.decode(pub_asn1)
priv_der=asn1.DerSequence()
priv_der.decode(priv_asn1)

# Get the modulus
pub_modulus=pub_der[1]
priv_modulus=priv_der[1]

if pub_modulus==priv_modulus:
print('Match')
else:
print('Oops')
[/code]

The idea is to get the modulus from the two DER structures and compare them. They should be the same.

Note: You can use the above under the MIT license. If it doesn’t fit your needs let me know. My intention is to make this usable by anyone for any kind of use with no obligation.

Verifying an SSL certificate with python

This one took me a considerable amount of time and had to figure some parts from scratch.

Unfortunately there doesn't seem to exist an easy (out-of-the-box) way for checking whether a certificate is signed by another certificate in python.

After days of searching and despair, here is a solution without using M2Crypto:

[code language="python"]
import OpenSSL
from Crypto.Util import asn1

c=OpenSSL.crypto

# This is the certificate to validate
# an OpenSSL.crypto.X509 object
cert=...

# This is the CA certificate to use for validation
# again an OpenSSL.crypto.X509 object
cacert=...

# Get the signing algorithm
algo=cert.get_signature_algorithm()

# Get the ASN1 format of the certificate
cert_asn1=c.dump_certificate(c.FILETYPE_ASN1, cert)

# Decode the certificate
der=asn1.DerSequence()
der.decode(cert_asn1)

# The certificate has three parts:
# - certificate
# - signature algorithm
# - signature
# http://usefulfor.com/nothing/2009/06/10/x509-certificate-basics/
der_cert=der[0]
der_algo=der[1]
der_sig=der[2]

# The signature is a BIT STRING (Type 3)
# Decode that as well
der_sig_in=asn1.DerObject()
der_sig_in.decode(der_sig)

# Get the payload
sig0=der_sig_in.payload

# Do the following to see a validation error for tests
# der_cert=der_cert[:20]+'1'+der_cert[21:]

# First byte is the number of unused bits. This should be 0
# http://msdn.microsoft.com/en-us/library/windows/desktop/bb540792(v=vs.85).aspx
if sig0[0]!='\x00':
raise Exception('Number of unused bits is strange')

# Now get the signature itself
sig=sig0[1:]

# And verify the certificate
try:
c.verify(cacert, sig, der_cert, algo)
print "Certificate looks good"
except OpenSSL.crypto.Error, e:
print "Sorry. Nope."
[/code]

Note: You can use the above under the MIT license. If it doesn’t fit your needs let me know. My intention is to make this usable by anyone for any kind of use with no obligation.

X509v3 Authority Key Identifier pains (authorityKeyIdentifier)

"X509v3 Authority Key Identifier" or "authorityKeyIdentifier" is an X509v3 extension that's added to X509 certificates and identifies the CA that signed the Certificate. I suppose that this speeds up the certificate validation process by eliminating multiple checks.

Short version

Edit openssl.cnf and make sure that authorityKeyIdentifier does not include "issuer"

Long version

There's an issue when using the default OpenSSL configuration or when basing a config on that: the default OpenSSL configuration has the following:

authorityKeyIdentifier=keyid,issuer

In the section that lists options for user certificates (i.e. not the CA section). The above results in new certificates using the extension and include two identifiers for the signing CA:

• The Key ID of the CA's cert (because if "keyid")


• The subject and the serial number of the CA's cert (because of issuer)

For example:

X509v3 Authority Key Identifier: 
    keyid:7E:E5:82:FF:FF:FF:15:96:9B:40:FF:C9:5E:51:FF:69:67:4D:BF:FF
    DirName:/C=UK/O=V13/OU=V13/CN=V13 Certificate Authority
    serial:8E:FF:A2:1B:74:DD:54:FF

And this is where the pain and the suffering happens: If you ever decide that you want to re-create the CA's certificate using the same private key then you won't be able to do so because all certificates that are already signed dictate  the subject and the serial number of the old certificate as the CA certificate identifier. Thus your new CA certificate will not be able to verify the existing certificates.

Thus the only way to replace your certificate would be:

• To start from scratch recreating all certificates, or


• to create another CA certificate with the same subject and serial number (not tested)

Recreating a certificate with the same details (like serial number) will make it impossible to have both certificates available and will most probably cause a mess.

The best approach is to completely remove the "issuer" from authorityKeyIdentifier from the configuration file. Then only the Key ID will be used to identify the CA which should be more than enough.

So use the following and live a happy life:

authorityKeyIdentifier=keyid

Raspberry Pi under QEMU

What: Run raspberry pi system under QEMU
Why: I wanted to have builder environment for Raspberry PI. Emulating it it much faster that running something on it
Disclaimer: I am not a qemu/raspberry-pi expert - Some things may be wrong here

Introduction

Running qemu-arm is slightly different than running qemu for x86. There is no BIOS there and it needs to boot with a kernel image directly. The tricky part is that this image is not the standard Raspberry Pi (from now on mentioned as "raspi"). You can either download one from online or create your own.

I ended up creating my own qemu arm image

Disk image

Get the official raspberry pi disk image from here. Place it somewhere and uncompress it.

By default the image does not have much free space. A trick to change that is to extend it with something like the following:

# dd if=/dev/zero of=2013-02-09-wheezy-raspbian.img \
 bs=1024000 conv=notrunc \
 seek=4000 count=1

This will write 1MB after the first 4000 MBytes on the image. This will cause the image to become approximately 4GB but it will leave a hole in the file if the underlying filesystem supports it.

Kernel Image (optional)

If you want to compile your own kernel image for qemu then follow first the instructions  here. Follow just the instructions on cloning git and patching the kernel.

Instead of configuring it yourself get this configuration (will speed your life) which has some additional stuff like built-in sound, vfat support, IPv6, nice fonts, etc. Put this under the cloned kernel directory with name .config and then run:

# make ARCH=arm oldconfig
# nice make ARCH=arm -j 4   # Adjust for your number of cores
# cp arch/arm/boot/zImage /path/to/raspi/directory

You will need the arm compilation toolchain. If you're running Debian then follow the instructions in http://www.emdebian.org/

The package you are looking for is gcc-4.4-arm-linux-gnueabi

Note: Raspberry Pi supports hardfloat while the above compile is not using hardfloat. There should be no problems with that since the kernel is not using floating point math itself and it won't make a difference. The attached kernel configuration however has hardfloat support and so does qemu.

Booting

I am using vde networking with qemu so your setup may be slightly different. In any case, this is a nice setup to get you started:

# qemu-system-arm \
        -cpu arm1176 \
        -kernel kernel-v13-qemu-130323 \
        -name 'builder-raspi' \
        -hda 2013-02-09-wheezy-raspbian.img \
        -m 256 \
        -M versatilepb \
        -no-reboot \
        -serial stdio \
        -net vde,sock=/var/run/vde2/tap0.ctl \
        -net nic \
        -append "root=/dev/sda2 panic=0 ro

Use the proper path for the disk image and the kernel.

Now, here's the catch: The latest (as of 2013-03-23) raspberry pi image will have some issues booting with the above setup.

Fix latest filesystem image

Change the qemu -append parameter to: "root=/dev/sda2 panic=0 ro single" and boot the image.

This should leave you with a nice prompt. Then:

# mount / -o remount,rw

Next, edit fstab and change mmcblk0p1 and mmcblk0p2 to sda1 and sda2 respectively

Then create the file /etc/udev/rules.d/90-qemu.rules with the following contents:

KERNEL=="sda", SYMLINK+="mmcblk0"
KERNEL=="sda?", SYMLINK+="mmcblk0p%n",

This should ensure that /dev/mmcblk0p1/2 exist and will make other stuff work.

Also edit /etc/ld.so.preload (if it exists) and remove or comment everything in it.

Now shutdown the qemu image, restore the -append parameter and run it again. If raspi-config does not start by itself, run it yourself after logging in as user "pi". You can now select "expand_rootfs" and do another reboot to get more space on the root partition.