Debian i386 to amd64 conversion

After doing this two times, I finally conclude that it *is* possible to convert an i386 debian installation to an amd64 one. At the end of the procedure most of the system will work. Here is how:

NOTE: Don't do anything that is mentioned here unless you fully understand what you are doing. You will most probably need to customize this procedure a bit!

You'll need enough free space for the procedure to work.

While you still have a working system, print this document and download the latest debian netinst image. This can be used as a rescue disk if something goes wrong. It is trivial to restore your system using a rescue CD since we will not delete anything at all.


Step 1: Backup

I cannot stress this enough. Backup all your data, especially databases and anything binary that needs an executable file to be accessed. Most probably files at your home directory don't risk at all but better be safe than sorry.

It is absolutely essential to backup your PostgreSQL database if you have one. The i386 and amd64 disk formats are not compatible (bad postgre... bad bad) and you'll need to restore the database. This most probably comes to:

# # Backup:
# su - postgres
$ pg_dumpall > mydump
# # Restore
$ cat mydump | psql template0


I've done this procedure twice without backing up anything :-) except from the PostgreSQL database which is required. The reason I didn't backed-up anything is that I was sure I could rollback to my old system using a rescue CD. If you don't know how just backup.

Step 2: Install an amd64 kernel
You'll need to boot your i386 with an amd64 kernel. Do:

# apt-get install libc6-amd64 linux-image-2.6-amd64
# # Most probably you would like to have X working while you run this procedure so install your favorite closed-source modules too:
# apt-get install nvidia-kernel-2.6-amd64

and reboot

Step 3: Install a base amd64 system.

# mkdir /amd64
# dpkg --get-selections '*' > /amd64/sel1
# debootstrap --arch=amd64 lenny /amd64 http://myproxy/

(NOTE: The above code was corrected to include "--arch=amd64" which I forgot to include. Thanks to Slawek Wernikowski)

Step 4: Install an identical amd64 system
Since this is a time consuming procedure and many things can go bad, I suggest you run this under "screen". This way, if the X server crashes you will not loose what you've done.

# chroot /amd64 /bin/bash
# vi /etc/apt/sources.list
# # Add all sources.list entries that exist in your main system but not the CDs
# vi /etc/apt/apt.conf
# # Add the line: APT::Default-Release "your-release";
# # if needed
# apt-get install locales
# mount /proc
# mount /sys
# mount -t devpts none /dev/pts
# export TERM=xterm
# dpkg --set-selections < /sel1

Now run the following lines until everything is installed. Most probably parts of the installation may fail and you'll need to rerun those commands. It should be OK. At the end there can be some packages that will not be configured. Just ignore them for now.

# apt-get -o 'Dpkg::Options={"--force-confdef";"--force-confnew"};' dselect-upgrade
# dpkg --configure --force-confdef --force-confnew -a

I repeat: You'll have to repeat the above commands until everything is installed. If you have trouble just fix it yourself. Don't forget that you're working in a chrooted amd64 system.

Step 5: Change the system
This is the part that I don't have written so there might be some this missing. Key points are here and you should be able to figure what you need to do if something goes wrong. It is really a very simple procedure:

# init 1
# mkdir /i386

# # Move /usr
# mv /usr /i386
# mv /amd64/usr /

# # Backup etc
# cp -pR /etc /i386

# # Move /lib32 and /lib64 away if they exist
# mv /lib32 /i386
# mv /lib64 /i386

# # Create links
# ln -s /lib /lib64
# ln -s /emul/ia32-linux/lib /lib32

## Move other dirs
# mv /amd64/emul /
# mv /sbin /i386
# mv /amd64/sbin /

Now is the tricky part where you move /lib and /bin:

# # /bin is easy
# mv /bin /i386
# /i386/bin/mv /amd64/bin /
# # Don't forget that you can only run commands from /i386 from now on until /lib is replaced.
# # So, to do ls just write: /i386/bin/ls

# # Time for /lib
# mv /lib /i386
# # At this point you cannot execute anything at all but don't panic. Just run everything using the runtime linker. This is what happens whenever you run a command.
# export LD_LIBRARY_PATH=/i386/lib
# /i386/lib/ld-2.7.so /i386/bin/mv /amd64/lib /
# # TATA! Welcome to your amd64 system!
# ls


Step 6: Fix /var
This step is highly dependent on what you have installed. Most probably you can remove old /var and keep the new one. If you'd like to keep the old one there are some dirs that you need to replace:

# cd /var
# mv cache cache.i386
# mv /amd64/var/cache .
# # I suggest that you replace the whole /var/lib tree:
# mv lib lib.i386
# mv /amd64/var/lib .


Whatever you do you should know that you should replace /var/lib/dpkg and /var/cache/apt with the new one.

Step 7: Make system bootable
Finally you should ensure that your system is bootable. A nice approach is to clean your /boot from whatever kernel is installed (don't forget that with your new system, no package will own existing kernels) and force-reinstall your current kernel:

# cd /boot
# mkdir i386
# mv System* initrd* vmlinuz* config* i386
# apt-get install --reinstal linux-image-2.6.26-amd64
# # Replace linux-image-2.6.26-amd64 with your current kernel


This is it. Unless I've forgotten something you should just reboot to your know system. After rebooting you will have to fix the unconfigured packages and possible rerun dselect-upgrade:

# dpkg --configure -a
# apt-get dselect-upgrade


Finally, restore your PostgreSQL database and you're done.
MySQL doesn't need dump and restore but wou'll have to copy/move old files from /var/lib.i386/myslq to /var/lib/mysql.

VMWare Server 2 RC2 fails to boot VMs

For a couple of days now, VMWare Server 2 RC2 was refusing to start virtual machines. It kept waiting at 95%.

It turns out that VMWare cannot boot virtual machines (or upgrade them) when kernel modules kvm_intel, kvm etc are loaded.

Here is the error message from hostd.log:

Question info: The virtualization capability of your processor is already in use. Disable any other running hypervisors before running VMware Server.

Of course the obvious solution is to rmmod the kvm related modules:

# lsmod |grep kvm
# rmmod kvm_intel
# rmmod kvm_amd
# rmmod kvm

The modules are most probably auto-loaded by /etc/init.d/kvm (but there may be other reasons too).

Run secureley wireshark as a user

Sometimes there is the need to allow normal users to run wireshark and capture packets from the network. Running wireshark with sudo is a security hole since anyone can overwrite any file.

A secure one-liner that solves this problem is:

# (sudo dumpcap -w -) | wireshark -k -i -

Assuming that sudo is configured to allow the user to run "dumpcap -w -" as root.

This should be 100% secure (except from the traffic monitoring issue) and will work well in (for example) labs.

fglrx (Catalyst 8.8) + kernel 2.6.26 + PAT

It seems that PAT support that is included in linux kernel 2.6.26 has some problems. If you are using a recent ATI Catalst (fglrx) driver (at the time of this writting, the lattest was 8.8) with 2.6.26 kernel with PAT support you may have problems.

Problems may include freezes every couple of seconds when playing videos or full-screen 3D apps/games.

To solve this problem you will have to disable PAT support in kernel. Fortunately there is a kernel boot parameter that does with without requiring recompilation. Just add "nopat" to kernel boot parameters and you should be OK. Debian users should not worry since PAT is disabled in the default kernel.

The above is half the truth. When disabling PAT you should make sure that MTRR is being set-up correctly. Start the X server and look at Xorg.0.log for the string "Linear":

$ grep Linear /var/log/Xorg.0.log
(--) fglrx(0): Linear framebuffer (phys) at 0xc0000000

Then look at lspci -v outpout:

01:00.0 VGA compatible controller: ATI Technologies Inc Mobilitiy Radeon HD 3600 Series
Subsystem: ASUSTeK Computer Inc. Device 01e4
Flags: bus master, fast devsel, latency 0, IRQ 16
Memory at c0000000 (64-bit, prefetchable) [size=256M]
Memory at ff6f0000 (64-bit, non-prefetchable) [size=64K]
[...]

("size=256M" is what we need). Don't worry if you have more than 256MB of memory in your graphics card but it only shows 256MB. This is expected behaviour.

Then look at /proc/mtrr for a line like this one:

reg04: base=0xc0000000 (3072MB), size= 256MB: write-combining, count=1

The base address is from the Xorg.0.log file and the size is from lspci output (0x10000000 is 256MB (=256*1024*1024 bytes)). For most cases this should be taken care of by the X server

If you don't see it then you'll have to setup MTRR by yourself. There are many documents on the web but at the end you'll be running something like this:

echo "base=0xc0000000 size=0x10000000 type=write-combining" > /proc/mtrr


Trouble uploading files to wordpress

I had problems uploading files to wordpress hosted at the local machine. Conditions are:

• Firefox 3.0


• Flash 9.0


• Local squid proxy 2.7.STABLE3


• Firefox setup for using local proxy except from accessing local machine. This makes no difference at all.


• Environment variable http_proxy set to http://127.0.0.1:8080/

The result of trying to upload images to wordpress was "HTTP Error". A sample wireshark capture of the failed request looks like this:

Request:

POST http://localhost/wordpress/wp-admin/async-upload.php HTTP/1.1 
Host: localhost 
Pragma: no-cache 
Accept: */* 
Proxy-Connection: Keep-Alive 
User-Agent: Shockwave Flash
Connection: Keep-Alive
Cache-Control: no-cache 
Content-Length: 40861 
Expect: 100-continue 
Content-Type: multipart/form-data; boundary=----------------------------5ce16f6dfa7a

Response:

HTTP/1.0 417 Expectation failed 
Server: squid/2.7.STABLE3 
Date: Sun, 27 Jul 2008 09:31:50 GMT 
Content-Type: text/html 
Content-Length: 1507 
Expires: Sun, 27 Jul 2008 09:31:50 GMT 
X-Squid-Error: ERR_INVALID_REQ 0 
X-Cache: MISS from XXXXX.XXXX.XXXX 
X-Cache-Lookup: NONE from XXXXX.XXXX.XXXX:3128 
Via: 1.0 XXXXX.XXXX.XXXX:3128 (squid/2.7.STABLE3) 
Connection: close 

[...]
A page saying:
ERROR: The requested URL could not be retrieved
[...]

It is obvious that flash plugin is used to perform the upload and it is using the proxy. It does a request with an "Expect:" header and for an (unknown to me) reason the expectation fails because of the squid proxy.

Disabling proxy in firefox, restarting it and bypassing cache doesn't help. The problem lies in that the flash plugin uses the http_proxy environment variable to determine the proxy.

Solution:
Unsetting http_proxy in a terminal and starting firefox from there solves the problem.

Routes with greater prefix and Proxy ARP ~= IP Mobility

Inside an Autonomous System, it is possible to move a machine inside a network, keeping its IP address even though it goes to a network segment that doesn't serve the corresponding Network.

Something like this:

RouterA -------- Network Segment
                        |
         Host A (10.1.0.2/24, GW: 10.1.0.1)

It is possible to move Host A to another network segment, lets say to interface FastEthernet of RouterB. Of course this would require address and possibly other configuration changes to Host A. Changin the IP address of a server is not always a good idea (tm).

Lets say that we move Host A to interface FastEthernet of Router B. Supposing that a routing protocol is setup and works in the Autonomous System, Host A may keep its IP address by configuring RouterB (cisco commands):

RouterB(config)# ip route 10.1.0.2 255.255.255.255
   FastEthernet 0/1

RouterB(config)# interface FastEthernet 0/1
RouterB(config-if)# ip proxy-arp      # This is the default

RouterB(config)# router eigrp 1
RouterB(config-router)# redistribute static

RouterA(config)# interface FastEthernet 0/1
RouterA(config-if)# ip proxy-arp

This will work because:

• Whenever Host A tries to reach a host in its subnet (ARP request), Router B will respond with its mac address. This is what Proxy ARP does.


• All routers within the A.S. will learn the 10.1.0.2/32 route. Even Router A will prefer to use this one instead of the directly connected 10.1.0.0/24 since it has a longest preffix (routing table lookups are longest matching preffix lookups)


• Since Router A will also learn this route and since it has Proxy ARP enabled, it will respond to ARP requests for 10.1.0.1 with its address
  

Maemo internet connection from command-line

Using OS2008/Diablo in maemo (tested in N800 but should work elsewhere) it is possible to initiate (and terminate) an Internet connection from the command line. This is useful for automating tasks.

The whole thing is easily done using some dbus functionality that is available from running services.

To initiate a new connection:

dbus-send --system --print-reply --type=method_call \
    --dest=com.nokia.icd \
    /com/nokia/icd com.nokia.icd.connect \
    string:"Connection Name" uint32:0

Where "Connection Name" is the connection name as it is listed in the UI when selecting where to connect to. It can be a wireless connection, a GPRS connection via bluetooth, a PAN connection or whetever will be supported in the future.

To terminate the connection:

dbus-send --system \
    --dest=com.nokia.icd \
    /com/nokia/icd_ui com.nokia.icd_ui.disconnect \
    boolean:true