You must be using the system in a remarkable fashion.
I'll let you decide if it's a remarkable fashion: I migrated my development system from a virtual machine to a chroot jail including the desktop running in an Xnest server. Works like a charm, as fast as can be, memory limitation is not as tight, which is nice, since I'm compiling on tmpfs, and building Qt was always a bit tight there. The link is relative, so it works from outside as well as inside the chroot jail.
BTW: since this is a 64 bit "host", I run the start of the system using "linux32", so all autodetection of the architecture work as expected, since the reported kernel architecture is not the same as the userland tools. This might be also useful for the 64bit initrd.
Greetings,
SvOlli
[edit: fixed a lot of ugly typos]