I had the same dependency on X when I built open-vm-tools, so I built two versions: open-vm-tools and open-vm-tools-withx. There was a 2x difference in size and number of dependencies when including X, but I also wanted a small extension which would include the kernel modules and simple stuff. Basically, an extension that works for microcore and one that will work for tinycore.