Custom Distro for Linode

[tc@tinycore:#/]

Oh! Makes sense. Learning something new about TC right now. I appreciate your patience. I might be getting over my head, being a novice Linux user, wanting to attempt a custom kernel, but this thread has definitely made sense of things. I think I can compile the kernel and modules now, especially since I know the difference between bzImage (which I had a feeling was the kernel itself) and tinycore.gz/microcore.gz (which I didn't have a clue concerning). Heck, even thinking about it, extlinux even lists the kernel image as, well, bzImage. I suppose I'm an idiot for not figuring this out sooner.

[majbthrd]

I was also curious to see if TCL (specifically microcore) could be run as a paravirtualized Xen guest VM.

I have it working somewhat... the remaining problem I have is that the console output stops at:

Loading Tiny Core Extensions...Done
login[213]: root login on 'tty1'

TCL appears to still be running, and the console starts outputting text again when the VM is shutdown.

I've tried boot options like "text" and whatnot, but to no avail.

So, I assume that all that remains is to figure out how to coax TCL to use whatever virtual tty that Xen expects.

I'm documenting the process I've used in the hopes that it helps others, and perhaps someone much more familiar than I could suggest what remains to be done.

I'm going to divide this into several posts so it isn't one huge post.

[majbthrd]

Rather than try to patch the standard code, I took the easy way out, and used the patched code and config file available here:

http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/3.x/release/src/kernel/

# extract linux kernel source code

tar xjf linux-2.6.33.3-patched.tbz2

# go to created subdirectory

cd linux-2.6.33.3

# prep

make mrproper

# copy in default .config

cp ../config-2.6.33.3-tinycore .config

make oldconfig
make menuconfig

# alter configuration with menuconfig in this order:
# Processor type and features --> Processor family --> Pentium-Classic
# Processor type and features --> High Memory Support --> 64GB
# Processor type and features --> PAE Support
# Processor type and features --> Paravirtualized quest support --> Xen guest support
#
# Note: Pentium-Classic or higher is needed for XEN support
# this means a change from the existing TCL modules (which are compiled for 486)

# build kernel

make bzImage

# build modules

make modules

# copy modules (in this case, to ~/newmodules/, but you can pick your own directory of choice)

mkdir ~/newmodules/
make INSTALL_MOD_PATH=~/newmodules modules_install firmware_install

[majbthrd]

# create raw image file (64 * (512 * 63 * 16 = 516096) = ~32MBytes)
# adjust the size to your tastes

dd if=/dev/zero of=image.bin count=64 bs=516096c

# create partition (-C64 means 64 cylinders, the size we decided upon in the previous step)

fdisk -cu -C64 -S63 -H16 image.bin

n (new partition)
p (primary partition)
1 (partition #1)
63 as start sector
default as last sector
a (toggle bootable)
1 (partition #1)
p (confirm it shows as bootable and that the start sector is 63)
w (write changes)

# mount image for formatting (32256 = 63 * 512)

sudo losetup -o32256 /dev/loop0 image.bin

# format

sudo mkfs.ext3 -b1024 /dev/loop0

# mount filesystem

sudo mount /dev/loop0 /mnt

# create directories

sudo mkdir -p /mnt/boot
sudo mkdir -p /mnt/tce

OR

sudo mkdir -p /mnt/boot
sudo mkdir -p /mnt/tce

# copy files

sudo cp ~/microcore.gz /mnt/boot/
sudo cp ~/linux-2.6.33.3/arch/x86/boot/bzImage /mnt/boot/

# unmount

sudo umount /mnt
sudo losetup -d /dev/loop0

# create XVA
# this requires python and xva.py; the latter is available here:
# http://www.xen.org/files/xva/xva.py

python xva.py -n my_test_image --is-pv --disk image.bin --filename=output.xva

[majbthrd]

Copy the .xva file that you've created over to the XEN server.  Then:

xe vm-import filename=output.xva

IMPORTANT: note the UUID that it spits back out
if for some reason, you miss that UUID, you can figure it out with the command:

xe vm-list

note that we didn't bother putting a /boot/grub/menu.lst (or a syslinux.cfg)
This is because XEN uses pygrub

In theory, pygrub reads /boot/grub/menu.lst.

I originally tried putting a /boot/grub/menu.lst file in the partition, but I could NOT get pygrub in XEN to properly read it.

The workaround seems to set some additional XEN parameters.

All parameters for a particular VM (specified by the UUID which you should have saved earlier) can be listed with:

xe vm-param-list uuid=yourUUID

We need to change two of them:

xe vm-param-set uuid=yourUUID PV-bootloader-args="--kernel /boot/bzImage --ramdisk /boot/microcore.gz"

xe vm-param-set uuid=yourUUID PV-args="quiet console=hvc0 text initrd=/boot/microcore.gz"

Note that you can change the second to suit (and I'm not sure that what I have shown is right).

I am seeing a difference in behavior (for example, "quiet" versus no "quiet"), so I know it is being passed to the kernel.

[majbthrd]

# Note: Pentium-Classic or higher is needed for XEN support
# this means a change from the existing TCL modules (which are compiled for 486)

Note that I left out replacing the modules in the microcore.gz/tinycore.gz image in my previous posts.

This is well documented in the wiki: http://wiki.tinycorelinux.com/Remastering

In my testing, I tried both replacing the modules and not, and neither changed the behavior of the VM (apart from the warnings when the kernel was compiled for Pentium and the modules were for 486).

Also note that all the modules take an obscene amount of disk space, which goes against the principle of TCL.  So, some sort of intelligent subset of all the modules should be copied over.

[maro]

Regarding your "obscene" disk space issue: Have a look at this script. AFAIK it is used by the Core team to "slim down" the output of the kernel build process.

[majbthrd]

Thanks, maro, for the pointer on paring down the size of the modules.

Answering my own question, the solution to the halt of console activity after:

login[213]: root login on 'tty1'

is covered in a previous forum thread:

http://forum.tinycorelinux.net/index.php?topic=5809.0

For running TCL as a Xen guest, I commented out the tty1 line in /etc/inittab and replaced it with:

hvc0::respawn:/sbin/rungetty hvc0

So, yes, it is perfectly viable to customize TCL to run as a Xen guest (aka domU).

[slamet]

Not being knowledgeable enough, this looks a bit out of my league at present.  However, linode is the main reason I am very interested in micro core.  If this can be done successfully, then we may see uptake on micro / tiny's adoption by more users.

Suggestion: Depending on the final size of the build, can it be considered either to turn this into the base micro core or have another core (we can call it server / VPS core maybe).  This is up to the group obviously - just a suggestion.

We are not deviating from the "core" and toolkit concept.  Just to make the core suitable for running in VPS set up which is the way to go for many internet servers.  And there is a lot of demand out there to have something that runs frugally, the way MC does.  Once linode works, then someone else can try AWS and slicehost and then MC will rule the VPS market.

I understand about adding security and what not, but that is step 2.

[curaga]

Depends on the VPS We run fine in KVM ones and probably OpenVZ (that one depends on the host kernel). Don't know about VMWare ones, but I don't think they're that popular for VPS uses.

[slamet]

Interesting.  A quick search shows there are options.  KVM seems to cost more than Open VZ.

Can you (or anyone) please share which vps hosts has been used / proven to run MC?  Many thanks.