Booting on EC2 HVM - some questions and requests for core team

[as]

Hi there

I'm trying to get TinyCore running on Amazon EC2.  Amazon EC2 uses Xen. Some background is that on Xen/EC2 there are two types of virtual machine - PV (paravirtual) and HVM (hardware assisted virtualisation).  On EC2, PV is effectively phased out in favor of HVM, although PV still remains and is supported. I am attempting to make TinyCore work on HVM which is the way forward.

So far I have achieved two things - first is that TinyCore boots fine on my local Xen server in HVM mode and the network comes up and everything is just dandy. The exact same TinyCore boot image also starts fine on EC2 HVM, EXCEPT the network does not come up.

The reason for this I suspect is explained as follows:

1: The vanilla flavor of Xen server provides a virtual network device which is the RTL8139 adapter.  TinyCore picks this up and uses it when running on a local Xen server.
2: Amazon EC2 does not enable the RTL network device.  Instead, any operating system running on Amazon EC2 HVM must use the PVHVM drivers which provide virtual network and disk drivers.

So, TinyCore boots on EC2 HVM but no network is configured. Obviously not alot of work gets done past that point.

What I believe is needed from here is for TinyCore to use the Xen PVHVM drivers that provide virtual storage and virtual network. FYI all flavors of Linux that run on EC2 HVM have the PVHVM drivers configured and running - if you want network on EC2 then you must have PVHVM drivers.

Quoting from this link: "Today upstream Linux kernels automatically and out-of-the-box include these PVHVM drivers."
http://wiki.xen.org/wiki/Xen_Linux_PV_on_HVM_drivers#Xen_Project_PVHVM_drivers_in_upstream_Linux_kernel

I believe I see the PVHVM drivers in the TinyCore kernel source - (see list of files at the end of this message).

So a few questions for the TinyCore gurus:

1: Do you see value in building Xen PVHVM drivers as part of the main distribution of TinyCore? Seems to me that supporting EC2 out of the box would be a good thing (certainly from my perspective :-) but I think others would too).
2: Can someone please suggest the most effective way for me to get drivers to test? Is it to compile the kernel somehow with those drivers enabled?
3: If there is a TinyCore compile/driver expert who can do that quickly and is willing to help that would be awesome because I'm not a deep expert here and having already spent alot of time getting to this point I'm anxious about heading down the path of building my own kernel and spending more days possibly without success - so asking straight up if someone more experienced with TinyCore is willing to help me to shortcut that process. If not then that's fine I'll do it myself but thought it worth asking cause it might be a quick job for an expert. Or maybe somehow the PVHVM drivers are already built in and I just don't know how to enable them?

Thanks for reading! I'm pretty much up to my technical depth at this point so any help and guidance from here is greatly appreciated.

Background info follows:

FYI here's a blog post where someone shows on RedHat how they disable the RTL on Xen and enable the PVHVM drivers:
http://djlab.com/2009/12/xen-netfront-intead-of-8139cp-8139too-on-centos-redhat-fedora/


FYI some Linux distributions provide the PVHVM drivers in binary form. For example this link leads to an RPM containing the PVHVM drivers for OpenSuse:
info page: http://rpmfind.net//linux/RPM/opensuse/updates/13.2/x86_64/xen-kmp-default-4.4.2_06_k3.16.7_24-25.1.x86_64.html
source RPM: http://download.opensuse.org/update/13.2/src/xen-4.4.2_06-25.1.src.rpm
binary RPM: ftp://fr2.rpmfind.net/linux/opensuse/update/13.2/x86_64/xen-kmp-default-4.4.2_06_k3.16.7_24-25.1.x86_64.rpm
# these are the files contained in the binary RPM:

Code: [Select]

./lib
./lib/modules
./lib/modules/3.16.7-24-default
./lib/modules/3.16.7-24-default/updates
./lib/modules/3.16.7-24-default/updates/balloon
./lib/modules/3.16.7-24-default/updates/balloon/xen-balloon.ko
./lib/modules/3.16.7-24-default/updates/platform-pci
./lib/modules/3.16.7-24-default/updates/platform-pci/xen-platform-pci.ko
./lib/modules/3.16.7-24-default/updates/usbfront
./lib/modules/3.16.7-24-default/updates/usbfront/xen-usb.ko
./lib/modules/3.16.7-24-default/updates/scsifront
./lib/modules/3.16.7-24-default/updates/scsifront/xen-scsi.ko
./lib/modules/3.16.7-24-default/updates/blkfront
./lib/modules/3.16.7-24-default/updates/blkfront/xen-vbd.ko
./lib/modules/3.16.7-24-default/updates/netfront
./lib/modules/3.16.7-24-default/updates/netfront/xen-vnif.ko
root@ugbuildserver:/home/ubuntu/ubuntucore/tinycorekernel/xen-kmp-default-4.4.2_06_k3.16.7_24-25.1.x86_64#


Here's a list of files from the TinyCore kernel source that I believe to be the PVHVM drivers:

Code: [Select]

./tools/lib/traceevent/plugin_xen.c
./Documentation/tpm/xen-tpmfront.txt
./Documentation/devicetree/bindings/arm/xen.txt
./Documentation/ia64/xen.txt
./Documentation/ABI/stable/sysfs-devices-system-xen_memory
./Documentation/ABI/stable/sysfs-bus-xen-backend
./Documentation/ABI/testing/sysfs-driver-xen-blkback
./Documentation/ABI/testing/sysfs-driver-xen-blkfront
./Documentation/ABI/testing/sysfs-devices-system-xen_cpu
./drivers/video/fbdev/xen-fbfront.c
./drivers/net/xen-netfront.c
./drivers/net/xen-netback
./drivers/net/xen-netback/Makefile
./drivers/net/xen-netback/common.h
./drivers/net/xen-netback/netback.c
./drivers/net/xen-netback/xenbus.c
./drivers/net/xen-netback/interface.c
./drivers/xen
./drivers/xen/platform-pci.c
./drivers/xen/xen-selfballoon.c
./drivers/xen/Makefile
./drivers/xen/xen-acpi-processor.c
./drivers/xen/dbgp.c
./drivers/xen/cpu_hotplug.c
./drivers/xen/gntalloc.c
./drivers/xen/biomerge.c
./drivers/xen/xen-stub.c
./drivers/xen/balloon.c
./drivers/xen/xen-acpi-pad.c
./drivers/xen/privcmd.h
./drivers/xen/grant-table.c
./drivers/xen/acpi.c
./drivers/xen/sys-hypervisor.c
./drivers/xen/gntdev.c
./drivers/xen/xen-pciback
./drivers/xen/xen-pciback/Makefile
./drivers/xen/xen-pciback/conf_space.h
./drivers/xen/xen-pciback/conf_space_quirks.h
./drivers/xen/xen-pciback/conf_space_capability.c
./drivers/xen/xen-pciback/vpci.c
./drivers/xen/xen-pciback/pciback_ops.c
./drivers/xen/xen-pciback/passthrough.c
./drivers/xen/xen-pciback/pciback.h
./drivers/xen/xen-pciback/xenbus.c
./drivers/xen/xen-pciback/conf_space_quirks.c
./drivers/xen/xen-pciback/conf_space_header.c
./drivers/xen/xen-pciback/conf_space.c
./drivers/xen/xen-pciback/pci_stub.c
./drivers/xen/features.c
./drivers/xen/manage.c
./drivers/xen/fallback.c
./drivers/xen/swiotlb-xen.c
./drivers/xen/evtchn.c
./drivers/xen/privcmd.c
./drivers/xen/xen-balloon.c
./drivers/xen/Kconfig
./drivers/xen/pcpu.c
./drivers/xen/xenbus
./drivers/xen/xenbus/Makefile
./drivers/xen/xenbus/xenbus_client.c
./drivers/xen/xenbus/xenbus_probe.h
./drivers/xen/xenbus/xenbus_comms.h
./drivers/xen/xenbus/xenbus_probe_frontend.c
./drivers/xen/xenbus/xenbus_dev_backend.c
./drivers/xen/xenbus/xenbus_probe.c
./drivers/xen/xenbus/xenbus_probe_backend.c
./drivers/xen/xenbus/xenbus_comms.c
./drivers/xen/xenbus/xenbus_xs.c
./drivers/xen/xenbus/xenbus_dev_frontend.c
./drivers/xen/events
./drivers/xen/events/Makefile
./drivers/xen/events/events_base.c
./drivers/xen/events/events_fifo.c
./drivers/xen/events/events_2l.c
./drivers/xen/events/events_internal.h
./drivers/xen/mcelog.c
./drivers/xen/tmem.c
./drivers/xen/xen-acpi-memhotplug.c
./drivers/xen/xenfs
./drivers/xen/xenfs/Makefile
./drivers/xen/xenfs/xenfs.h
./drivers/xen/xenfs/super.c
./drivers/xen/xenfs/xenstored.c
./drivers/xen/xen-acpi-cpuhotplug.c
./drivers/xen/pci.c
./drivers/watchdog/xen_wdt.c
./drivers/input/misc/xen-kbdfront.c
./drivers/pci/xen-pcifront.c
./drivers/block/xen-blkfront.c
./drivers/block/xen-blkback
./drivers/block/xen-blkback/Makefile
./drivers/block/xen-blkback/common.h
./drivers/block/xen-blkback/blkback.c
./drivers/block/xen-blkback/xenbus.c
./drivers/tty/hvc/hvc_xen.c
./drivers/char/tpm/xen-tpmfront.c
./arch/arm/boot/dts/xenvm-4.2.dts
./arch/arm/xen
./arch/arm/xen/Makefile
./arch/arm/xen/enlighten.c
./arch/arm/xen/hypercall.S
./arch/arm/xen/p2m.c
./arch/arm/xen/grant-table.c
./arch/arm/xen/mm.c
./arch/arm/include/asm/xen
./arch/arm/include/asm/xen/page.h
./arch/arm/include/asm/xen/events.h
./arch/arm/include/asm/xen/interface.h
./arch/arm/include/asm/xen/hypervisor.h
./arch/arm/include/asm/xen/page-coherent.h
./arch/arm/include/asm/xen/hypercall.h
./arch/arm64/xen
./arch/arm64/xen/Makefile
./arch/arm64/xen/hypercall.S
./arch/arm64/include/asm/xen
./arch/arm64/include/asm/xen/page.h
./arch/arm64/include/asm/xen/events.h
./arch/arm64/include/asm/xen/interface.h
./arch/arm64/include/asm/xen/hypervisor.h
./arch/arm64/include/asm/xen/page-coherent.h
./arch/arm64/include/asm/xen/hypercall.h
./arch/x86/xen
./arch/x86/xen/Makefile
./arch/x86/xen/enlighten.c
./arch/x86/xen/platform-pci-unplug.c
./arch/x86/xen/pci-swiotlb-xen.c
./arch/x86/xen/xen-asm_64.S
./arch/x86/xen/xen-asm_32.S
./arch/x86/xen/p2m.c
./arch/x86/xen/time.c
./arch/x86/xen/multicalls.h
./arch/x86/xen/grant-table.c
./arch/x86/xen/apic.c
./arch/x86/xen/debugfs.c
./arch/x86/xen/suspend.c
./arch/x86/xen/smp.c
./arch/x86/xen/multicalls.c
./arch/x86/xen/debugfs.h
./arch/x86/xen/smp.h
./arch/x86/xen/xen-ops.h
./arch/x86/xen/mmu.h
./arch/x86/xen/irq.c
./arch/x86/xen/spinlock.c
./arch/x86/xen/Kconfig
./arch/x86/xen/trace.c
./arch/x86/xen/xen-asm.S
./arch/x86/xen/vga.c
./arch/x86/xen/vdso.h
./arch/x86/xen/xen-asm.h
./arch/x86/xen/xen-head.S
./arch/x86/xen/mmu.c
./arch/x86/xen/setup.c
./arch/x86/pci/xen.c
./arch/x86/include/asm/xen
./arch/x86/include/asm/xen/interface_64.h
./arch/x86/include/asm/xen/page.h
./arch/x86/include/asm/xen/trace_types.h
./arch/x86/include/asm/xen/swiotlb-xen.h
./arch/x86/include/asm/xen/events.h
./arch/x86/include/asm/xen/interface.h
./arch/x86/include/asm/xen/hypervisor.h
./arch/x86/include/asm/xen/page-coherent.h
./arch/x86/include/asm/xen/hypercall.h
./arch/x86/include/asm/xen/pci.h
./arch/x86/include/asm/xen/interface_32.h
./include/uapi/xen
./include/uapi/xen/privcmd.h
./include/uapi/xen/gntalloc.h
./include/uapi/xen/Kbuild
./include/uapi/xen/evtchn.h
./include/uapi/xen/gntdev.h
./include/xen
./include/xen/platform_pci.h
./include/xen/grant_table.h
./include/xen/hvc-console.h
./include/xen/xenbus.h
./include/xen/page.h
./include/xen/tmem.h
./include/xen/acpi.h
./include/xen/swiotlb-xen.h
./include/xen/events.h
./include/xen/xen.h
./include/xen/xen-ops.h
./include/xen/features.h
./include/xen/interface
./include/xen/interface/platform.h
./include/xen/interface/memory.h
./include/xen/interface/grant_table.h
./include/xen/interface/sched.h
./include/xen/interface/xen-mca.h
./include/xen/interface/callback.h
./include/xen/interface/xen.h
./include/xen/interface/vcpu.h
./include/xen/interface/hvm
./include/xen/interface/hvm/hvm_op.h
./include/xen/interface/hvm/params.h
./include/xen/interface/features.h
./include/xen/interface/event_channel.h
./include/xen/interface/version.h
./include/xen/interface/elfnote.h
./include/xen/interface/physdev.h
./include/xen/interface/io
./include/xen/interface/io/kbdif.h
./include/xen/interface/io/fbif.h
./include/xen/interface/io/console.h
./include/xen/interface/io/netif.h
./include/xen/interface/io/blkif.h
./include/xen/interface/io/xenbus.h
./include/xen/interface/io/tpmif.h
./include/xen/interface/io/xs_wire.h
./include/xen/interface/io/ring.h
./include/xen/interface/io/protocols.h
./include/xen/interface/io/pciif.h
./include/xen/xenbus_dev.h
./include/xen/balloon.h
./include/xen/hvm.h
./include/trace/events/xen.h
    [EDIT]: Added code tags.  Rich

[as]

An update is that I've managed to make the kernel compile by following the steps in this thread http://forum.tinycorelinux.net/index.php?topic=15656.0

It's still not clear to me exactly how I enable/build the Xen PVHVM drivers in that compile/build process.

Any guidance valued.

thanks

[Juanito]

You need to look at the kernel config.

In the 6.x config I see:

Code: [Select]

CONFIG_NETXEN_NIC=m

..and in the 5.x config:

Code: [Select]

CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y
..so either things changed in the 6.x kernel or something needs enabling - you could try searching using "make menuconfig"

[as]

So is my goal to try to find something in "make menuconfig" that will make the following appear in the .config file?

CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y

I've gone into make menuconfig several times and selected various things and exited out and looked again at the config file but it's still showing:

CONFIG_NETXEN_NIC=m

I'll keep trying.

[Juanito]

I don't know if "CONFIG_NETXEN_NIC" in tc-6.x refers to the same kernel module as "CONFIG_XEN_PVHVM" in tc-5.x or if it is something else entirely.

"CONFIG_NETXEN_NIC=m" causes a kernel module to be compiled, but I don't know what the name of that module is - you could try using the apps gui "provides" function to search for it?

[as]

I haven't been able to do anything yet that results in .config looking like this:

CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y

I wonder why the above would have been in the config for version 5 but not version 6?

I'll admit to be a bit lost here.  Any suggestions for next steps to get the PVHVM drivers compiled?

thanks

[curaga]

It's a matter of the CPU chosen. In our 32-bit releases, we support 486 minimum, and Xen in current kernels requires a higher minimum CPU - so to even be able to see the options, you'd need to pick a higher minimum CPU.

However, our 64-bit kernel has all the relevant Xen options enabled. So you should be able to run either Core64 or CorePure64 on Xen as-is.

Core64 is the 32-bit userspace but with a 64-bit kernel, while pure64 is all 64-bit. The easiest path would be to use the 64-bit kernel on your current 32-bit installed image:
- create core64.gz by "cat rootfs.gz modules64.gz > core64.gz"
- in your bootloader, replace vmlinuz by vmlinuz64 and core.gz by core64.gz

All the mentioned files can be downloaded in the distribution_files directory, under the current release download dir.