Setting up Qemu

[nick65go]

In my naive knowledge, and until someone will prove the contrary (with complete command line example),
the TCDV (Tiny Core Disk Virtual) boot code will not work in the scenario with "qemu -kernel aaa -initrd bbb -drive file=ccc,if=virtio, ..."
Please correct the statements were I am wrong below.Thanks.

1. Qemu boots the kernel (given on its command line)
2. Kernel sees initrd (a ramfs type-temp memory, with no blocks) and decompress it in ram (still no device-blocks aware)
3. Kernel searches, in few hard-coded places, to find the starting init command (or init=/blah if given as a kernel boot parameter)
    if init=/bin/sh then the shell needs ONLY /dev/console, optionaly /dev/null (I tested this)
4. Kernel will automatically populate a lot of devices, in 3 cases: mount devpts, mdev or udev
    with "mount devpts" kernel mounts, over original /dev, with its new default devices  (like 46 of tty# etc)
    in tinycore case /dev/ were already populated, and devpts is not used to mount over them
5. Kernel now has mountpoints /dev/*, even without them being listed in /etc/fstab, neither mounted as /etc/mtab
6. Kernel gives execution further to init, and supervise in the background
7. init can only runs commands or scripts from RAM (uncompressed rootfs.gz) and any mounted (if any) devices (block, nfs etc)
8. So initial /TEST/sda1.qcow2 (placed on host machine) will only expose its "content" to kernel and its rootfs system, if was on qemu command line
9. Kernel does not see original files sda1.qcow2 and rootfs.gz anymore
10. the TCVD=abc.img is useless now, because "inside /TEST/sda1.qcow2" is no abc.img to mount

Summary: TCVD is not for qemu + virtio. Q.E.D.

[Rich]

Hi nick65go

Virtual disk support (tcvd, tiny core virtual disk) is new in Tiny Core v1.4

This was primarily setup for Qemu support, but is useful for anyone.
It uses the Qemu virtual disk image ext2, a loopback file.

To use first specify the 'harddisk' file with the tcvd boot option followed
by your regular boot optons using tcvd, examples:


boot: tinycore waitusb=10 tcvd=harddisk.img tce=tcvd restore=tcvd

This will autoscan for the file named harddisk.img in level one directories and
setup mount capabilities as device tcvd. The subsequent tce=tcvd and restore=tcvd
will use the virtual drive special device.


boot: tinycore waitusb=10 tcvd=sda1/harddisk.img tce=tcvd restore=tcvd

This will look for the loopback file named harddisk on sda1 and setup a special
device tcvd used by the other boot opitons.

boot: tinycore waitusb=10 tcvd=sda1/qemu/harddisk.img tce=tcvd restore=tcvd

This is/was the typical Qemu setup that I used in Damn Small Linux.
Again a special device tcvd will be setup and then other boot options are able
to use it.

[nick65go]

Hi Rich,
For clarity, I do not have a "harddisk.img" (neither inside the root "/" [level 1 directory] , nor in other disk / partitions) INSIDE my container (named XXX.qcow2) which is used "after"/simultaneouslyI bootthis GUEST-machine (under qemu) by kernel + core.gz
 
My container is file, no partitions on it, was ext2-type formatted, and inside the container I have only a /tce folder populated with *tcz etc. So qemu boots from a container. The kernel and core.gz are not in the container.

The idea to have a TCVD boot code (in T.1.4) was based on the assumption that the kernel could see the HOST-machine root files(its /) and HOST-machine's partitions (/sda1), which is NOT the case with my qemu simulation as I explained. In my case, kernel does not see any image "harddisk.img" to mount in its environment.

PS: The only small bug here is that (by default) an instruction like "mount /dev/vda" was not automatically issued; which I could remaster myself.

[Rich]

Hi nick65go
Having never used Qemu, I decided to install it on one of my machines
and play with it.

------- Boot ISO file, desktop auto loads from cde directory, additional extensions auto load from tce directory. -------
First I created an empty formatted image file. I mounted the file and created
a  tce/optional  directory structure and installed  mc.tcz.
I also downloaded:
http://tinycorelinux.net/10.x/x86/release/TinyCore-10.1.iso

Then I ran:

Code: [Select]

qemu-system-i386 -enable-kvm -hda Apps.img -m 1G -cdrom TinyCore-10.1.iso -boot d
The syslinux boot loader displayed, I hit enter and a few moments later was presented with
a desktop. The  wbar  had an  mc  icon, showing that it found and loaded  mc.tcz  from my
Apps.img  file which was mounted as  /mnt/sda.  The  Apps  utility found the  tce  directory
and was able to install extensions.



--------------------- Boot vmlinuz and core.gz, all extensions auto load from tce directory. ---------------------
First I created an empty formatted image file. I mounted the file and created
a  tce/optional  directory structure and installed  Xvesa.tcz flwm_topside.tcz wbar.tcz aterm.tcz mc.tcz.
I also downloaded:
http://tinycorelinux.net/10.x/x86/release/TinyCore-10.1.iso
Then:

Code: [Select]

tc@E310:~/Qemu$ mkdir iso
tc@E310:~/Qemu$ sudo mount TinyCore-10.1.iso iso
mount: /home/tc/Qemu/iso: WARNING: device write-protected, mounted read-only.
# Copy vmlinuz and core.gz to current directory:
tc@E310:~/Qemu$ cp iso/boot/* .
cp: -r not specified; omitting directory 'iso/boot/isolinux'
And launch Qemu:

Code: [Select]

qemu-system-i386 -enable-kvm -hda Apps.img -m 1G -kernel vmlinuz -initrd core.gz "quiet"As before, I was presented with a desktop and  wbar  had an  mc  icon.

The 2 attached files list all of the commands used to set up these tests.
The  FetchExt.sh  command is a script that can be found attached to this post:
https://forum.tinycorelinux.net/index.php/topic,23034.msg164745.html#msg164745

[nick65go]

Hi Rich, thank you for info, feed-back and perseverance!
I hope I will not annoy you with my remarks. I am truly for collaboration. So, may I repeat myself again?

1. using old simulated hardware TC works! I mean if using sintax "qemu -hda xxx.img -cdrom yyy.iso ". In this case a PATA/SATA/AHCI is simulated for hdd and SCSI for cdrom, and motherboard in PXII etc. All is OK, as I said in my previous posts. The mount points are /dev/sdx for HDD and /dev/sr# for CDROM.

2.

Code: [Select]

qemu-system-i386 -m 128M -kernel vmlinuz -initrd MyCore.gz -drive file=MyDVD.iso,if=virtio,media=cdrom,readonly=on -drive file=Mydisk.qcow2,if=virtio,media=disks
Using new VIRTUAL devices [ the main part in code: ,if=virtio,] virtio-blk for HDD, and virtio-scsi for CDROM, will NOT work automatically. The mount points are like /dev/vda for HDD and /dev/vdb for CDROM, depending if HDD is first and cdrom is second in controllers like virtio-pci, PS: When I say virtio all devices, is like I do not use an implicit old-simulation intel E1000 network card, but instead used a virtio-nic etc. The speed is 2x up to 10x time faster -- in theory. It is no point in qemu to simulate all low-level real hardware, instead qemu could simulates (by-pass / simplify internal manufactured logic circuits) to just correct in/out data flow of the virtual device.

[patrikg]

Do you know if there are the correct modules for loading virtio instead of the e1000e mod in your iso file.
I think the modules are in some separate file, and you have to extract the correct ones or even compile sometimes.

[nick65go]

@patrikg: Yes, I check-it, all the "legacy" modules are in modules.gz (or they are build-in), so that the old syntax is working.
Also I check-it, and all necessary "virtio-xxx" are in modules.gz, For each virtio-blk/scsi/pci/net I check-it that all dependencies are present, by using "modinfo -F dependence xxx.ko.gz".

To summarize: I can boot in a shell /bin/sh, logged as tc. Then AFTER I MANUALY mount /dev/vda /mnt/vda, (and make the proper link to /etc/system/tcedir) then I can load manually each *.tcz and voila, Xvesa shows-up. Nothing is missing as files. Just that the virtual disk is not auto-mount, so I initially stay in RAM, with only kernel and core.gz loaded (like I would issue boot-code = base norestore).
PS: I do not have an ISO file. I use from TC14: vmlinuz + (rootfs.gz + modules.gz => core.gz) and manually downloaded Xvesa.tcz + Xprogs.tcz + all their dependencies into a ext2-formated no-partition xxx.qcow2 file,

[nick65go]

@Rich, to convince yourself that both your syntax and my syntax for qemu are correct you can combine together.

1. Just make a xxx.img file with dd with 1MB size; mkfs.ext2 format it -- it will have no partition by default; mount it somewhere; cd into mounted loop; and touch a.txt (to have a file inside it for identification; unmount it.

2. use your normal qemu syntax plus mine, like this (to attach also this xxx.img virtual disk):

Code: [Select]

qemu-system-i386 -enable-kvm -m 128M -cdrom TinyCore-10.1.iso -boot d -drive file=xxx,img,if=virtio,media=diskFYI: my qemu 3.1 needs "-accel kvm", instead of "-enable-kvm"
Inside TC do "mount /dev/vda /mnt/vda"; and "ls /mnt/vda" will see the a.txt file
This will prove the correct qemu syntax was used, plus that all kernel modules are available;
 see with lsmod what is loaded -- virtio-xxx, the legacy modules are mostly built-in and will not show for TC14.
The info about the syntax I learned from internet for qemu 7.2 documentation

[Rich]

Hi nick65go
The  diskfile.img  contains a  tce  directory with the following installed onboot:
Xvesa.tcz, flwm_topside.tcz, wbar.tcz, aterm.tcz, mc.tcz, and grabber.tcz.

This detects  vda , mounts it, loads all extensions, and displays a desktop:

Code: [Select]

qemu-system-i386 -enable-kvm -m 1G -kernel vmlinuz -initrd core.gz -append "quiet waitusb=2" -drive file=diskfile.img,if=virtio,media=disk,format=raw
Remember, Tinycore only automounts drives with tce, persistent home, or persistent opt directories.

[nick65go]

Hi Rich, thank you for your testing.
I will re-do my test cases again, because now for me is not working auto-mounting /dev/vda.
I used qemu 5.1.0 in win10, and qemu 3.1 in Tinycore 14, with vmlinuz + core,gz from TC13 and TC14.

[nick65go]

New tests were done in win10 with qemu 5.1, with an EMPTY ext2 [raw image] abc.img 10 MB, and vmlinuz + core.gz from TC13.

case1: in abc.img there is NOT a /tce folder;
A: qemu without -append tce=sda" will not mount /mnt/sda
B: qemu with -append tce=sda" will mount /mnt/sda AND will populate it with folders /tce/optional /tce/ondemand and file tce/onboot.lst. OK!

case2: in abc.img WITH an empty /tce folder;
A: qemu without -append tce=sda/tce" will mount /mnt/sda. OK!
 Summary: (/tce inside abc.img) OR "qemu -append tce=/sda" to mount /dev/sda and to see /mnt/sda/tce

case3: in abc.img WITHOUT a /tce folder;

Code: [Select]

qemu-system-i386.exe -m 128M -kernel vmlinuz -initrd core.gz \
 -drive file=abc.img,if=virtio,media=disk A: qemu without -append tce=..." will not mount /mnt/vda, same as case 1.A
B: qemu with -append tce=vda" will NOT mount /mnt/vda. !?
C. case 3.B plus "-append waitusb=3". Finaly /dev/vda is mounted, and I am like in case 1.B, with new auto-created folders /tce/optional /tce/ondemand and file tce/onboot.lst

 Summary: for virtio, "qemu -append tce=vda waitusb=3", it does not matter with/wihout folder /tce inside abc.img, to mount /dev/vda and to see /mnt/vda/tce.

Rich, thank you! You highlighted two main things (I missed waitusb=):
1. tce= must be on -append line, to auto-create folder "/tce" IF it did not existed.
2. waitusb=3 must be on -append line, to wait for virtio disk to show-up

PS: it is a race condition in qemu, the kvm emulation for a very fast CPU (running tc-config) versus the slow read of the abc.img from the host disk.

[curaga]

Oh, a slow disk, wouldn't have expected that when SATA disks don't need the wait. Use the label/uuid additions to waitusb to have a fast boot.

[nick65go]

@curaga: Thanks!
in win10 I have both a fast CPU (Intel i5-8350U), and a fast SSD (solid state disk). But unfortunately NO acceleration  (I am not admin, and no acceleration drivers can be installed); qemu using tcg translator x86 qemu-cpu into x86_64 real-CPU, so is still faster than I/O reading a file from SSD disk.

In linux I have a slow CPU (AMD APU A6-6300) but KVM acceleration; CPU is still faster than I/O reading a file from rotational SATA disk.
For files from TC13, waitusb=3 was enough; but for files from TC14 I need waitusb=5+.
So, yes, unbelievable, I did not expected these in qemu inner logic, and I did not used waitusb=; my mistake, sorry.
 

[Rich]

Hi nick65go
Use this for fastest response:

Code: [Select]

tc@box:~/Qemu$ blkid -s UUID diskfile.img
diskfile.img: UUID="f0fb31eb-8b59-4091-a381-93bf508a83e7"
tc@box:~/Qemu$
tc@box:~/Qemu$ qemu-system-i386 -enable-kvm -m 1G -kernel vmlinuz -initrd core.gz -append "quiet waitusb=20:UUID=f0fb31eb-8b59-4091-a381-93bf508a83e7" -drive file=diskfile.img,if=virtio,media=disk,format=raw
tc-config  checks for the presence of that UUID 4 times per second and continues as soon as it finds it.

[nick65go]

@Rich: Thanks!  Because I could change any /both of LABEL or UUID, then:
1. Which is faster: waitusb=seconds:LABEL=TCVM or waitusb=seconds:UUID=12345678-aaaa-bbbb-cccc-12346789012 ?
2. What difference / reduction (in seconds) should we expect? from your experience: like [0.5 ... 1.0] seconds? or less than  2 seconds?
3. Could I use a short and personalized UUID,, like UUID=1234 ? Or is any mandatory template-format with 32 characters?