WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Can't see the volume that I booted from on Google Compute Engine  (Read 2456 times)

Offline as

  • WikiUser
  • *
  • Posts: 29
I'm booting on Google Compute Engine which uses KVM virtualisation.

I'm booting only to ram with a grub.cfg that looks like this:

            menuentry 'tinycore64' {
                insmod part_msdos
                insmod ext2
                set root='hd0,msdos1'
                linux /boot/vmlinuz64 tce=/opt/tce modules=ext4 console=ttyS0,115200 waitusb=10
                initrd /boot/corepure64.gz /boot/ug_initramfs.gz
                echo   'Loading tinycore64...'
            }


The system boots up fine but strangely I can't find the block device that the operating system booted from.  My objective is to get it mounted but there's no sign of it.

Below is the boot messages and at the bottom I got bootlocal.sh to run a number of commands to show me what is going on in the system.

blkid outputs this:
/dev/zram0: UUID="bdad4331-1072-45f5-b1be-4290579384de" TYPE="swap"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"

It's strange - if I run the same image on Xen then I can see the boot volume as /dev/xvda1 I would have expected on KVM it would be at /dev/sda1

Any ideas would be much appreciated.


Start bios (version 1.7.2-20150226_170051-google)
Unable to unlock ram - bridge not found
Ram Size=0x26600000 (0x0000000000000000 high)
Relocating low data from 0x000e5810 to 0x000ef780 (size 2161)
Relocating init from 0x000e6081 to 0x265d3540 (size 51612)
CPU Mhz=2510
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 4 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: map device bdf=00:03.0  bar 0, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:04.0  bar 0, addr 0000c040, size 00000040 [io]
PCI: map device bdf=00:03.0  bar 1, addr febfe000, size 00001000 [mem]
PCI: map device bdf=00:04.0  bar 1, addr febff000, size 00001000 [mem]
PCI: init bdf=00:01.0 id=8086:7110
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.3 id=8086:7113
Using pmtimer, ioport 0xb008, freq 3579 kHz
PCI: init bdf=00:03.0 id=1af4:1004
PCI: init bdf=00:04.0 id=1af4:1000
Found 1 cpu(s) max supported 1 cpu(s)
MP table addr=0x000fdaf0 MPC table addr=0x000fdb00 size=240
SMBIOS ptr=0x000fdad0 table=0x000fd990 size=314
Memory hotplug not enabled. [MHPE=0xffffffff]
ACPI DSDT=0x265fe070
ACPI tables: RSDP=0x000fd960 RSDT=0x265fe030
Scan for VGA option rom
Machine UUID 0ede7055-3661-82dd-b295-f5c92a7fa5b5
Found 4 serial ports
found virtio-scsi at 0:3
Searching bootorder for: /pci@i0cf8/*@3/*@0/*@0,0
Searching bootorder for: /pci@i0cf8/*@3/*@0/*@1,0
virtio-scsi vendor='Google' product='PersistentDisk' rev='1' type=0 removable=0
virtio-scsi blksize=512 sectors=2097152
Searching bootorder for: /pci@i0cf8/*@3/*@0/*@2,0

edited - Searching bootorder.... REPEATS

Searching bootorder for: /pci@i0cf8/*@3/*@0/*@252,0
Searching bootorder for: /pci@i0cf8/*@3/*@0/*@253,0
KBD: int09 handler: AL=0
PS2 keyboard initialized
All threads complete.
Scan for option roms
Searching bootorder for: HALT
drive 0x000fd920: PCHS=0/0/0 translation=lba LCHS=1024/32/63 s=2097152
Space available for UMB: 000c0000-000eb800
Returned 122880 bytes of ZoneHigh
e820 map has 6 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 00000000265fe000 = 1 RAM
  4: 00000000265fe000 - 0000000026600000 = 2 RESERVED
  5: 00000000fffbc000 - 0000000100000000 = 2 RESERVED
Unable to lock ram - bridge not found
KBD: int09 handler: AL=0
enter handle_19:
  NULL
Booting from Hard Disk 0...
Booting from 0000:7c00
[?25lGNU GRUB  version 2.02~beta2-22ubuntu1.1


+----------------------------------------------------------------------------+||||||||||||||||||||||||+----------------------------------------------------------------------------+     Use the ^ and v keys to select which entry is highlighted.         

      Press enter to boot the selected OS, `e' to edit the commands       

      before booting or `c' for a command-line.                            *tinycore64                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         The highlighted entry will be executed automatically in 1s.                    The highlighted entry will be executed automatically in 0s.                 [?25hLoading tinycore64...

Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Initializing cgroup subsys cpuacct
Linux version 3.16.6-tinycore64 (tc@box) (gcc version 4.9.1 (GCC) ) #777 SMP Thu Oct 16 10:21:00 UTC 2014
Command line: BOOT_IMAGE=/boot/vmlinuz64 tce=/opt/tce modules=ext4 console=ttyS0,115200 waitusb=30
Disabled fast string operations
e820: BIOS-provided physical RAM map:


<snip>


Freeing unused kernel memory: 1224K (ffff8800014ce000 - ffff880001600000)
Freeing unused kernel memory: 720K (ffff88000174c000 - ffff880001800000)
Booting Core 6.4 
Running Linux Kernel 3.16.6-tinycore64.
Checking boot options... Done.
Starting udev daemon for hotplug support.../input: PC Speaker as /devices/platform/pcspkr/input/input3
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10
 Done.
ppdev: user-space parallel port driver
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
psmouse serio1: alps: Unknown ALPS touchpad: E7=10 00 64, EC=10 00 64
tsc: Refined TSC clocksource calibration: 2500.002 MHz
input: ImPS/2 Generic Wheel Mouse as /devices/platform/i8042/serio1/input/input5
floppy0: no floppy controllers found
work still pending
loop: module loaded
zram0: detected capacity change from 0 to 137347072
random: mkswap urandom read with 5 bits of entropy available
Adding 134124k swap on /dev/zram0.  Priority:-1 extents:1 across:134124k SSFS
Scanning hard disk partitions to create /etc/fstab 
Setting Language to C Done.
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Possible swap partition(s) enabled.
Loading extensions.../-/ Done.
Setting keymap to us Done.
Setting hostname to box Done.
login[552]: root login on 'tty1'

/opt/bootlocal.sh: line 1: !#/bin/sh: not found
udhcpc (v1.23.1) started
Sending discover...
Sending select for 10.240.0.2...
Lease of 10.240.0.2 obtained, lease time 268435455
deleting routers
route: SIOCDELRT: No such process
route: SIOCADDRT: Network is unreachable
adding dns 169.254.169.254
adding dns 10.240.0.1
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.240.0.1      0.0.0.0         UG    0      0        0 eth0
10.240.0.0      0.0.0.0         255.255.0.0     U     0      0        0 eth0
127.0.0.1       0.0.0.0         255.255.255.255 UH    0      0        0 lo
search bad
nameserver 169.254.169.254
nameserver 10.240.0.1
/opt/bootlocal.sh: line 9: host: not found
search bad
nameserver 169.254.169.254
nameserver 10.240.0.1
major minor  #blocks  name

 251        0     134128 zram0
   7        0         72 loop0
   7        1         16 loop1
   7        2       2088 loop2
   7        3        476 loop3
   7        4       5852 loop4
   7        5        364 loop5
   7        6        916 loop6
fdisk
blkid
/dev/zram0: UUID="bdad4331-1072-45f5-b1be-4290579384de" TYPE="swap"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/loop4: TYPE="squashfs"
/dev/loop5: TYPE="squashfs"
/dev/loop6: TYPE="squashfs"
ls /
bin      etc      home     lib      mnt      proc     run      sys      usr
dev      extras   init     linuxrc  opt      root     sbin     tmp      var
mount: mounting /dev/sda1 on /mnt/sda1 failed: No such device or address
mount: mounting /dev/hda1 on /mnt/hda1 failed: No such device or address
mount: mounting /dev/xvda1 on /mnt/xvda1 failed: No such file or directory
.
./xvda1
./hda1
./sda1

ls /dev

agpgart             loop4               sr0                 tty5
atibm               loop5               sr1                 tty50
audio               loop6               stderr              tty51
beep                loop7               stdin               tty52
block               mcelog              stdout              tty53
char                mem                 tgt                 tty54
console             net                 tty                 tty55
core                network_latency     tty0                tty56
cpu_dma_latency     network_throughput  tty1                tty57
fb0                 null                tty10               tty58
fd                  nvram               tty11               tty59
fd0                 pktcdvd             tty12               tty6
fd0D360             port                tty13               tty60
fd0D720             ppp                 tty14               tty61
fd0H1440            psaux               tty15               tty62
fd0H1722            ptmx                tty16               tty63
flash               pts                 tty17               tty7
full                ram                 tty18               tty8
fuse                ram0                tty19               tty9
hda                 ram1                tty2                ttyS0
hda1                ram2                tty20               ttyS1
hda2                ram3                tty21               ttyS2
hda3                ram4                tty22               ttyS3
hda4                ram5                tty23               urandom
hda5                ram6                tty24               usb
hda6                ram7                tty25               vcs
hda7                random              tty26               vcs0
hda8                rfkill              tty27               vcs1
hda9                rtc                 tty28               vcs2
hdb                 rtc0                tty29               vcs3
hdb1                sda                 tty3                vcs4
hdb2                sda1                tty30               vcs5
hdb3                sda2                tty31               vcs6
hdb4                sda3                tty32               vcs7
hdb5                sda4                tty33               vcsa
hdb6                sda5                tty34               vcsa0
hdb7                sda6                tty35               vcsa1
hdb8                sda7                tty36               vcsa2
hdb9                sda8                tty37               vcsa3
hdc                 sda9                tty38               vcsa4
hdd                 sdb                 tty39               vcsa5
hpet                sdb1                tty4                vcsa6
inportbm            sdb2                tty40               vcsa7
input               sdb3                tty41               vfio
kmem                sdb4                tty42               vga_arbiter
kmsg                sdb5                tty43               watchdog
logibm              sdb6                tty44               watchdog0
loop-control        sdb7                tty45               zero
loop0               sdb8                tty46               zram0
loop1               sdb9                tty47
loop2               shm                 tty48
loop3               snapshot            tty49

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 9461
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #1 on: October 23, 2015, 04:32:58 AM »
tce=/opt/tce is not really a supported option, but shouldn't affect this. Try "dmesg | grep -i scsi".
The only barriers that can stop you are the ones you create yourself.

Offline as

  • WikiUser
  • *
  • Posts: 29
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #2 on: October 23, 2015, 02:36:51 PM »
tc@box:/$ dmesg | grep -i scsi
SCSI subsystem initialized
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
VMware PVSCSI driver - version 1.0.5.0-k
scsi0 : pata_legacy
scsi1 : pata_legacy
tc@box:/$

Doing grep -i scsci on the serial console output from bootup give slightly more information

found virtio-scsi at 0:3
virtio-scsi vendor='Google' product='PersistentDisk' rev='1' type=0 removable=0
virtio-scsi blksize=512 sectors=2097152
virtio-scsi vendor='Google' product='PersistentDisk' rev='1' type=0 removable=0
virtio-scsi blksize=512 sectors=20971520
SCSI subsystem initialized
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
VMware PVSCSI driver - version 1.0.5.0-k
scsi0 : pata_legacy
scsi1 : pata_legacy


Offline as

  • WikiUser
  • *
  • Posts: 29
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #3 on: October 23, 2015, 02:43:28 PM »
A few more bits of information:

Google specifies here the requirements for a machine running as a guest VM, I'm not sure to what extent tinycore meets their requirements.
https://cloud.google.com/compute/docs/tutorials/building-images

The link says:

Required Linux Kernel Options
# to enable paravirtualization functionality.
CONFIG_KVM_GUEST=y
# to enable the paravirtualized clock (if applies to your kernel version).
CONFIG_KVM_CLOCK=y
# to enable paravirtualized PCI devices.
CONFIG_VIRTIO_PCI=y
# to enable access to paravirtualized disks.
CONFIG_SCSI_VIRTIO=y
# to enable access to the networking.
CONFIG_VIRTIO_NET=y
Recommended Linux Kernel Options
# to enable high performance interrupt delivery (needed for Local SSD)
CONFIG_PCI_MSI=y

Wild guessing in the dark here whether this might be related somehow:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/sect-Managing_storage_controllers_in_a_guest.html


Also if it is any help:

tc@box:/$ lspci
00:01.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 03)
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:03.0 Non-VGA unclassified device: Red Hat, Inc Virtio SCSI
00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
tc@box:/$
« Last Edit: October 23, 2015, 02:58:28 PM by as »

Offline Rich

  • TinyCore Moderator
  • Hero Member
  • *****
  • Posts: 5300
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #4 on: October 23, 2015, 03:19:06 PM »
Hi as
If you unpack the linux-3.16 file found here:
http://tinycorelinux.net/6.x/x86_64/release/src/toolchain/
you can check the config file to see if it meets your requirements.

Offline as

  • WikiUser
  • *
  • Posts: 29
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #5 on: October 23, 2015, 03:42:27 PM »
Hi Rich

I've unpacked it.  Which file specifically should I be looking at?

thanks

as

Offline as

  • WikiUser
  • *
  • Posts: 29
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #6 on: October 23, 2015, 04:05:51 PM »
I did "make menuconfig" which created a .config I think

The setting that looks suspicious in .config is

CONFIG_SCSI_VIRTIO=m

whereas the Google docs say it should be CONFIG_SCSI_VIRTIO=y

Does this mean I need to do something to activate the SCSI_VIRTIO driver?

Offline Rich

  • TinyCore Moderator
  • Hero Member
  • *****
  • Posts: 5300
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #7 on: October 23, 2015, 04:16:47 PM »
Hi as
My mistake, try the one of the config files here:
http://tinycorelinux.net/6.x/x86/release/src/kernel/

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 9461
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #8 on: October 24, 2015, 01:13:50 AM »
All those entries are enabled. Hm, maybe "dmesg | grep -i sd" has something more.
The only barriers that can stop you are the ones you create yourself.

Offline as

  • WikiUser
  • *
  • Posts: 29
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #9 on: October 24, 2015, 03:04:34 AM »
OK here it is - I'm really hoping someone has some idea of what has happened to the boot disk because I'm stuck at this stage.  My only suspicion was that there was some sort of driver not loaded.

tc@box:/$ dmesg | grep -i sd
ACPI: RSDP 0x00000000000FD960 000014 (v00 Google)
ACPI: RSDT 0x00000000265FE030 000034 (v01 Google GOOGRSDT 00000001 GOOG 00000001)
ACPI: DSDT 0x00000000265FE070 0015B6 (v01 GOOG   GODSDT   00000001 INTL 20140214)
ACPI: SSDT 0x00000000265FF780 00073D (v01 Google GOOGSSDT 00000001 GOOG 00000001)
usbcore: registered new interface driver ums-isd200
usbcore: registered new interface driver ums-sddr09
usbcore: registered new interface driver ums-sddr55
tc@box:/$
« Last Edit: October 24, 2015, 03:09:48 AM by as »

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 9461
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #10 on: October 24, 2015, 03:59:43 AM »
Testing in Qemu, I get a /dev/vda disk:
qemu -cdrom TinyCore-current.iso -drive file=kovo,if=virtio
The only barriers that can stop you are the ones you create yourself.

Offline as

  • WikiUser
  • *
  • Posts: 29
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #11 on: October 24, 2015, 03:31:18 PM »
After further investigation I'm edging towards convinced that the problem is that Google Compute Engine requires the Virtio-SCSI driver for virtual block device disk access as specified here: https://cloud.google.com/compute/docs/disks/ and also requires the NVMe driver for accessing locally attached SSDs https://en.wikipedia.org/wiki/NVM_Express - I haven't done any investigation of NVMe because for my immediate purposes I don't need it, but it would be good if it came out of the box with TinyCore so that TinyCore ran on Google Compute Engine without need for fiddling with drivers.

The hardware manifest bears out this line of investigation https://cloud.google.com/compute/docs/tutorials/building-images where it specifies that the machine contains "Virtio-SCSI Storage Controller"

I believe that virtio_blk won't do the job, virtio_scsi is required

Unfortunately it appears that 64 bit tinycore does not include a compiled virtio_scsi driver.

I'm a bit puzzled because the messages from Rich above refer to the config for both 64 and 32 bit tinycore, Curaga says that "all those entries are enabled" but as far as I can tell in 64 bit tinycore there's no virtio_scsi driver.

What's the most effective way for me to get a 64 bit virtio_scsi driver to test?

Thanks for all help!

Offline Juanito

  • Administrator
  • Hero Member
  • *****
  • Posts: 9252
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #12 on: October 24, 2015, 08:12:04 PM »
You can double-check if config-3.16.6-tinycore64 has virtio_scsi enabled or not and, if not, re-compile the kernel to get virtio_scsi compiled in or available as a module.

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 9461
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #13 on: October 25, 2015, 01:03:25 AM »
The 64-bit config has scsi_virtio as a module, right - will be changed for 7.x. For this you'll need to remaster the module into base, if using the 64-bit kernel. NVME is enabled in both.
The only barriers that can stop you are the ones you create yourself.

Offline as

  • WikiUser
  • *
  • Posts: 29
Re: Can't see the volume that I booted from on Google Compute Engine
« Reply #14 on: October 26, 2015, 03:32:03 PM »
OK I'm setting out to compile the drivers/virtio_scsi module

I have downloaded the source from here: http://www.gtlib.gatech.edu/pub/tinycore/6.x/x86_64/release/src/toolchain/linux-3.16.2.tar.xz

I believe step one is to do "make oldconfig"

Where can I find the old .config file?  I've dug around but can't see it.

thanks