Tiny Core Linux

General TC => Tiny Core on Virtual Machines => Topic started by: denbkh on April 17, 2018, 03:52:50 PM

Title: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 17, 2018, 03:52:50 PM
Hello!
I downloaded the v9.0 CorePlus-current.iso and installed it using vmware workstation 12.5.9 build-7535481. Then I installed open-vm-tools-desktop(10.2.5.3619) and did a reboot. After that I can't get gui working anymore.
When I try to start Xorg I have this error:
Code: [Select]
[   141.834] (EE) AIGLX: reverting to software rendering
[   141.840] (EE)
[   141.840] (EE) Backtrace:
[   141.840] (EE) 0: /usr/local/lib/xorg/Xorg (xorg_backtrace+0x43) [0x8141cef]
[   141.840] (EE) 1: /usr/local/lib/xorg/Xorg (0x8048000+0x11ce65) [0x8164e65]
[   141.840] (EE) 2: linux-gate.so.1 (__kernel_rt_sigreturn+0x0) [0xb7fd5b20]
[   141.840] (EE) 3: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1ee0ba) [0xb4e790ba]
[   141.840] (EE) 4: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1ee419) [0xb4e79419]
[   141.840] (EE) 5: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1ee9ce) [0xb4e799ce]
[   141.840] (EE) 6: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1eeaf4) [0xb4e79af4]
[   141.840] (EE) 7: /usr/local/lib/xorg/modules/extensions/libglx.so (0xb7a5c000+0x17321) [0xb7a73321]
[   141.840] (EE) 8: /usr/local/lib/xorg/modules/extensions/libglx.so (0xb7a5c000+0x1c32a) [0xb7a7832a]
[   141.840] (EE) 9: /usr/local/lib/xorg/Xorg (InitExtensions+0x5e) [0x80d80bb]
[   141.840] (EE) 10: /usr/local/lib/xorg/Xorg (0x8048000+0x1ffae) [0x8067fae]
[   141.840] (EE) 11: /lib/libc.so.6 (__libc_start_main+0x156) [0xb7afa7c6]
[   141.840] (EE) 12: /usr/local/lib/xorg/Xorg (0x8048000+0x20791) [0x8068791]
[   141.840] (EE)
[   141.840] (EE) Illegal instruction at address 0xb4fc54a3

Please check full Xorg log file in attachment.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 17, 2018, 08:29:55 PM
Do you have 3D hardware acceleration enabled under the display adapter options? Is this 32-bit or 64-bit? What host are you using, and does it have a working 3D video driver? There are known problems with software rendering. More precisely, we know there are problems, we just haven't found THE problem or any solutions yet. Also, you may need to add:

Code: [Select]
mks.gl.allowBlacklistedDrivers = "TRUE"
to your ~/.vmware/preferences file.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 18, 2018, 03:45:21 AM
Host: Windows 7 Pro (x64) SP1 - with all updates
Host video driver: NVIDIA 390.77 - works without any problems in 3d apps
Tinycore: 9.0 x86
uname -a: Linux box 4.14.10-tinycore #876 SMP Mon Jan 1 15:00:32 UTC 2018 i686

My first run was with 3d acceleration disabled in VM display settings.
But with 3d acceleration enabled I have the same "illegal instruction" bug. Please check the attached log file.

Adding mks.gl.allowBlacklistedDrivers = "TRUE" to the preferences file did not help, got the same error.

I also attached the .vmx file with VM settings, hope this can be useful.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 18, 2018, 06:16:03 AM
I have tried to replicate the problem on my work laptop. It's a Dell 7510. According to the NVIDIA system information dialog, it is Windows 7 Pro 64-bit SP1 (version 6.1.7601) and NVIDIA video driver version 390.77. I am using VM Player 12.5.7, a new VM with TC 9.0 32-bit and open-vm-tools-desktop 10.2.5.3619, so other than the slight VM Player version difference so far everything is about the same.

What is different is that it works for me. I cannot reproduce the problem. My onboot.lst file looks like this:

Code: [Select]
open-vm-tools-desktop.tcz
flwm_topside.tcz
aterm.tcz
wbar.tcz

Somewhere in your dmesg output you should have something like:

Code: [Select]
[drm] DX: yes.
[drm] Atomic: yes
fbcon: svgadrmfb (fb0) is primary device
Console: switching to colour frame buffer device 160x48
[drm] Initialized vmwgfx 2.14.0 20170612 for 0000:00:0f.0 on minor 0

If DX is not yes, then 3d is not enabled.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 18, 2018, 12:32:44 PM
I have the same DX: yes in dmesg. I attached the dmesg log file.
Changing the onboot.lst did not fix the bug.
Could you please upload your VM to any file hosting service (dropbox, google drive, etc.. ) and post a link? I will try to run it.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 18, 2018, 01:51:58 PM
Thank you for the VM link in pm message.
I have run it and got the same bug with "Illegal instruction at address 0xb4fc54a3" :(

Can you try to run it with the latest VMware 12.5.9?
In the VMware changelog they mention adding some "Spectre" (CVE-2017-5715) mitigation ... maybe this could be the source of the problem ...

p.s. I have tried to reply on your pm message twice, but it look like broken - can't see my reply in the "sent items" (http://forum.tinycorelinux.net/index.php?action=pm;f=sent)
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: Rich on April 18, 2018, 01:58:21 PM
Hi denbkh
... p.s. I have tried to reply on your pm message twice, but it look like broken - can't see my reply in the "sent items" (http://forum.tinycorelinux.net/index.php?action=pm;f=sent)

When sending a PM there is a checkbox for saving a copy of your message.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 18, 2018, 02:45:33 PM
I upgraded my VM Player to 12.5.9 and the VM still works. Decreasing the CPU's to one didn't create the problem either. Neither did decreasing the amount of video memory. We both have Intel CPU's. There are two problems here. One is that when a VM is running on your system X doesn't detect DRM. The second is that software DRM fallback is crashing. When I disable 3D on my system DRM is not detected (DX: no) and software rendering works OK:

Code: [Select]
[     8.150] (II) vmware(0): Initialized VMWARE_CTRL extension version 0.2
[     8.172] (WW) vmware(0): Failed to initialize Gallium3D Xa. No render acceleration available.
[     8.172] (WW) vmware(0): Skipped initialization of direct rendering due to lack of render acceleration.
[     8.172] (--) vmware(0): Render acceleration is disabled.
[     8.172] (==) vmware(0): Rendercheck mode is disabled.
[     8.172] (--) vmware(0): Direct rendering (3D) is disabled.
[     8.172] (==) vmware(0): Backing store enabled
[     8.172] (==) vmware(0): Silken mouse enabled
[     8.174] (II) vmware(0): RandR 1.2 enabled, ignore the following RandR disabled message.
[     8.174] (==) vmware(0): DPMS enabled
[     8.174] (II) vmware(0): No 3D acceleration. Not setting up textured video.
[     8.174] (--) RandR disabled
[     8.174] (II) AIGLX: Screen 0 is not DRI2 capable
[     8.174] (EE) AIGLX: reverting to software rendering
[     8.358] (II) IGLX: enabled GLX_MESA_copy_sub_buffer
[     8.358] (II) IGLX: Loaded and initialized swrast
[     8.358] (II) GLX: Initialized DRISWRAST GL provider for screen 0

I cannot reproduce the problem. Maybe you have something else installed on your system which is interfering with VMware? Another hypervisor like VirtualBox also installed with services running in the background? Maybe check BIOS settings? Maybe update system BIOS?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 18, 2018, 03:21:11 PM
Quote
... Maybe you have something else installed on your system which is interfering with VMware? Another hypervisor like VirtualBox also installed with services running in the background? Maybe check BIOS settings? Maybe update system BIOS?
Hmm, I don't have VirtualBox on my PC.
The interesting thing is that open-vm-tools-desktop in my LUbuntu 17.10 x86 VM works without any problems with full 3d acceleration. Checked that with glxinfo and glmark2. So that I think the problem is not with my BIOS.
Any ideas how can I help you reproduce the bug?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 18, 2018, 04:50:30 PM
Those are good points. Did you upgrade VM Player from a previous version, or was it a clean install? When X crashes at startup does it fall back to a console prompt? If so, maybe you could use strace with startx to get some more info? How do you know that Ubuntu is using 3D and not software? Have you checked the Xorg.0.log?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 18, 2018, 06:00:24 PM
Quote
Did you upgrade VM Player from a previous version, or was it a clean install?
clean install
Quote
When X crashes at startup does it fall back to a console prompt?
yes, see the screenshot https://yadi.sk/i/HEQpTH8f3UYdwF
Quote
If so, maybe you could use strace with startx to get some more info?
could you please provide the full command?
Quote
How do you know that Ubuntu is using 3D and not software?
$ glxinfo | grep "direct rendering"
direct rendering: Yes


I have attached the logs from LUbuntu if they can help.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 18, 2018, 07:19:59 PM
After it crashes, if you type startx at the console prompt does it work or crash again? Have you tried 64-bit TinyCore? For strace, load the strace extension and try:

Code: [Select]
strace -D -f -s 254 -o strace-x.log startx
Maybe some AV is doing something? I've tried every combination of settings in VM Player I can find and I can't reproduce the problem. It works in spite of everything I do to try to break it.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 03:44:12 AM
Quote
After it crashes, if you type startx at the console prompt does it work or crash again?
Yes, it crashes with the same error.
Quote
Have you tried 64-bit TinyCore?
Yes. It crashes with the same "Illegal instruction" error, but with 64-bit address.
Quote
Maybe some AV is doing something?
I tried to boot VM with disabled AV(Kaspersky free 18.0.0.405(g)) - got the same error.

Please check the log files with strace for both x64 and x86 VMs.
https://yadi.sk/d/0YxnzAaf3UZaYG (can't attach big files here)
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 19, 2018, 05:24:21 AM
Looks like things go bad around here:

Code: [Select]
7467  stat64("/dev/dri/renderD128", {st_mode=S_IFCHR|0660, st_rdev=makedev(226, 128), ...}) = 0
7467  readlink("/sys/dev/char/226:128/device/subsystem", "../../../bus/pci", 4096) = 16
7467  openat(AT_FDCWD, "/sys/dev/char/226:128/device/uevent", O_RDONLY|O_LARGEFILE) = 17
7467  fstat64(17, {st_mode=S_IFREG|0644, st_size=4096, ...}) = 0
7467  read(17, "DRIVER=vmwgfx\nPCI_CLASS=30000\nPCI_ID=15AD:0405\nPCI_SUBSYS_ID=15AD:0405\nPCI_SLOT_NAME=0000:00:0f.0\nMODALIAS=pci:v000015ADd00000405sv000015ADsd00000405bc03sc00i00\n", 4096) = 161
7467  close(17)                         = 0
7467  openat(AT_FDCWD, "/sys/dev/char/226:128/device/vendor", O_RDONLY|O_LARGEFILE) = 17
7467  fstat64(17, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
7467  read(17, "0x15ad\n", 4096)        = 7
7467  close(17)                         = 0
7467  openat(AT_FDCWD, "/sys/dev/char/226:128/device/device", O_RDONLY|O_LARGEFILE) = 17
7467  fstat64(17, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
7467  read(17, "0x0405\n", 4096)        = 7
7467  close(17)                         = 0
7467  openat(AT_FDCWD, "/sys/dev/char/226:128/device/subsystem_vendor", O_RDONLY|O_LARGEFILE) = 17
7467  fstat64(17, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
7467  read(17, "0x15ad\n", 4096)        = 7
7467  close(17)                         = 0
7467  openat(AT_FDCWD, "/sys/dev/char/226:128/device/subsystem_device", O_RDONLY|O_LARGEFILE) = 17
7467  fstat64(17, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
7467  read(17, "0x0405\n", 4096)        = 7
7467  close(17)                         = 0
7467  getdents64(16, /* 0 entries */, 32768) = 0
7467  close(16)                         = 0
7467  SYS_355(0xbff36444, 0x4, 0x1, 0x91cf720, 0x4) = 4
7467  openat(AT_FDCWD, "/etc/drirc", O_RDONLY|O_LARGEFILE) = 16
7467  read(16, "<!--\n\n============================================\nApplication bugs worked around in this file:\n============================================\n\n* Unigine Heaven 3.0 and older contain too many bugs and can't be supported\n  by drivers that want to be complia"..., 4096) = 4096
7467  SYS_355(0xbff36414, 0x4, 0x1, 0x91cf720, 0x4) = 4
7467  read(16, "tion name=\"allow_glsl_builtin_variable_redeclaration\" value=\"true\" />\n        </application>\n\n        <application name=\"Dead Island Riptide Definitive Edition\" executable=\"DeadIslandRiptideGame\">\n            <option name=\"allow_glsl_builtin_variable_red"..., 4096) = 4096
7467  read(16, "<option name=\"mesa_glthread\" value=\"true\"/>\n        </application>\n\n        <application name=\"Borderlands 2\" executable=\"Borderlands2\">\n            <option name=\"mesa_glthread\" value=\"true\"/>\n        </application>\n\n        <application name=\"Civilizati"..., 4096) = 4096
7467  read(16, "l_sync_control\" value=\"true\" />\n        </application>\n    </device>\n    <device driver=\"radeonsi\">\n        <application name=\"ARK: Survival Evolved (and unintentionally the UE4 demo template)\" executable=\"ShooterGame\">\n            <option name=\"radeonsi"..., 4096) = 348
7467  read(16, "", 4096)                = 0
7467  close(16)                         = 0
7467  openat(AT_FDCWD, "/root/.drirc", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
7467  --- SIGILL (Illegal instruction) @ 0 (0) ---

Your CPU is "Intel(R) Xeon(R) CPU E5472  @ 3.00GHz (family: 0x6, model: 0x17, stepping: 0x6)" which I wouldn't think would be a problem.

Did you change any of the NVIDIA driver settings from their defaults? What NVIDIA chip do you have?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: Rich on April 19, 2018, 06:30:02 AM
Hi andyj
The listing in the first post of this thread looks virtually identical to what is shown here:
http://forum.tinycorelinux.net/index.php/topic,21737.msg136290.html#msg136290
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 06:39:12 AM
Quote
Did you change any of the NVIDIA driver settings from their defaults?
No, default settings for everything.
Quote
What NVIDIA chip do you have?
Card: NVIDIA GeForce GTX 760 (ASUS GTX760 DirectCU II OC)
GPU: GK104-225
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 19, 2018, 06:55:23 AM
The listing in the first post of this thread looks virtually identical to what is shown here:
http://forum.tinycorelinux.net/index.php/topic,21737.msg136290.html#msg136290

Yes, it does, but just about everything else was different. The part I really don't get is that the VM that works on my laptop, I zipped it up and sent it to @denbkh, it didn't work on his computer when as far as we can tell almost everything is the same.

@denbkh,

Do you have any other graphics adapters like Intel (I do, so it isn't necessarily a problem) or other maybe unusual hardware peripherals and drivers?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 07:06:18 AM
Quote
Do you have any other graphics adapters like Intel (I do, so it isn't necessarily a problem) or other maybe unusual hardware peripherals and drivers?
Hmm, no, I have only one GPU with official drivers.
I can try to run strace on LUbuntu with their version of open-vm-tools-desktop, maybe it can help. But for this I need a full command because LUbuntu does not have "startx" in $PATH
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 07:22:43 AM
And I also can run strace on tinycore without open-vm-tools-desktop if you need that.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 19, 2018, 10:06:29 AM
I don't know if it's an actual illegal instruction (compiled with wrong target, very unlikely) or trying to execute data (much more likely, and a bug). Try changing the first line of ~/.xsession to:

Code: [Select]
LIBGL_DEBUG=verbose /usr/local/bin/Xorg -nolisten tcp -logverbose 7 -verbose 7 2>~/Xorg.0.err &

Maybe we'll get something useful.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 11:25:30 AM
I have attached the log files.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 19, 2018, 03:17:03 PM
The logs aren't very revealing, other than something has changed because the hardware rendering is being detected and used now. I don't suppose upgrading VM Player to 14.1 is an option? Is installing TC to a USB drive and booting from that an option? What model computer is this that has a Xeon processor running Windows 7?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 03:48:22 PM
Quote
The logs aren't very revealing, other than something has changed because the hardware rendering is being detected and used now
If you check the logs from message #3 in this topic you can see that 3d acceleration was detected there. I have compared the logs and they look almost identical. So that no surprises that when I enable 3d in VM it will be detected, right?
Code: [Select]
[   318.384] (--) vmware(0): Render acceleration is enabled.
[   318.384] (==) vmware(0): Rendercheck mode is disabled.
[   318.384] (--) vmware(0): Direct rendering (3D) is enabled.
Quote
I don't suppose upgrading VM Player to 14.1 is an option?
Tried that. Unfortunately, VMware does not support my CPU in 14.x
Quote
Is installing TC to a USB drive and booting from that an option?

Do you mean without VM and without open-vm-tools-desktop?
Quote
What model computer is this that has a Xeon processor running Windows 7?
This is a custom built PC. I attached server CPU to the desktop. Google for Xeon pin mod for LGA775 if you are interesting. Don't think that it could be a source of this bug because open-vm-tools-desktop from LUbuntu works fine.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 19, 2018, 04:13:09 PM
I doubt open-vm-tools-desktop is the problem. To test that use this for onboot.lst and reboot the VM:

Code: [Select]
graphics-KERNEL.tcz
xf86-input-vmmouse.tcz
xf86-video-vmware.tcz
Xorg-7.7-3d.tcz
gtkmm.tcz
rxvt.tcz
flwm_topside.tcz
wbar.tcz

It's interesting that your CPU isn't compatible with VM Player 14. I'm wondering if X and/or the GL libraries we're compiled with an option your particular CPU doesn't support that Ubuntu didn't use. Have you tried any of the compatibility check boxes under the Virtual Processors option?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 04:32:45 PM
I tried with your onboot.lst and got the same "Illegal instruction" error.
Is it possible to find which instruction is "illegal"? And which component contains it?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: denbkh on April 19, 2018, 04:41:46 PM
Quote
Have you tried any of the compatibility check boxes under the Virtual Processors option?
If you are about "Hardware > Processor > Virtualization engine" than I tried with "Disable acceleration for binary translation" - same error. With other two checkboxes VM does not boot.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: Rich on April 19, 2018, 05:52:53 PM
Hi denbkh
I tried with your onboot.lst and got the same "Illegal instruction" error.
Is it possible to find which instruction is "illegal"? And which component contains it?
Based on the address reported here:
Code: [Select]
[   141.840] (EE) Illegal instruction at address 0xb4fc54a3I would say this is the most likely suspect:
Code: [Select]
[   141.840] (EE) 3: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1ee0ba) [0xb4e790ba]
[   141.840] (EE) 4: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1ee419) [0xb4e79419]
[   141.840] (EE) 5: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1ee9ce) [0xb4e799ce]
[   141.840] (EE) 6: /usr/local/lib/dri/swrast_dri.so (0xb4c8b000+0x1eeaf4) [0xb4e79af4]
and is provided by  Xorg-7.7-3d.tcz.
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: andyj on April 19, 2018, 09:03:11 PM
I tried with your onboot.lst and got the same "Illegal instruction" error.
That eliminates open-vm-tools and open-vm-tools-desktop as the problem. We're back to some instruction in TC 9 your CPU doesn't like. Did you try TC 9 from a USB drive? Have you tried any earlier versions of TC like 8 or earlier in a VM?
Title: Re: Can't start Xorg after installing open-vm-tools-desktop: "Illegal instruction"
Post by: Juanito on April 20, 2018, 12:15:54 AM
Xorg-7.7 is limited to pentium-m and up