Boot hangs without monitor attached?

[gerald_clark]

That is going to be a bit difficult with no persistent storage, stock core ISO and no monitor.

[nurbles]

gerald_clark:  The ISO is useful as a simple to use, commonly known test case.  It should be able to boot to the $ prompt with no monitor attached.  It cannot.  Why should I waste the time and effort to build a compact flash of TC if the standard ISO cannot boot?

rich:  I don't understand your obsession with bios.  The bios doesn't change just because I inserted a TC CDROM instead of ttylinux or DSL, or Ubuntu or whatever.  *ALL* of the other bootable CDs will boot without a monitor attached and be fully functional if I connect one.  TC will not.

tinypoodle:  serialconsole would be fine as long as it is something I can enable it on a commonly distributed TC image.  At this point, you have all tried to point to so many other things that I am unwilling to try anything that is not provided by TC because anything I make will be suspect by definition.  I'm not trying to be difficult, but 30 years of experience (on both sides of support calls) has taught me to use ONLY packages from the "vendor" in pristine condition when debugging an issue.  If the vendor wants to supply a test build, that's fine, but neither you nor I will trust anything *I* build up, nor should you.

To all:  I understand not wanting to believe that your package/code/favorite has a problem, but bugs exist.  Why must everything I say be wrong?  Why cannot TC simply have an issue with my hardware platform?  Once we can admit that, we can start looking at things that might try to probe the monitor during boot.  It is quite likely that one of those is hanging, or not setting a value that something else needs to work properly.

Regardless, none of the three different ISO image versions of TC will boot on my hardware without a monitor attached.  **ALL** other versions of linux I've tried, plus Windows and FreeDOS *WILL* boot without a monitor attached.  Given those facts, it is NOT POSSIBLE that BIOS is the problem (though TC may have an issue /with/ the BIOS).

Lastly, I'm not trying to malign or impune TC at all.  I think it is a great distro and I really want to use it (to the point that we had hardware made to work around this TC bug!)  But since this bug exists, it might become more prominent in a future release and/or on future hardware, so I thought that the TC gurus might be interested in looking into it.  I'm willing to run test code, if anyone wants to provide it.  If no one is interested or the TC gurus just want to keep their heads in the sand, that's fine, too.  Let me know and I'll give up on this line of questioning.

[gerald_clark]

I can install to a flash drive, install dropbear, set it up for ssh login, reboot and login from another computer.
Then I shut it down, disconnect the monitor and keyboard, and power it on.
I login with ssh.

If you are not willing to do the same, I can't see how we can help troubleshoot a problem only you are having.

[Rich]

Hi nurbles
I'm not obsessed with the BIOS, after the BIOS comes the bootloader and then the kernel. All three of these should be
writing something to the video RAM, yet the screen is blank when the monitor gets connected. Here's something you can
try, take the USB stick that boots with a monitor attached, open the  extlinux.conf  file and remove the word  quiet.
Boot the stick without a monitor attached, then attach the monitor. If it got to the kernel, there should be something on
the screen showing where it stopped.

[gerald_clark]

The fact that there is no visible display does not mean that core is not booted and running.
It only means that your monitor is not displaying anything.
You really need to enable a telnet or ssh login to attempt to examine the non-displaying system.

[tinypoodle]

tinypoodle:  serialconsole would be fine as long as it is something I can enable it on a commonly distributed TC image.

I am not aware of any obvious reason why you couldn't. Are you?

Here is a good and detailed example of troubleshooting:
http://forum.tinycorelinux.net/index.php/topic,11402.msg60034.html#msg60034

To all:  I understand not wanting to believe that your package/code/favorite has a problem, but bugs exist.  Why must everything I say be wrong? Why cannot TC simply have an issue with my hardware platform?  Once we can admit that, we can start looking at things that might try to probe the monitor during boot.

At current this is not about believing or you being wrong.
All we have is a bug report lacking any evidence and without anyone else being able to reproduce.

it might become more prominent in a future release and/or on future hardware, so I thought that the TC gurus might be interested in looking into it.

Speculative, at least as much a chance it might just vanish with a different kernel version (or config).

I'm willing to run test code, if anyone wants to provide it.

No idea what you have in mind by that, and supposed you would be provided with anything such, I can't see how that could be of help as long as you cannot access any kernel output which is the reason noone could help more at current any further than making suggestions to change exactly that.

[nurbles]

First, sorry about the delay ... holiday season has me going too many directions at once. Replies, by #:

#17:  This all started with a fully functional system that has dropbear installed.  When the monitor was disconnected (as opposed to powered off) it was no longer a functional system.

#18:  There are at least two times during the TC Linux boot (AFTER BIOS) that the screen is cleared before more data is written.  I suspect that the hang may be occurring at one of those points.

#19:  When there is a monitor attached, TC booted from a CDROM will at least start its network using DHCP and respond to pings at the address it was given.  With no monitor attached, it never requests an address from the DHCP server and therefore, cannot respond to network traffic.  Ergo, I call it hung.  It will also not respond to a keyboard or mouse once a monitor IS connected, even if they were present at boot.  Again, I call that hung.

#20:  OK, I figured out how to activate a serial console using a standard CDROM distro.  Using the minimal Core image, at the boot: prompt I'm entering 'mc console=tty0 console=ttyS0' (without the quotes) and I capture the following output from a boot with a monitor attached:

    Booting Core 5.1
    Running Linux Kernel 3.8.13-tinycore.
    Checking boot options... Done.
    Starting udev daemon for hotplug support... Done.
    Scanning hard disk partitions to create /etc/fstab
    Setting Language to C Done.
    Possible swap partition(s) enabled.
    Loading extensions... Done.
    Setting keymap to us Done.
    Setting hostname to box Done.
    login[342]: root login on 'tty1'

Booting without a monitor attached produces *NO OUTPUT* on the serial line.  I tried typing the command line blind as well as disconnecting the monitor between typing the command line and pressing ENTER to activate it.  In both cases, the CDROM spins up like it is loading, then nothing.

re "All we have is a bug report lacking any evidence and without anyone else being able to reproduce.":  Unless someone has taken the time and effort to obtain the same Acrosser AR-ES0631 hardware, then no one has actually tried to reproduce the problem.  It doesn't exist on anything else I've tried, but neither does it exist on our hardware with any other operating system I've tried.  Since the beginning, I've tried to be clear that this TC problem is only manifesting on one hardware platform that I know of.  Unfortunately, it is the one that my boss already bought 20 of and I *MUST* use it.  For now, we just labeld the dummy VGA connectors we were forced to build to work around this TC bug as "TinyCore Linux Bug Prevention Dongle" and are proceeding from there.

Yes, of course my idea that a known (but rare) bug might become less rare in the future is speculative.  Since it exists in TC at least back to the early 3.x releases and is still present (but is NOT in any other linux I've tried) I'm comfortable assuming that TC will carry it until it is fixed.  If any other hardware starts working like the Acrosser AR-ES0631 hardware that we are using, it will appear again.

For now, I guess I give up.  I don't want to keep bothering (and likely annoying) the good people here who are honestly trying to help.  As a software developer, I understand the urge to dismiss/ignore bugs one cannot duplicate, because being able to duplicate them means that one may use many tools to make the debugging easier.  Unfortunately, that is rarely an option for me and I must still go into the code -- without a debugger! -- to find the causes for things I cannot duplicate.  Since this is a free product, I obviously cannot ask for that level of dedication and work.  If no one cares about bugs they cannot duplicate, then they will remain and that's just the way it is.

Thanks for listening and for all of the suggestions.

EDIT:  My boss just told me that I can offer to send someone one of the Acrosser boxes we're using in order to duplicate, debug and hopefully resolve this issue.  Assuming, of course, that anyone is interested.  You can contact me at svalli at e-visions dot com.

[tinypoodle]

I didn't see any feedback from you wrt link posted in Reply #3.

[tinypoodle]

#20:  OK, I figured out how to activate a serial console using a standard CDROM distro.  Using the minimal Core image, at the boot: prompt I'm entering 'mc console=tty0 console=ttyS0' (without the quotes) and I capture the following output from a boot with a monitor attached:

    Booting Core 5.1
    Running Linux Kernel 3.8.13-tinycore.
    Checking boot options... Done.
    Starting udev daemon for hotplug support... Done.
    Scanning hard disk partitions to create /etc/fstab
    Setting Language to C Done.
    Possible swap partition(s) enabled.
    Loading extensions... Done.
    Setting keymap to us Done.
    Setting hostname to box Done.
    login[342]: root login on 'tty1'

Hmm, this appears to be all output of Core only, nothing of the kernel whatsoever...

[nurbles]

re #22:  Sorry, I somehow missed message #3, it looks promising.  I do not know how to determine which version of grub is included with the standard ISO images of TC.  Apparently GNU GRUB 0.97 was what I got when I had TC grab a copy of grub for my compact flash image (a 3.x series TC release).  However, if it is a grub issue, i I wonder if TC is planning to switch to grub2 any time soon?  I will, however, try to build a compact flash with grub2 instead of grub tomorrow morning when I get to work.  I'll let you know how it goes.

re #23:  I agree and was surprised that I saw no output from the kernel.  Apparently I still don't know the correct kernel command line switches to get it to write anything out to the serial.  If someone could provide them, I'll try them.

[tinypoodle]

re #22:  Sorry, I somehow missed message #3, it looks promising.  I do not know how to determine which version of grub is included with the standard ISO images of TC.

AFAIK there is no grub included in any of the .iso's.

[Rich]

Hmm, this appears to be all output of Core only, nothing of the kernel whatsoever...

Maybe the   quiet   bootcode needs to be removed.

[maro]

Well, it's kind of obvious that one can't expect that the stock standard TC ISO image is particular good for the more advanced troubleshooting that is called for here. It would therefore be more appropriate to use a USB device for booting as the necessary changes to the kernel boot codes are a bit easier to implement than possibly repeatedly burning CD-ROMs (or rather CD-RW).

I've written a few postings (e.g. here and other ones referenced) some time ago about related subjects. I'd suggest to start with only 'console=ttyS0,38400' as the boot code (which has to go in the respective stanza of the boot loader chosen). Note, that the speed will need to be adjusted according to what the receiving serial interface will be able to "understand".

I personally would probably also only start with the absolute minimum (e.g. 'vmlinuz' and 'rootfs.gz' from here), but then I do have the advantage that I know how to build up layer by layer the experiment as and when required. The results found when using the TC files should then be compared with a similar experiment using another minimal boot set-up from a distribution that does not show the apparent "misbehaviour".

What I've written so far helps only if TC contains the "bug", but what could be done if the boot loader (again) is at fault?  I've just done a quick test for the ISOLINUX bootloader. I started with 'TinyCore-5.1.iso' and changed 'boot/isolinux/isolinux.cfg' to a more minimalist one:

Code: [Select]

SERIAL 0 38400
DEFAULT core
PROMPT 1
TIMEOUT 100

LABEL core
KERNEL /boot/vmlinuz
INITRD /boot/core.gz
APPEND debug console=ttyS0,38400Note that I've enabled a serial port to act as the console for ISOLINUX itself (via the first line) as well as sending all the kernel boot process output to the same (via the last line). So, if a properly connected serial terminal does not show

Code: [Select]

ISOLINUX 4.05 0x4f92e181  Copyright (C) 1994-2011 H. Peter Anvin et al
boot:then I'd blame ISOLINUX. If the prompt shows up the user has 10 seconds to either hit 'Enter' to straight away continue with the (default) 'core' boot stanza, or by hitting the 'Tab' key gets the (rather short) list of all available labels.
(I'm aware that using an ISO image goes a bit against my earlier suggestion, but it allowed me at least to do this test rather quickly and seemed to be quite suitable for the particular angle of investigation).

Alternatively I've also done a quick test with a Super Grub2 Disk. I implemented a change in 'boot/grub/grub.cfg' insofar as I moved the three lines of menuentry "Enable serial terminal" to the very top of the file. This way the serial interface is by default enabled, and might therefore allow to see how this boot loader handles the problematic situation.

[curaga]

However, if it is a grub issue, i I wonder if TC is planning to switch to grub2 any time soon?

We support all bootloaders, from lilo to grub 1, grub 2 to the extlinux family. Our installer uses syslinux/extlinux, IIRC it never used grub (the manual install instructions were for grub 0.97).

[nurbles]

First, let me say that I've rebuilt my compact flash using grub2 instead of grub-0.97 and it now boots just fine without a monitor attached!  My problem is resolved (and I feel like an idiot for overlooking #3!)

re #25:  Whatever the TC ISOs are currently using for a boot loader has the same hanging bug that grub1 has when run on my hardware.  Since someone discovered that grub1 was the cause, I didn't expect other boot loaders to have the same problem.  My bad.

re #26:  Too bad there aren't "anti-switches" for the linux command line that counteract other switches on the command line.     (I know, security nightmare!)  If they existed, then one could still use a stock distro to debug by re-enabling boot time messages.

re #27:  As a developer myself, I always try to use a standard/stock version of anything that isn't working right.  Often I discover that the stock version works fine and something I tweaked is at fault.  Or, when I try to report my bug, all I get is fingers pointing at what I've done because none of the developers can trust it (nor should they).  So, while you are correct and offer what appears to be a very good procedure, I would expect to spend more time defending the procedure and config than tracking the bug.  At least, that's been my experience, including here when the discussion started and I was told (repeatedly) that 3rd party installers aren't supported (even though I needed to boot a USB in order to use TC's standard installer).

re #28:  Those manual instructions are what I followed to create my bootable compact flash image.  If I'd decided to trust a 3rd party instead of the source, I might never have encountered my issue.  I found some other TC-to-CF instructions that use grub2 instead at http://dev.funkynerd.com/knowledgebase/articles/10 in case anyone is interested in incorporating them into the TC documentation.

So, the bottom line appears to be that some of the Geode based systems without a monitor physically attached will hang in some boot loaders (notably, whatever is on the TC ISO images, whatever the Universal USB Installer is using and grub prior to grub2) but grub2 works just fine.

Thanks to everyone for your help!