Cutting boot time

[andyj]

I have a PI 2, which with my few apps and their zillions of dependencies means I have 89 tcz files to load at boot. This takes about 35 or so seconds to get into X. In an attempt to shorten the boot time I created a disk image file, mounted it through a loop device, and used unsquashfs to put the extensions into it. This results in a few directories like etc, lib, var, and usr. The first three are little and I copied them into the stock ramfs image and it's only marginally bigger. The usr directory is about 70 megs so I don't want or necessarily need it in memory (and we're pretty much stuck with the SD card anyway), so I compiled overlayfs into the kernel and hacked tce-setup to, in effect, slide the additional usr files under the existing root tree /usr directory. This works OK and cuts 20+ seconds from the boot time.

What I think would be cleaner, but I have been unsuccessful at doing, is to mount the overlay under the / file system. According to many posts on other distros it's supposed to be possible, but they switch root file systems somewhere in the boot process and we don't, if I understand the process correctly, which I think is the root (pardon the pun) of my problem. This way I wouldn't have to patch the stock ramfs with the etc, lib, and var directories from the extensions and all the other kludges that I have to do to get it going. Maybe I do something in /init or rcS?

Does anyone have any other ideas to cut boot time?

[bmarkus]

Overlay file system is not supported in TC due to performance and stability, see core concept. I'm working on a new startup time tcz loader written from scratch in C. It already offers shorted time. Currently I'm adding multithreading for parallel operation using all CPU cores.

[andyj]

Both of those sound good. And yes, overlay fs is newly merged into 3.18 mainline, and they admit there are some features not yet done. I don't have a problem with the existing boot process in a dev environment, but when I start making production gadgets I'd like them to boot as fast as possible. The an overlay idea is based on the idea of copy2fs for tcz, but doing the copying in advance. I tried preloading the extensions into a big ramfs, but it took a long time to load so I gave up on that. Another idea I had for a prod system is to grab a snapshot of the filesystem late in the boot process and then boot with that. I like the core concepts of small and hard to break which is why I'm using it, but the implementation seems to me aimed more towards development. In production I think we can take the core concepts further and keep it small, make the boot process simpler and faster, and overall even harder to break.

[curaga]

For deployment you can do many things, but I don't think it ever leads to simpler or harder to break Just like you can boot an old embedded ARM to Qt GUI in under two seconds, but everything is so tuned to that particular board that it's a pile of hacks (from an old ELCE presentation).

Combining extensions is one, then finding read-only subtrees is other. If your entire /usr/foo is read-only for example, you can make that an independent squashfs that just gets mounted there. No symlinks, no overlayfs hazards.

Anyway, you should measure exactly what takes the most time. Bootchart would be the usual tool (see the wiki), you'd need to compile it though as the precompiled tarball is for x86.

[andyj]

I'm trying to keep the hacks to a minimum because obviously it makes staying current harder. I'm not interested in starting a fork. Mostly I'm looking for the low hanging fruit in shortening boot time. I can watch and see that loading extensions takes the most amount of time, so I've been concentrating there. I don't want to change a bunch of little things for hardly any benefit. if I can make just one or two changes and get a huge benefit that's good enough for now. Bootchart seems like overkill. If we had a way to spit out the time with each message from tc-config while it's running like the kernel time that would be good enough. But of course adding all that would make it take longer, so that defeats the purpose. If I didn't have so many extensions to load I probably wouldn't care.

[curaga]

The bootcode printk.time=1 will help some.

[Rich]

Hi andyj
The  syslog  bootcode might also be helpful. That will show if any extensions are taking a long time to load in  /tmp/log/messages.

[andyj]

I've managed to pare it down enough that according to /var/log/Xorg.0.log it starts 5.7 seconds after the kernel. The last entry is at 6.8 seconds, but it seems like it takes a few more seconds before the desktop appears. I don't think that there is anything I can do about this. It also seems to take 2-3 seconds to detect all the USB devices. Not much I can do here either. As accurate as I can be with my phone's timer, it's about 4 seconds from power on to motd, then another 6 to X desktop.