Hi Pacca
I don't see a whole lot to work with, so here are a few minor observations:
I don't have any Pi hardware so the only baseline I have to compare against is x86. On 3 machines running different TC kernels
the rootfs unpack time is 0.20 to 0.25 seconds. Your Pi takes ~0.64 seconds. Depending on the trade off between file read
speed and decompression speed, you might be able to pick up a couple hundred  milliseconds  by lowering or increasing the
compression ratio.
Jan  1 00:00:04 box user.info kernel: [    0.350381] Trying to unpack rootfs image as initramfs...
Jan  1 00:00:04 box user.info kernel: [    0.994230] Freeing initrd memory: 4464K (dbb94000 - dbff0000)
The  squashfs  driver is taking almost a full second to load. My machines (oldest to newest) take 0.04, 0.125, and 0.04 seconds.
I may be comparing apples to oranges, so just for some context, my oldest machine is a Dell 800 Mhz Pentium III from 2002 running TC4.
Jan  1 00:00:04 box user.info kernel: [    4.307435] squashfs: version 4.0 (2009/01/31) Phillip Lougher
Jan  1 00:00:05 box user.info kernel: [    5.388360] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
Jan  1 00:00:05 box user.info kernel: [    5.499410] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)My understanding is Pi boots off of  mmcblk0p1  and then unmounts it when booting is complete and extensions are stored on
mmcblk0p2  which stays mounted. My question is, why is  mmcblk0p2  being mounted twice?
How big is your  mydata.tgz  file? It looks like it takes about 3 seconds to restore.
Jan  1 00:00:17 box local2.notice sudo:     root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/bin/tar -C / -zxf /mnt/mmcblk0p2/tce/mydata.tgz
Jan  1 00:00:20 box user.notice kernel: [   20.388901] random: crng init done