Tiny Core Linux

Tiny Core Extensions => TCE Bugs => Topic started by: polikuo on October 05, 2020, 02:25:59 AM

Title: Unsquashfs then Resquashfs results in different sizes
Post by: polikuo on October 05, 2020, 02:25:59 AM

Hardware: Raspberry Pi 3 model B
OS: aarch64 beta release
I'm not sure if it happens on other platform so I post it here.

I'm compiling FLTK with XFT support, which should be bloat. (fltk-xft.tcz)
When I squash the extension, the tcz size is actually smaller than the standard one on the repo.  :o
Then I unsquash the repo tcz and resquash it, the new one is also smaller.

Here's what I've done:
Literally unsquashfs and mksquashfs immediately.

Code: [Select]

SQUASH=$(mktemp -d)
cd $SQUASH
unsquashfs /mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz
mksquashfs squashfs-root/ fltk-resquashed.tcz

Compare the result:

Quote

$ du /mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz fltk-resquashed.tcz /tmp/tmp.H7oCSVptRt/fltk-xft.tcz
508K    /mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz
468K    fltk-resquashed.tcz
492K    /tmp/tmp.H7oCSVptRt/fltk-xft.tcz

What kind of dark magic is happening here ?  ???

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: jazzbiker on October 05, 2020, 02:35:26 AM

Hi, polikuo!
The first comes into mind, that original .tcz was squashed with the less compression level for faster access. You can check

Code: [Select]

mksquashfs --helpHave a nice Core!

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: Juanito on October 05, 2020, 06:13:28 AM

Do you get the same as this:

Code: [Select]

$ file /mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz
/mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz: Squashfs filesystem, little endian, version 4.0, zlib compressed, 516285 bytes, 10 inodes, blocksize: 4096 bytes
.i.e. zlib compression, 4k block size?

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: polikuo on October 05, 2020, 06:40:06 AM

Quote from: Juanito on October 05, 2020, 06:13:28 AM

Do you get the same as this:

Code: [Select]

$ file /mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz
/mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz: Squashfs filesystem, little endian, version 4.0, zlib compressed, 516285 bytes, 10 inodes, blocksize: 4096 bytes
.i.e. zlib compression, 4k block size?

Nope  :(

Code: [Select]

tc@box:/mnt/mmcblk0p2/compile/fltk-xft-aarch64$ file /mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz
/mnt/mmcblk0p2/tce/optional/fltk-1.3.tcz: Squashfs filesystem, little endian, version 4.0, zlib compressed, 516285 bytes, 10 inodes, blocksize: 4096 bytes, created: Tue Aug 11 12:37:14 2020
tc@box:/mnt/mmcblk0p2/compile/fltk-xft-aarch64$ file /mnt/mmcblk0p2/tce/optional/fltk-xft.tcz
/mnt/mmcblk0p2/tce/optional/fltk-xft.tcz: Squashfs filesystem, little endian, version 4.0, zlib compressed, 502144 bytes, 13 inodes, blocksize: 131072 bytes, created: Mon Oct  5 08:29:13 2020

IIRC, the core book says the squashfs-tools provides by TC is coded and compiled to use the right size by default.

BTW, I see there's a bajillion extensions packed with blocksize: 131072 bytes

Is it really THAT necessary ? No offense...

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: Juanito on October 05, 2020, 07:24:27 AM

The default tinycore block size for squashfs is meant to be 4k - there’s a patch to set this as the default.

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: Paul_123 on October 05, 2020, 08:19:24 AM

Initial builds of squashfs on aarch64 had some problems, not knowing where the problem came from, some of the patches got dropped.
 When the culprit of the build problems was found, I forgot to go back and rebuild with the 4k blocksize.   

I'll put it on my list to rebuild.

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: Juanito on October 05, 2020, 10:15:13 AM

Ah yes, I'd forgotten about that..

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: curaga on October 05, 2020, 01:10:38 PM

aarch64 is 64-bit with gigs of memory, like x86_64, so using other algos or bigger block sizes there would be fine by me.

Title: Re: Unsquashfs then Resquashfs results in different sizes
Post by: Paul_123 on October 05, 2020, 07:59:00 PM

I went ahead and rebuilt using the 4k blocksize patch.  Feel free to override the default block size.

Note that both arm and aarch64 have zstd support in squashfs-tools, but we have not enabled all of the compression algos in the squashfs kernel module.....so we should keep the default gzip for now.