Tiny Core Linux

Tiny Core Extensions => TCE Talk => Topic started by: Jason W on November 11, 2009, 11:28:58 PM

Title: Squashfs extensions
Post by: Jason W on November 11, 2009, 11:28:58 PM
As mentioned in another thread, squashfs caches to ram when the contents of the mounted extension is read.  The default block size used with mksquashfs is 128kb.  When 4kb block size is used, the runtime memory overhead of squashfs extensions is substantially reduced, almost cut in half.  Both before and after flushing memory cache.  The only price is slightly larger tcz extensions.  But that is worth the performance benefits.  

The squashfs-tools extension has been updated to default to 4kb block size.  Use the new extension, or use this command when making extensions:

mksquashfs dirname/ extensionname.tcz -b 4096

I will soon update the extensions in the repo to the smaller block size.

Title: Re: Squashfs extensions
Post by: Jason W on November 16, 2009, 10:26:37 AM
This change is being reverted due to filesystem errors. 

Thank you for your patience while I reupload the recently submitted extensions.
Title: Re: Squashfs extensions
Post by: Jason W on November 16, 2009, 10:32:18 AM
Repo has been restored to it's state as of 11.11.09.

I will repost any updates tonight that happened since then.
Title: Re: Squashfs extensions
Post by: bmarkus on November 16, 2009, 10:43:00 AM
squashfs-tools-4.0.tcz is still the new with default 4k block size after the rollback.
Title: Re: Squashfs extensions
Post by: Jason W on November 16, 2009, 11:09:59 AM
Thanks. 

I have the old one and will repost tonight around 8pm EST when I get home.

For now, use "mksquashfs dir/ extension.tcz -b 131072"
Title: Re: Squashfs extensions
Post by: Jason W on November 16, 2009, 11:12:27 AM
Lets hold tight for the moment before making any more extensions.  This issue is still under investigation and will be resolved as soon as possible.

The old repo is online for now pending results of testing.
Title: Re: Squashfs extensions
Post by: Jason W on November 16, 2009, 11:39:21 AM
It appears busybox mount is the culprit.

The 4096kb repo is now reinstated, no one needs to change anything.

Title: Re: Squashfs extensions
Post by: jpeters on November 16, 2009, 05:07:17 PM
It appears busybox mount is the culprit.

The 4096kb repo is now reinstated, no one needs to change anything.



What about squashfs-tools-4.0.tcz ? Is the present version correct to use in default. (without -b )?
Title: Re: Squashfs extensions
Post by: Jason W on November 16, 2009, 05:17:13 PM
Correct, the default (-b 4096) is what to use.

Block size does not appear to be the issue seen in the latest RC.
Title: Re: Squashfs extensions
Post by: Jason W on November 18, 2009, 10:20:25 AM
Now that Squashfs with 4096 block size is the only kind of extension that will work as of TC 2.6,  here is a handy script that will convert your existing repo to save having to redownload everything.  Of course you need to have loop 254 free for this to work.  This will create a directory called newextensions that will contain the updated extensions.

EDIT:  Do not attempt to convert cramfs extensions as they will not work in all TC modes due to lack of time stamps.  If any of your extensions are cramfs, update them instead.

And this will best work with the util-linux-ng extension as needing the squashfs-tools-4.0.tcz and findutils

Code: [Select]
#!/bin/sh

cd `cat /opt/.tce_dir`

 [ -d newextensions ] || mkdir newextensions
for I in `ls | xargs file | grep "Squashfs" | cut -f1 -d:`; do
 
 mkdir "$I".pkg;
 losetup /dev/loop254 "$I"
 mount /dev/loop254 "$I".pkg;


mksquashfs "$I".pkg newextensions/"$I" -b 4096 &&
 umount "$I".pkg;
 losetup -d /dev/loop254


 rmdir "$I".pkg;
 done