squashfs zstd vs lz4

[CardealRusso]

Comp	Size	Ram	Decomp (Real)
zstd(128k)	119MB	391MB	0.37s
lz4(128k)	181MB	381MB	0.35s
gzip(128k)	131MB	379MB	0.32s
zstd(4k)	154MB	385MB	1.12s
gzip(4k)	160MB	374MB	1.15s

[Juanito]

squashfs-tools and deps updated in x86/x86_64 repos in case anybody wishes to test further.

[GNUser]

Hi, Juanito. I updated and ran into this:

Code: [Select]

$ mksquashfs --help
mksquashfs: error while loading shared libraries: liblz4.so.1: cannot open shared object file: No such file or directoryIt seems liblz4.tcz needs to be added to squashfs-tools.tcz.dep

P.S. I also noticed that liblz4.tcz.list, liblzma.tcz.list, etc. are empty.

[Juanito]

dep file adjusted - I'm not sure what happened there.

It seems:

Code: [Select]

for file in *.tcz; do unsquashfs -ll -d '' "$file" | grep -v '^d' | sed -e 's#.* /#/#' -e 's# -> .*##' -e 1,3d > "$file".list ; done..no longer works - *list files reposted.

[GNUser]

Hi, Juanito.

It seems:

Code: [Select]

for file in *.tcz; do unsquashfs -ll -d '' "$file" | grep -v '^d' | sed -e 's#.* /#/#' -e 's# -> .*##' -e 1,3d > "$file".list ; done..no longer works

I decided to investigate this.

With old version of squashfs-tools:

Code: [Select]

$ unsquashfs -ll -d '' liblz4.tcz
Parallel unsquashfs: Using 4 processors
3 inodes (29 blocks) to write

drwxr-xr-x tc/staff                 26 2023-06-26 13:29
drwxr-xr-x root/root                28 2023-06-26 13:28 /usr
drwxr-xr-x root/root                26 2023-06-26 13:30 /usr/local
drwxr-xr-x root/root                74 2023-06-26 13:30 /usr/local/lib
lrwxrwxrwx root/root                15 2023-06-26 11:10 /usr/local/lib/liblz4.so -> liblz4.so.1.9.4
lrwxrwxrwx root/root                15 2023-06-26 11:10 /usr/local/lib/liblz4.so.1 -> liblz4.so.1.9.4
-rwxr-xr-x root/root            108856 2023-06-26 13:30 /usr/local/lib/liblz4.so.1.9.4
With new version of squashfs-tools, notice that there is no 3-line header:

Code: [Select]

$ unsquashfs -ll -d '' liblz4.tcz
drwxr-xr-x tc/staff                 26 2023-06-26 13:29
drwxr-xr-x root/root                28 2023-06-26 13:28 /usr
drwxr-xr-x root/root                26 2023-06-26 13:30 /usr/local
drwxr-xr-x root/root                74 2023-06-26 13:30 /usr/local/lib
lrwxrwxrwx root/root                15 2023-06-26 11:10 /usr/local/lib/liblz4.so -> liblz4.so.1.9.4
lrwxrwxrwx root/root                15 2023-06-26 11:10 /usr/local/lib/liblz4.so.1 -> liblz4.so.1.9.4
-rwxr-xr-x root/root            108856 2023-06-26 13:30 /usr/local/lib/liblz4.so.1.9.4
So with new version of squashfs-tools we need to change the command that generates .list files from this:

Code: [Select]

for file in *.tcz; do unsquashfs -ll -d '' "$file" | grep -v '^d' | sed -e 's#.* /#/#' -e 's# -> .*##' -e 1,3d > "$file".list ; doneTo this:

Code: [Select]

for file in *.tcz; do unsquashfs -ll -d '' "$file" | grep -v '^d' | sed -e 's#.* /#/#' -e 's# -> .*##' > "$file".list ; done
P.S. I just took a peek at submitqc to see whether it is affected by this issue. It is not affected because its strategy to create the .list file does not use unsquashfs -ll:

Code: [Select]

find -not -type d | sort | sed 's/^\.//' > ${NEWLIST}


[Juanito]

Thanks for that 

This also works:

Code: [Select]

for file in *.tcz; do unsquashfs -lc "$file" > "$file".list ; done && sed -i 's/squashfs-root//g' *tcz.list

[GNUser]

This also works:

Code: [Select]

for file in *.tcz; do unsquashfs -lc "$file" > "$file".list ; done && sed -i 's/squashfs-root//g' *tcz.list

It sure does. And is a lot easier to understand.

Thanks for that 

You're welcome

[Rich]

Hi Juanito

This also works:

Code: [Select]

for file in *.tcz; do unsquashfs -lc "$file" > "$file".list ; done && sed -i 's/squashfs-root//g' *tcz.list

That will also include empty directories.

This will work with both versions of  unsquashfs:

Code: [Select]

tc@E310:~$ unsquashfs -ll -d '' /mnt/sda1/tce/optional/sbc.tcz | grep -v '^d' | grep '/' | sed -e 's#.* /#/#' -e 's# -> .*##'
/usr/local/bin/sbcdec
/usr/local/bin/sbcenc
/usr/local/bin/sbcinfo
/usr/local/lib/libsbc.so
/usr/local/lib/libsbc.so.1
/usr/local/lib/libsbc.so.1.1.0
tc@E310:~$ This was run with the old version of  unsquashfs.