Man pages compressed?

[gutmensch]

Heya,

since the man viewer is able to decompress gzip'ed man pages on the fly (at least it should be if configured correctly, says the Makefile), I'd suggest putting man pages into extensions only compressed and with suffix .gz like in other distros. What do you think? Maybe even the extension audit script could check for uncompressed man pages...

Best regards,
Robert

[Jason W]

Perhaps a good point.  To do that and preserve the symlinks that are broken by the renaming to .gz, Slackware uses this routine in the Slackbuild.  I can also add a "check and repack if not gzipped" function in the audit routine.  I will add and test it before using it on live packages.

Code: [Select]

if [ -d $PKG/usr/man ]; then
  ( cd $PKG/usr/man
    for manpagedir in $(find . -type d -name "man*") ; do
      ( cd $manpagedir
        for eachpage in $( find . -type l -maxdepth 1) ; do
          ln -s $( readlink $eachpage ).gz $eachpage.gz
          rm $eachpage
        done
        gzip -9 *.?
      )
    done
  )
fi



[tinypoodle]

Just an observation:

busybox gzip does not appear to have a "-9" option.
However, when specifying this option from command line, it simply gets ignored, it seems.

[Jason W]

Actually, tonight I will test and see what difference it makes to gzip the individual files versus just letting the squashfs compression take care of it. 

But gzipping the files themselves will help those who use the copy-to-filesystem loading as opposed to mounted mode.

[Jason W]

Oh, and I would likely use gnu gzip in the audit routine to use the -9 option, but I will see what difference it makes too.

[Jason W]

I made a script to compress man pages,  I will post it here for testing for now and I will upload it to programming and scripting after any needed changes are made.  I can also plug this into the audit script when checking.  Basically it compresses man pages, fixes the symlinks, and rebuilds the package in the case uncompressed man pages are found.

Also, the man command supports bzip2, so that is what is used here.

I see there are various places that man directories are found in extensions, and I think we should standardize on one, and /usr/local/man would likely be the best.

##Script moved to programming and scripting - 11.4.10

[curaga]

Was it smaller than just bare squashfs?

[Jason W]

I see a space savings of about 10% in the tests I ran for the size of the actual tcz.  But there is an issue with hard links that exist in some doc extensions, I will have to work that in.

But for copy-2-fs mode the space savings would be a much higher margin.  Since man pages take up only a small part of the average person's system, we will need to ponder if the extra trouble is worth it, but it at least is worth pondering.

[jur]

As a n00b I would massively appreciate the integration of the man pages into apps. Often I have too google around for info and more often than not i run across, 'man xyz' and lament I can't do it.

[gerald_clark]

There are many internet man pages readable from TCL.
EX:
http://www.manpagez.com/man/1/ls/

[tinypoodle]

As a n00b I would massively appreciate the integration of the man pages into apps. Often I have too google around for info and more often than not i run across, 'man xyz' and lament I can't do it.

http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/3.x/tcz/man-pages.tcz.info

[jur]

I am aware of these and others but specific apps man pages is what I was referring to.

[Jason W]

check out the  -doc.tcz extensions, that is where they are normally found.

I have an aversion to tampering with files in extensions submitted that is not absolutely necessary to, so I am still thinking if this is a road we want to go down just to gain 10% reduction in extension size of doc extensions.  And with the hard link situation I ran into, it may present a host of potential issues that can creep up.  Still pondering.

[gutmensch]

I see the compressing of the man pages just as one step and an objective to obtain a clear and optimal file structure in extensions. There are so many extensions, which don't have a -dev although they should have (e.g. jack), some with really big man pages (like lvm), some with libs in them, which should be outsourced, etc.

It would even be possible to patch the man page viewer to use xz, which could reduce the size even more and since it's in base, why not make use of it?

So in the tradition of trying to "make it better" than other distros I'd of course prefer xz for man pages ;-)

[Jason W]

Ok, I have dealt with the hard link situation by converting them to softlinks to allow compression of the files.  Tcl-doc was originally 1.187mb and after compressing man pages with gzip it is not 1015mb.  A decent reduction in tcz size, 85% of the original.  Also, for those who copy-2-fs, the space taken up by the extension in RAM is 3.2mb before compression, and 1.4mb after, around 44% of the original space taken up.  However, I only noticed about a 1 or 2 percent space savings by using bzip2 instead of gzip, which is consistent with other reports I have read.

Most of the largest -doc extensions are made up of html pages, which we cannot compress.  And man pages as well as html abound on the internet.  But for sake of principal if it can be done simply and reliably, I think we can compress the man pages as a general rule.  I would like for it to be used or tested a bit before trying to do any wholesale conversion of doc extensions.  For now, those who are interested can use the script and check the result.  

I have edited the above script with the changes, works with gzip or bzip2 by changing the variable, but may as well use gzip as no real benefit to bzip2.