Tiny Core Linux
Tiny Core Base => TCB Tips & Tricks => Topic started by: btheado on January 09, 2011, 02:21:52 PM
-
I stumbled across http://lxr.linux.no/linux+v2.6.37/Documentation/early-userspace/buffer-format.txt and discovered the initramfs buffer format supports multiple gzipped cpio archive files concatenated together.
This means a gzipped cpio archive can be 'overlaid' on top of another archive using nothing more than 'cat'. The alternative to using cat is to use gunzip, cpio, gzip and requires sudo to preserve the file permissions.
So tinycore.gz could be constructed from microcore.gz and the .gz files at http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/3.x/release/microcore/ like this:
cat microcore.gz Xlibs.gz Xprogs.gz Xvesa.gz > tinycore.gz
Maybe that is already the way tinycore.gz is created--I don't know. I just tried it and I either did something wrong or there is some other difference in tinycore.gz because it booted into X but there was no wbar and mouseclicks on the background didn't produce a menu.
I use this method to maintain an alternate version of tinycore.gz usable with colinux. As described at http://forum.tinycorelinux.net/index.php?topic=7348, I needed the kernel modules that match the colinux kernel and I needed a small patch to /usr/sbin/rebuildfstab in order to load extensions from colinux cobd devices.
I used sudo+cpio+gzip to create colinux-modules.gz containing the kernel modules and rebuildfstab patch. Now that I have that file, I can upgrade to new versions of tinycore using just 'cat' (assuming my rebuildfstab patch doesn't interfere with any future tinycore changes).
A small convenience, but I find it useful.
-
You are missing a window manager and icons, flwm_topside and wbar both which are currently extensions, although could be made into cpio .gzs.
-
I quite like this concept as currently flwm.tcz, flwm_topside.tcz, and wbar.tcz are really pseduo extensions as they do not install to /usr/local. They really are core compenents of Tiny Core and their use is really only viable via Micro Core.
By moving them to .gzs not only is the advantage of quick and easy tinycore.gz creation but more importantly all of Micro Core compenents will be archived together. See: http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/3.x/archive/?C=M;O=A
As it is now, by being extensions, only the latest version is available in the repository which might cause issues with prior versions of Micro Core, i.e., a mismatched set of compenents.
Of course there is absolutely no impact to Tiny Core users.
Look for this change in v3.5 of Micro Core.
-
Your previous post had me thinking too.
Actually, you would not need two distributions.
Tiny Core / Micro Core CD can simply be a boot option.
A nogz boot option could suppress loading of the included gz files from the CD.
Then you could include a boot menu on the CD that allows you choose TC or MC.
When creating a persistent tce directory, the user can include the X*.gz if desired,
and use or not use the nogz boot option.
They can still select Xvesa vs Xorg, and the window manager of choice.
-
Of course there is absolutely no impact to Tiny Core users.
Not sure if I entirely understood.
Is the case of Tiny Core users which would currently prefer to make use of flwm (as opposed to flwm_topside) included?
-
Tiny Core / Micro Core CD can simply be a boot option.
A nogz boot option could suppress loading of the included gz files from the CD.
Then you could include a boot menu on the CD that allows you choose TC or MC.
The concatenation of the gz files needs to have been done prior to boot (unless the bootloader has some way of concatenating or of passing in multiple initrd which the kernel would concatenate). Or are you thinking something different like changing MC init scripts to load the extra gz files?
-
What would the need or advantage of concatenating be - versus just passing multiple cpio archives to the kernel by the bootloader?
-
If the bootloader can pass multiple cpio archives, that is better, I think. Does that functionality exist? A quick search didn't give me the answer. I only found this link which made me think the functionality is not prevalent in bootloaders: https://bugs.launchpad.net/qemu/+bug/393569.
Once the archives are concatenated, the kernel can handle it fine--that's all I know.
-
If the bootloader can pass multiple cpio archives, that is better, I think. Does that functionality exist?
Yes. Not entirely sure if supported by every bootlader.
For example with sylinux family bootladers:
initrd=tinycore.gz,my_initramfs1.gz,my_initramfs2.gz
Works fine for me with linld as well, despite not having seen it documented anywhere.
Edit: Content of your link appears to refer exclusively to Qemu, so the issue might be rather about emulation than about real life circumstances.
-
It has nothing to do with the boot loader.
The TC/MC init scripts load any .gz files found in the tce directory.
http://wiki.tinycorelinux.com/Adding+a+Desktop+to+MicroCore
-
There is nothing new with the handling of .gzs.
It has and remains SOP to specify either in the boot loader for those that support it or as Gerald has posted in the tce/ directory. Not with the extensions in tce/optional.
The only difference is that I am moving the pseudo extensions, as I stated above, to gz.
If you want to run Tiny Core and desire flwm then simply place flwm.gz on the boot loader line or in the tce/ Although it would be more efficient to use Micro Core and gzs.
-
That will eliminate the slightly troublesome problem of tce-update downdating wbar.tcz on the release candidates.
-
After having time to reflect on Gerald's suggestion for a single ISO distribution, this is what I thinking:
Extend the toolkit approach by:
1. Eliminate a prebuilt tinycore.gz
Only the.gz, as such is not needed. However, if desired, it can easily be rebuilt by using 'cat' as shown above in this thread, or one can unpack the cpio archive(s) and repack as is known from Wiki docs on remastering.
2. Continue to offer tinycore.iso
TinyCore the ISO will continue to be offered and will boot into the usual X(vesa), flwm_topside, and custom fltk system programs. It will however be constructed from microcore.gz, Xlibs.gz, Xvesa.gz, Xprogs.gz, flwm_topside,gz, and wbar.gz
3. Eliminate the microcore.iso. Instead add the nogz boot code for booting the tinycore.iso into microcore.
Benefits from the above
* More modular extends the philosophy of the toolkit approach.
* More flexible as a toolkit, e.g., no need for Xvesa, don't include it, No need for wbar don't include it, etc...
* Possible smaller downloads. Only download the components that you need or have been upgraded.
* Easier to remaster as no need to unpack tinycore.gz to remove unneeded component(s).
* Better documentation possible with a single modular construction kit approach. As it is now, many ask for documentation specific for microcore. Having a singular approach should lessen duplication and confusion caused by such.
* No visible impact to those using or booting from the iso.
-
great !
TC is getting to be more coherent and mature.
Just one suggestion, why not go further, up to the 'core' of the concept of tc as a toolkit, and gain more coherence and flexibility all the way, by splitting microcore, and create 'CLIprogs.gz' in the same spirit of 'Xprogs.gz'? (everything that is specific and tailor made for tc, like tc's utilities, scripts and so... ).
-
About the only command line progs in MC are busybox and the other programs necessary for the init scripts.
-
this is what I meant. what's made for tc {even the init script) and what is not.
busybox and all other libs and so apart.
At the first glimpse, my suggestion may sound like a total nonsense.
-
You cannot extract out the progrms that are needed to load the programs.
-
Robert, sounds like a good plan. I guess the boot time should not really suffer too much from having to uncompress multiple gz-files before extracting the cpio archives. Or have you already done a comparison? If it makes a difference there is always the option to remaster again an "old-style tinycore.gz".
One thing I'm personally not sure about is the name of the boot code: Whilst "nogz" is short and I'm always in favour of typing fewer keystrokes that I could get wrong, the name is maybe not a good representation of what it means. Sure those are gzip-ed cpio archives, that form together an initrd. But the fact that gzip was used to create them does not really point towards what might be achieved by using this boot code. AFAIK there are alternative compression formats (e.g. bzip2 or LZMA) that could have been used. In light of this I wonder whether something like 'core', 'micro' or 'nox' (as in "no X server") are more to the point.
-
I agree micro is a better choice for the new boot code.
As I see it the benefits, as I have enumerated, out weight the slight performance hit on booting. And If, as you said, the user can always repack ala remastering and with just the components required.
-
I just used nogz as an example ( do not load any gz files ).
I think 'micro' or 'mc' are quite descriptive of the intent.
-
In my measurements, the effect of catted initramfs was 6% longer cpio extraction time.
-
The 6% represents the following average time:
extract single tinycore.gz cpio 1.234 seconds
extract multiple cpio 1.312 seconds
We are talking on average a difference of 78ms.
-
After having time to reflect on Gerald's suggestion for a single ISO distribution, this is what I thinking:
Extend the toolkit approach by:
1. Eliminate a prebuilt tinycore.gz
Only the.gz, as such is not needed. However, if desired, it can easily be rebuilt by using 'cat' as shown above in this thread, or one can unpack the cpio archive(s) and repack as is known from Wiki docs on remastering.
My two cents worth ...
While I for one do not want to stand in the way of progress ... I read the suggestion by Roberts to eliminate the pre-built tinycore.gz with a little apprehension. I never use an iso to boot up tc. I always drop the bzImage and the tinycore.gz into my boot area and use grub menu to configure a number of boot up modes. I like being able to drop in the latest tinycore.gz without changing anything else and so far I have not experienced problems with startup scripts breaking.
I hope that the tinycore.gz is not eliminated as it is what I use primarily and I will guess the majority of users are tinycore rather than microcore users, correct me if I am wrong. Having to download and cat a number of gz's to boot seems to make it unnecessarily complicated for users who like to keep things simple when one is not using an iso.
-
I appreciate the comments and concerns. While no decision is final until a final release is cut. Many times during release candidates I try new options. Most recently it was mfm which was replaced by fluff.
But to address the concern of having to cat several files...
No need to do such. No need to overly complicate grub with multiple .gzs either.
With the proposed 'more modular' method you would change grub entry from tinycore.gz to microcore.gz and then in your tce base directory include only those components needed, e.g. Xlibs.gz and Xprogs.gz.
If you do not use Xvesa, then no need to download it and load it upon each boot.
If you do not use flwm, then no need to download it and load it upon each boot.
If you do not use wbar then no need to download it and load it upon each boot.
This is a proposal. So comments and concerns are welcomed.
-
It has already been expressed to me the need to keep offering microcore.iso and I agree.
-
There would be an impact in 2 other aspects not mentioned so far:
1. Cases - besides from CD boot - in which a /tce would not have been used so far.
A particular case I can think of is netboot (using pxelinux).
My estimation is that in specific case it might be most suitable for users who have already a customized permanent routine to just append the extra cpio archives in the boot config file.
However, an adaption to reflect the changes in 'tc-terminal-server' might be in order for firsttimers.
2. For those who at current are booting tinycore with help of a DOS based linux bootloader like e.g. loadlin or linld, 'microcore.gz' - as opposed to 'tinycore.gz' - could not be directly used without adapting the filename to the 8.3 naming scheme.
Just my thoughts, by no means implying that the already mentioned benefits - first of all a more finetuned choice - could not outweigh some minor inconveniences.
-
Having to download and cat a number of gz's to boot seems to make it unnecessarily complicated for users who like to keep things simple when one is not using an iso.
An alternative approach could be to download the tinycore .iso, loopmount it and then copy the [selected] contained cpio archives and the kernel.
-
Those using PXE boot will not have to add the additional cpio archives to the boot config file if they are using an NFS based tce directory.
The init scripts will find them and load them just as they would on a HD or USB based boot.
I PXE boot microcore all the time, and I only have bzImage and microcore.gz in my boot config.
-
With the proposed 'more modular' method you would change grub entry from tinycore.gz to microcore.gz and then in your tce base directory include only those components needed, e.g. Xlibs.gz and Xprogs.gz.
If you do not use Xvesa, then no need to download it and load it upon each boot.
If you do not use flwm, then no need to download it and load it upon each boot.
If you do not use wbar then no need to download it and load it upon each boot.
Roberts: Thank you for responding to my query about making booting into tinycore.gz more complicated ... however, unless I misunderstand, I still fail to see how the 'more modular' method simplifies things for me as I do use Xvesa, flwm and wbar since they come default in tinycore ... and having to download these components separately and put them in my tce base directory ... seems to me like a step towards further complexity ... when all I have to do at present is download tinycore.gz and I get all I want in one download.
I have no stats ... but I would guess the vast majority of tinycore users are using the tinycore setup as it is in the default tinycore.gz/iso without getting rid of the core components included therein or adding new windows managers, and other system components except for user apps.
This 'modular approach' seems to me a neat trick that would appeal most of all to those who like playing around at more or less remixing their own tinycore.gz, starting with microcore.gz but is that the tinycore market target?
I think that tinycore.gz as the tc base cpio filesystem should continue to be offered and the modular options available to those who want to play around in that area.
My humble opinion.
-
when all I have to do at present is download tinycore.gz and I get all I want in one download.
Perhaps a base update script could be used to wget all the current files? That would make it pretty easy to update.
Are there any more files in the base that would be editted and backed up? (not including those that are meant to be backed up, ie bootlocal.sh)
-
Does Xvesa change release to release? No.
Does Xlibs change release to release? No.
Does flwm_topside change? Rarely.
Wbar? Rarely.
Why download over and over code that has not changed?
Modularity results in less to download faster to upgrade.
Modularity is great with the extensions but not with the base?
You don't download all your extensions over and over, only the ones that change.
So why not for the base?
The arguments against modularity in the base escape me.
-
The Ibiblio messup has me bored, so I decided to do sone testing.
I converted the Xvesa.tgz ( Yes i use it instead of the stock ), flwm_topside.tgz and wbar.tgz to .gz cpio archive files ( with proper /etc/sysconfig entries ).
Everything loads and runs fine.
-
Michael: What is the impact to you of putting the various .gz files in your /tce directory and using microcore.gz instead of tinycore.gz in grub? Is there a one-time or recurring (per version change) cost? I agree that maintaining tinycore.gz on the distribution_files (http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/3.x/release/distribution_files/) page is a good idea, at least for the lifetime of the 3.x series.
I cannot think of a recurring cost from this adjustment. If there are any, they should be noted and discussed. It will be necessary to do a one-time adjustment of a currently-installed grub-based installation or a one-time update of instructions for performing a grub-based installation, but those aren't recurring.
I like the increased transparency. That is, instead of using tinycore.gz you use a smaller base and add in specific files to get the features you want. I'd like to see microcore.gz broken down into a smaller "core + kernel modules" (or maybe further), but that's just me wanting to see how far the rabbit-hole goes. :)
-
Why download over and over code that has not changed?
Modularity results in less to download faster to upgrade.
Modularity is great with the extensions but not with the base?
....
The arguments against modularity in the base escape me.
Roberts: I misread your post the first time I responded but I think that I agree with your basic argument that removing parts of the base that could be made modules elsewhere is a good thing if it results in less downloading when updating or upgrading and seems like a good idea.
I remember the first time I ever tried TC and my surprise to see a 10MB distro boot up so quickly into a GUI WIMP environment in which I could then tack on the apps that I wanted, all so relatively easy without the need to repartition. In comparison to the work I put into Gentoo and LFS to get from console to GUI TC was a dream come true.
If you can remove the more stable parts in the base and put them elsewhere, while retaining the same user friendly characteristics of TC, I am all for it. If the ease of setup and the 're-occuring costs' of upgrading to a new build can be kept simple so that newcomers to TC can retain that sense of amazement at TC's awesome simplicity ... then having one and only one base instead of the current two (TC and MC) would definitely seem like a step in the right direction. Let's see what can be done ... :-)
-
(NOTE: sorry text formatting is not working.)
Roberts said: "The arguments against modularity in the base escape me".
I add: Me too and more!
Just a 'tiny' flashback.
1- Tinycore started by being tiny and modular, so faster to load and download.
2- then came Microcore, a natural step towards greater modularity, and more identifiable core.
3- then the concept of 'kit' came loudly to surface, and part of the logo.
4- then... this thread... Can 'Tinycore' reach the 'core' ?
I think we're getting closer to what 'Tinycore' had to be: a collection of bricks that can be glued together, easily, instantly. A morphing, elastic Linux distro. With a result that's only decided by the user-builder.
We all know that this approach is not new. Many had already done it in a way or another. But they failed somewhere. Let me rather say they didn't reach what TC can accomplish now: total modularity. From top to bottom.
It's not a market, segment or niche problem. ( anyway I think it's a hobby in the first place, and after all if an identifiable market is necessary to survive, by being a brick, glue and distro builder, TC can be served and satisfy nearly any niche/segment/market...).
That's why I urge you to don't hesitate longer to split anything that can be splitted and modularized.
I suggested that the base be splitted into what's TC and what's not.
Either it is a gz a tcz or whatever. The most important thing is that a 'brick' could be replaced by another one that can do similar or improved functionnality in the chain. ( Is it feasable to split that much? yes it is, ).
I see a re-construction of Mc as:
LnxBase+TCBase
I see a re-construction of Tc as:
LnxBase+TCBase+Xserver+XWm+Xprogs+...
or
Mc+Xserver+TCX+...
So Updating/changing or replacing one brick of the chain is straight and will bring instantly a different flavor to life.
Actually the only bricks that are not yet one are LnxBase and TCBase.
Finally, those who are worried about Tc/Mc iso, etc... I think these can be easily provided on a page on the web site, with 'the flavor of the day'!.
Regards
-
Before splitting up everything into atoms there's still work to do, i.e. tidy up the remaining kernel modules from core, so that only the absolutely necessary ones get included.
But it's difficult to know every kernel module's function, so all users should be encouraged to make recommendations.
I might be a bit tired these days, but what was the reason again for having to use cpios instead of extensions in the case of the window environment?
Could they be placed into /opt for a tinycore.iso ?
-
I like the increased transparency. That is, instead of using tinycore.gz you use a smaller base and add in specific files to get the features you want. I'd like to see microcore.gz broken down into a smaller "core + kernel modules" (or maybe further), but that's just me wanting to see how far the rabbit-hole goes. :)
This should be trivial to test if I understand you correctly. Just remaster microcore.gz and remove unwanted modules.
But beware, most people will want ahci, fat, ethernet card support in core...
-
I might be a bit tired these days, but what was the reason again for having to use cpios instead of extensions in the case of the window environment?
I agree, why can't wbar, flwm_topside, Xvesa, Xlibs, and Xprogs be tcz instead of cpio.gz?
If TC is going to be truely modular then these pieces should be extensions as well.
-
* Easier to remaster as no need to unpack tinycore.gz to remove unneeded component(s).
I don't understand, isn't there still a microcore.gz which you might want to repack? What exactly gets easier now?
It will however be constructed from microcore.gz, Xlibs.gz, Xvesa.gz, Xprogs.gz, flwm_topside,gz, and wbar.gz
You might ditch the confusing micro and call it core.gz
The strongest benefits in my view:
less confusion, aims are more clear.
-
I might be a bit tired these days, but what was the reason again for having to use cpios instead of extensions in the case of the window environment?
I agree, why can't wbar, flwm_topside, Xvesa, Xlibs, and Xprogs be tcz instead of cpio.gz?
If TC is going to be truely modular then these pieces should be extensions as well.
They are not extensions because they are release sensitive. Unlike true extensions which are not.
-
* Easier to remaster as no need to unpack tinycore.gz to remove unneeded component(s).
I don't understand, isn't there still a microcore.gz which you might want to repack? What exactly gets easier now?
It will however be constructed from microcore.gz, Xlibs.gz, Xvesa.gz, Xprogs.gz, flwm_topside,gz, and wbar.gz
You might ditch the confusing micro and call it core.gz
The strongest benefits in my view:
less confusion, aims are more clear.
Typically tinycore is the target for remastering to remove Xvesa, flwm, and or wbar.
-
From an other thread:
Since you would otherwise have a catch-22 loading the extension, add the scsi extension to a separate initrd:
I guess in order to modularize even more (who doesn't wish to have a self-updating "make localmodconfig" kernel) the most important thing we'd have to care about is this.
We have to simplify the process of adding/converting an extension to initrd.
-
gzs are loaded before extensions!
gzs are loaded to typical nix file system locations.
-
I might be a bit tired these days, but what was the reason again for having to use cpios instead of extensions in the case of the window environment?
I agree, why can't wbar, flwm_topside, Xvesa, Xlibs, and Xprogs be tcz instead of cpio.gz?
If TC is going to be truely modular then these pieces should be extensions as well.
They are not extensions because they are release sensitive. Unlike true extensions which are not.
I understand that Xprogs, Xvesa, and Xlibs would be release sensitive, but how are wbar and flwm_topside? Also, if some of these (perhaps not all) are moved to extensions then they can be updated as an extension rather then having to wait for a new final TC release. Also this may be a goal for 4.x rather then 3.x.
-
(NOTE: sorry text formatting is not working.)
Roberts said: "The arguments against modularity in the base escape me".
I add: Me too and more!
Just a 'tiny' flashback.
1- Tinycore started by being tiny and modular, so faster to load and download.
2- then came Microcore, a natural step towards greater modularity, and more identifiable core.
3- then the concept of 'kit' came loudly to surface, and part of the logo.
4- then... this thread... Can 'Tinycore' reach the 'core' ?
I think we're getting closer to what 'Tinycore' had to be: a collection of bricks that can be glued together, easily, instantly. A morphing, elastic Linux distro. With a result that's only decided by the user-builder.
We all know that this approach is not new. Many had already done it in a way or another. But they failed somewhere. Let me rather say they didn't reach what TC can accomplish now: total modularity. From top to bottom.
It's not a market, segment or niche problem. ( anyway I think it's a hobby in the first place, and after all if an identifiable market is necessary to survive, by being a brick, glue and distro builder, TC can be served and satisfy nearly any niche/segment/market...).
That's why I urge you to don't hesitate longer to split anything that can be splitted and modularized.
I suggested that the base be splitted into what's TC and what's not.
Either it is a gz a tcz or whatever. The most important thing is that a 'brick' could be replaced by another one that can do similar or improved functionnality in the chain. ( Is it feasable to split that much? yes it is, ).
I see a re-construction of Mc as:
LnxBase+TCBase
I see a re-construction of Tc as:
LnxBase+TCBase+Xserver+XWm+Xprogs+...
or
Mc+Xserver+TCX+...
So Updating/changing or replacing one brick of the chain is straight and will bring instantly a different flavor to life.
Actually the only bricks that are not yet one are LnxBase and TCBase.
Finally, those who are worried about Tc/Mc iso, etc... I think these can be easily provided on a page on the web site, with 'the flavor of the day'!.
Regards
I am fine with futher split. But could be debatable on which is tcbase. Perhaps you meant extension related? Otherwise base system would become unusable. And finally how useful is Core without extensions?
-
I understand that Xprogs, Xvesa, and Xlibs would be release sensitive, but how are wbar and flwm_topside? Also, if some of these (perhaps not all) are moved to extensions then they can be updated as an extension rather then having to wait for a new final TC release. Also this may be a goal for 4.x rather then 3.x.
That is precisely the problem.
When testing release candidates, tce-update keeps updating wbar to the last release, breaking the release candidate.
Since wbar is a part of TC core, it needs to NOT be an onboot extension.
The wbar.tcz extension can still exist for people using alternate window managers that still want wbar.
-
What if we just call that base wbar extension different on each update?
-
We? Oh please, a .gz together with the release makes it clear, simple and archive able.
Placing .gz in /tce directory is also trivial for the user.
-
I was Just thinking aloud. But I think I don't see some of the complications you guys do. Perhaps because I always used microcore instead of tinycore.
-
I like what I read.
Extending this concept, perhaps all of the supported wms could become .gzs as well, not only flwm. Then you place your wm of choice in along with the other bits. So onboot could be obsoleted this way - anything onboot could be converted to gz.
Would such a process be faster than loading extensions onboot?
A base system script for converting onboot .tczs to .gz could do the job. This would eliminate the necessity of having two sets of extensions in the repo.
-
I don't see why this should be faster? Faster in which way?
-
I wouldn't know, I am asking. Would loading a series of gz at boot be faster than loading those extensions a bit later as onboot loop mounts?
-
You're of course free to try it, but I don't expect any great outcome.
-
I really like the ideas and would love to see this...
But one thing leaves me troubled. The more I think about this whole difference in relationship "Xvesa/Xorg gz/tcz Xprogs/included Xlibs/tc base" I don't find it very clear. Of course mc/tc should be a toolkit but what about simplicity here? A lucid simple solution for everyone to understand would be:
mc + Xvesa(.gz|.tcz) + wm/wbar stuff = Tinycore desktop variant 1
mc + Xorg(.gz|.tcz) + wm/wbar stuff = Tinycore desktop variant 2
Right now it would be something like (if I'm correct):
mc + Xvesa.gz, Xlibs.gz, Xprogs.gz in ../tce + maybe Xorg-7.5-lib.tcz in ../tce/optional + wm/wbar stuff = Tinycore desktop variant 1
mc + Xlibs.gz in ../tce + Xorg-7.5.tcz, Xorg-7.5-lib.tcz, ... in ../tce/optional + wm/wbar stuff = Tinycore desktop variant 2
IMHO that's too complicated.
I'm not saying that I can provide a better solution here, but graphical user interface "bases" like "minimal Xorg with libs and progs" or "Xvesa with libs and progs" could be each offered in a single file instead of ripping it apart since they are mutually exclusive when it comes to running. Of course that would mean redundancies and maybe bigger files (redundant libs etc.), but it would also produce benefits by avoiding problems (like forum entries: "libX... not found, what should I do?"). I'm guessing Xvesa.gz would be about 3-4 MB including Xprogs.gz and Xlibs.gz, Xorg.gz would be about 8-10 MB (only vesa driver, sw renderer, but also Xlibs.gz included, etc.). Xvesa.gz is not extendable either way, Xorg.gz would be extendable through repo.
-
What's the minimum set of files to get tce-load up and working? Everything else can go in a tcz.
(ahem) it seems to me.
-
that's a question of design... you don't need a gui to use tce-load, microcore alone is enough, but tinycore should be usable by people without shell skills too. so an initial small desktop environment for tinycore is definitely better than leaving the (unexperienced) user with a prompt.
-
But beware, most people will want ahci, fat, ethernet card support in core...
Oh really?
Not recalling what 'ahci' would be, I had to look the term up.
I have never seen that module being loaded in TC, so I wonder, could there be any benefits of attempting to manually load it?
(By the way, current ethernet support in base is limited to NICs which don't require firmware.)
-
But beware, most people will want ahci, fat, ethernet card support in core...
Oh really?
Not recalling what 'ahci' would be, I had to look the term up.
I have never seen that module being loaded in TC, so I wonder, could there be any benefits of attempting to manually load it?
(By the way, current ethernet support in base is limited to NICs which don't require firmware.)
You're right, it was a very bad guess :D
-
I like what I read.
Extending this concept, perhaps all of the supported wms could become .gzs as well, not only flwm. Then you place your wm of choice in along with the other bits. So onboot could be obsoleted this way - anything onboot could be converted to gz.
Would such a process be faster than loading extensions onboot?
A base system script for converting onboot .tczs to .gz could do the job. This would eliminate the necessity of having two sets of extensions in the repo.
I couldn't see the benefits, and had actually in mind that the exact opposite, i.e. having as much as possible mounted in squashed .tcz's rather than to extract them from cpio.gz archives, such could save quite a bit of memory.
Also basically there already exists a method of 'copy to fs' to use .tcz extensions but leading to same result as when using cpio archives.
My expectation would be that for the same data to be gunzip'ed from cpio archives at boot time it might take significantly longer than for just mounting squashfs.
Influence on boot time might highly depend on hardware (CPU and storage medium).
-
Why not just try it? I'll see if I have the time later today.
-
I just saw that ahci is not a module. I have to use it with my SATA controller.
-
Why not just try it? I'll see if I have the time later today.
Personally I would not have any reason to try, because whatever the differences in boot time possibly could be could never outweigh the priority of the criterium of memory usage.
Now if it is just for the hell of benchmarking per se, have fun ;)
-
I just saw that ahci is not a module. I have to use it with my SATA controller.
Phew, that would explain why it did not ring a bell here ;D
-
Tinypoodle is right. .gz will consume more memory than loop mounted .tcz.
Some thoughts:
Instead of using .gz files to update mc to tc, they should be made into tcz extensions. ( Poodles suggestion )
The .gz loads before .tgz issue could be resolved with an xincludes.lst similar to onboot.lst if necessary.
This optimizes for memory at the expense of the flexibility of including the .gz files in the initrd definition in the boot loader.
Is this a concern? Do we need X loaded with the kernel? I don't think so. Tc-config should be able to handle loading X.
-
But you yourself said earlier:
Since wbar is a part of TC core, it needs to NOT be an onboot extension.
With that logic: Since Xorg is a part of TC core, it needs to NOT be an onboot extension.
And well, roberts seems to have some other reasons I guess.
I would like to hear any reason why e.g. X is release sensitive. What do you even mean with "release sensitive"?
-
Tinypoodle is right. .gz will consume more memory than loop mounted .tcz.
Some thoughts:
Instead of using .gz files to update mc to tc, they should be made into tcz extensions. ( Poodles suggestion )
The .gz loads before .tgz issue could be resolved with an xincludes.lst similar to onboot.lst if necessary.
This optimizes for memory
The way you put it makes it somehow like a choice of the lesser evil; the one at the expense of the other.
Perhaps a way could be found so that needn't be so.
This may sound odd to some but...
What if taking one of those extensions in question and placing them into /opt/tce/optional into a gzip'ed cpio archive?
With the status quo, as an example, say if a user prefers the newer Xvesa.tcz version and one of the former default wm's (jwm.tcz, flwm.tcz [original]), it would not be anything uncommon to use "dynamic remastering" (see wiki) and load these extensions from an additional cpio archive.
In such a way the memory optimization provided by squashfs could be gained without being
at the expense of the flexibility of including the .gz files in the initrd definition in the boot loader.
-
I am merely presenting views and observations from several different angles in order to draw out discussion.
.gz and .tcz both have advantages and disadvantages.
I'm all for making MC the core with TC an add-on.
The question is how.
Currently, the ugrade from MC to TC is a mixture of .gz and .tcz. I feel they should all be one or the other.
I personally don't like the wbar.tcz and flwm.tcz as the tce-update downgrades them for release candidates.
Perhaps the core X .tcz files should not be in optional so extension updates don't touch them.
This would allow Xlibs and Xprogs to be loop mounted .tcz files with the associated RAM savings.
-
What I like in poodles suggestion of creating a .gz of /opt extensions is that it would be compatible to my practice of putting Xorg in /opt
Only the user doesn't notice, because it's hidden in the .gz
Also for people wanting to remaster it serves as a great example for what opt might be for.
-
I am merely presenting views and observations from several different angles in order to draw out discussion.
.gz and .tcz both have advantages and disadvantages.
I'm all for making MC the core with TC an add-on.
The question is how.
Currently, the ugrade from MC to TC is a mixture of .gz and .tcz. I feel they should all be one or the other.
This would heavily favor tcz for the additional option of mounting and update (zsync see below).
I personally don't like the wbar.tcz and flwm.tcz as the tce-update downgrades them for release candidates.
IMHO Testing release candidates should not weight heavily in such decision.
Perhaps the core X .tcz files should not be in optional so extension updates don't touch them.
This would allow Xlibs and Xprogs to be loop mounted .tcz files with the associated RAM savings.
I would think that the "check for updates" process these tczs should be included. It would make for an easier system update. Again sans any consideration of release candidate testing.
tczs also have the update advantage of zsync which would make their download time much quicker to perform a system update.
-
I agree with all those observations.
-
To clarify, zsync works with any kinds of files, but for tcz ones there's already automation.
-
I had no problem converting Xlibs Xvesa and Xprogs from gz to tcz files, and loading them from /tce/optional using onboot.lst instead of /tce. I don't quite get in what way Xlibs and Xvesa are release dependant, but I see how Xprogs is (it has Fluff, for instance).
One thing I noticed while poking through microcore.gz is that a fair part of it relates to getting networking up (drivers for various network adapters, for instance). Would there be an issue with seperating out the network drivers into a tcz, and modprobing what's needed later?
-
Networking has to be included at least in one way or another, because internet connectivity is important for any other app you want to load with the appbrowser.
IIRC to use NFS tce dirs you need a working LAN before loading your extensions. I'm not sure if this is solvable by using the /opt/tce dir for these extensions.
-
Networking has to be included at least in one way or another, because internet connectivity is important for any other app you want to load with the appbrowser.
IIRC to use NFS tce dirs you need a working LAN before loading your extensions. I'm not sure if this is solvable by using the /opt/tce dir for these extensions.
IMHO debatable if modular NIC support should be included in base, while at the same time the firmware (for all those NICs which require such) is missing.
Loading firmware for NIC and/or loading wireless modules and wireless tools - optionally as onboot - in /opt/tce with an additional cpio archive works fine at current.
-
Well, you have to draw a line if you don't want to include *all* these fancy networking devices, right?
If I wanted to strip the base down I wouldn't start with NIC support, because current way just works so great. Put in your cd and cable, download your wireless extensions and get going...
In this regard base is very much base already, there's not much left to throw out besides X (at least measured in bytes).
-
"include *all* these fancy networking devices" was not at all what I had in mind, but e.g. a Intel Pro 100 NIC (not uncommon to be found in laptops): the e100 driver of size 15KB getting loaded from base, but not in a working state until firmware which has a size of 539B is installed by user.
My point regarding ethernet support in base was that it could be debatable to include either drivers and respective firmware, or none of the both.
-
Do other distributions include that firmware?
-
Yes, and AFAIK, the singling out of firmware from drivers is a recent kernel development, with older versions there was no seperate firmware required, just the driver.
-
As I am trying to test the microcore + Xgui, I ran into a problem. This probably will not be an issue if tinycore.gz is replaced with microcore.gz and the GUI module gz's overlaid ... but when I tried to put the Xlibs.gz and Xprogs.gz and a Xvesa.gz (from xvesa.tcz) in the tce directory to be loaded at microcore boot time ... I ran into the following problem: later when I rebooted tinycore.gz the gz's files in the tce directory were also loaded and the GUI was all messed up.
Perhaps I am doing something wrong in using the same tce directory to boot both microcore and tinycore with gz's inside ... but moving them out every time I want to boot tinycore is troublesome under the 3.x current configuration.
-
You have absolutely no reason to boot tinycore.gz with the X gzs in the tce directory.
That will cause problems.
Use a different tce directory.
-
If I understood right, in this way you loaded two different Xvesa versions from cpio archives to different paths in the initramfs, which is an unforeseen case.
Besides from using separate tce dirs as gerald_clark suggested, loading the cpio archives via bootparam initrd= from a different path (e.g. where tinycore.gz resp. microcore.gz are located) would give most control, also in specifying the order of loading.