Custom ISO9660 for developing TCE

[Roberto A. Foglietta]

Hi all,

 I developed suite [1] that is able to create customised TinyCore ISO9660 or an image suitable for qemu.
 At the moment two configuration are proposed by default 8MB and 22MB of TCE packages included.
 However, I want to change this in a more useful way. Three configuration:

 - container: a 24MB image that is able to connect to the network and run a SSH server for running/testing applications.
 - develbase: the size of the image is not defined yet but enough big to compile the kernel, busybox and glibc, at least.
 - develx: big enough to run and compile Xorg and every TCE included those using Xorg.

 The most interesting for me are the first two because I want to use TC as a container but I need a native development environment due to the fact that compiling with ubuntu links against the ubuntu libraries. Obviously a cross-compile environment is always possible but not as versatile like a contained one. Possibly the compiling environment will be able to run also in a chroot jail specifically designed for its purpose. However, having a developing image is the straight way.

 I am here to collect suggestion especially about what TCE including. Thank you, R.

 [1] https://github.com/robang74/tinycore-editor


 

[nick65go]

just my opinion: if YOU want to develop applications (from zero ground?) and to link them to different C libary (libc, musl, etc.), or even to different arhitecture (x86, x86_64, arm7, etc.), why do you invent again the wheel?
Did you have a look at the ALREADY build environment of toybox [1]? Did you ever read about toybox?

I mean it starts a qemu (in console), load a MINIMAL linux kernel 5.x, and a shell (600 KB, toybox, musl static linked). So now you have a shell and you can add/copy whatever libs and appls which OTHERS have already done for you; or build from here (from scratch) with whatever NATIVE (not-cross) compiler you want.

So, you laizy "borrow" already built basic libs (libc, zlib, whatever) and build on them your apps. Or even more lazy, just copy (already built for you) the proper libs +apps from TC, in the proper folders. Job done.

*[1] https://landley.net/toybox/downloads/binaries/mkroot/0.8.5/

[Roberto A. Foglietta]

Did you have a look at the ALREADY build environment of toybox [1]? Did you ever read about toybox?
I mean it starts a qemu (in console), load a MINIMAL linux kernel 5.x, and a shell (600 KB, toybox, musl static linked)

I know Rob Landley, it was the busybox maintainer of busybox before Denis Vlasenko and I know toybox. I did not know that toybox is used to start-up an environment. That's a new.

Anyway, the wheel is the same but the tyres are different. I am going to explain. I need a container and I decided to base it on TC so a generic/small linux-toybox-roofs should not fit the bill. The advantage of using TC is the TCE, there is a lot of application just ready to be used.

A developing environment running in qemu is not a new. To have one based on TC, I need to build a customised ISO/image. Finally, I can use the customised image to create a chroot jail in such a way I do not waste time in qemu boot time and slowliness. So, the ISO/image would be interesting for real-hardware and qemu-test but the chroot jail will speed up the developing process.

I hope, I have explained my idea better than before.

P.S.: why TC do not use toybox instead of busybox?

[nick65go]

...I need a container and I decided to base it on TC so a generic/small linux-toybox-roofs should not fit the bill. The advantage of using TC is the TCE, there is a lot of application just ready to be used.

A developing environment running in qemu is not a new. To have one based on TC, I need to build a customised ISO/image. Finally, I can use the customised image to create a chroot jail in such a way I do not waste time in qemu boot time and slowliness. So, the ISO/image would be interesting for real-hardware and qemu-test but the chroot jail will speed up the developing process.

Let me simplify for you the basics:
A. You "work" with containers -> no comments..

B: You "work" with CHROOT: so you limit yourself to the same CPU arhitecure: you can not chroot from a x86_x64 (or x86) into an arm7 CPU arhitecure. Hmm, lets not talk about security in chroot. Neither about proper isolation (libs leakage) between host and guest (chroot).

C: You "work" with QEMU: did you try what I "recommended", the previous link?

So are you "developing" appls? This means you have a lot of RAM and CPU power (to compile) and huge HDD space to store the intermediar/test result etc.
And you are worry that a qmenu will boot that minimal linux kernel (2-4 MB) in 2-3 seconds; but your work, compile, test, etc is for many hours /day? Nice joke!

And you are worry that qmenu is slow? Because you did not try the qemu KWM-module accelerator to have same native speed (98-99%) in guest VM as in the host? nice joke again! Did you measured it before you speak?

You want to build an ISO/cdrom image -> so you still need boot a kernel. But qemu can boot FASTER than an ISO, because "qemu kernel=.. initrd=..". Lets not talk about you wasting aditional time with "mkisofs ISO" versus 3 seconds qemu boot.


I think you did not undestand me. Let me explain it again more cleary.
You manage to be in a "buiding environment" (by using a container, or chroot, or qemu, whatever). And you are now in a shell (toybox, busybox, whatever) with a rootfs (/bin, /sbin, /usr..) on whatever medium (locally, or mouned qemu sda, or loop mounted).
From this moment, you are free to "borrow" any new shell and utils: a busybox you want (if toybox in not enough for you). Even copy / paste a busybox from tc (libc, dynamic) or from AlpineLinux (musl, static linked) etc.

Further from here, you download and mount the tcz you need (compile.tcz, gcc.tcz -> compiler).

The "TCE" means: kernel + modules + busybox (shell) + few GUI-FLTK_apps (mount, editor, cpanel etc). What do you talk about in TCE basic? Do you need the FLTK apps? And you can not simple copy/paste them in your new "buiding environment"? You need another kernel verison? Or another shell+rootFS, and you did not find any already made for you?The external tcz appls are just squshFS archives, which you mount into whatever rootFS you choose.

Summary: you could build any-kind of TC, using any kernel version you want, and any shell you want. But if you want lazy (already made) tczs - which are libc linked -they are free to download already from servers.

[nick65go]

Another crazy idea: tcz are squashFS archives (/usr/local/*), basically extracted (or mounted on /). Inside tcz there are executables (ELF linux files) and their dependency libraries.

Nothing keep you to take any package, lets say from Alpine Linux, and make a squash archive from it. So all your TCE (tc extensions) are now new TCZ but linked with musl. And of course the "basis"/core must have a shell (static, or musl libc), be it toybox or busybox. Even more, you do not need to compile anything! Think about it.

Cavets: the TC GUI progs will not work, because they are libc (ex: editor, cpanel) and also they call aplets or libs linked to libc (fltk is linked to libc). FYI: Alpine Linux has FLTK, flwm, wbar .

And you need to understand how dependency recursivity was done (in tc / alpine), because few of your-new-built-tczs could ask for other depencency (ex: libicu5 -> libicu7 for libreoffice).

So, we are back to the main CONCEPT: core/tinycore is a kit (aka tools) to build /develop a linux system.
TC is currently based on busybox, appls linked to libc, and use an old kernel version (5.10.3).
However "nothing" prevents you to build a similar kit, based on another shell (toybox), another basic libc (musl), another kernel (5.14.x) or other tcz appls from other repositories (Alpine linux, Debian ?).

So, Thank you TC Team for sharing your great ideas, about simplicity and modularity to have a linux system, using atomic utils like: sh / sed / awk / grep.

[andyj]

The most sincere way you can say thanks is to take some of the load off the few people who maintain most of the extensions. I took on the LAMP stack and some of the other basic servers because I needed them for the project I was working on then. Are there any subsystems extensions you could maintain for the benefit of everyone for whatever project has brought you here? Proposals which appear to have little benefit outside whatever you are working on come across as selfish. Even though I have been here only nine years I have seen this more than a few times where someone wants us to make TC what they need. Because TC is a toolkit, it should be generic enough for nearly everyone to start from so whatever changes you propose should keep that in mind.

[Roberto A. Foglietta]

And you are worry that a qmenu will boot that minimal linux kernel (2-4 MB) in 2-3 seconds; but your work, compile, test, etc is for many hours /day? Nice joke!

And you are worry that qmenu is slow? Because you did not try the qemu KWM-module accelerator to have same native speed (98-99%) in guest VM as in the host? nice joke again! Did you measured it before you speak?

Dear nick65go,

frankly speaking I did not appreciate your tone nor your attitude towards me.
IMHO, you did a lot of consideration that are pure speculations and suppositions.

First of all, I need to run qemu into a virtual box and this drops performance even using kvm and cpuhost flags. It is insane but the company do not tollerate dual boot laptop. Almost everyone here uses laptops and compilation do not take hours because all the job is about software and system integration not compiling. Yes, chroot is safe enough because we do need to work on different architecture than x86_64 and we do not need to protect ourselves by ourselves because we are just working in a virtual machine.

Long story in short: please, do not speculate about people doing.

I am happy to spend my holydays in doing things that other people can find useful/easy to use.
If you do not like what I am doing, do not use it.

[nick65go]

@andyj: If I am your target in your replay, then please let me clarify that:

1. I really value TC work. In the far-away past I also contributed a little (opera-next), seldom reporting bugs, helping other how / when I could; I even shared a scripts (acpid HP laptop), some firefox tips & tricks etc. So I tried to give back a little to community.

 2. Currently I can not take some of the load off the TC Team, because: my APU is week to compile, I use 10 h/day win10 as employed, and linux max. 2-4 h/day as hobby. I do not do linux projects. I do not have programming skills, (maybe just few ash scripts).  Takeing maintenamce of a tcz involves long term commitment time, responsability and availability, which I can not provide. So I avoid to be a burden /pain to TC.

3. I did not propose anything to be changed in TC. I just shared what I read and I think I undestand about TCE (extensions). It's (still) a free world / forum and I legally grab what I can from servers (TC / Alpine linux) and then play with my petty custom linux. I do not demand anything.

The subject is about "developing TCE" - Tiny Core Extensions. Developing TCZ does not necesary means compiling, for me. (ex: I proposed to upload gnumeric.tcz, from debian).

What is wrong to share the idea that in TC someone could load / use a a new shell, such as toybox.tcz (aside of busybox) similar to bash.tcz? Or that a firefox90.tcz (musl linked) can run in a TC (like chroot)? What I exposed is that the kernel version should not matter, neither the shell (we have busybox.aliases). Or the dependency libraries for tcz (if they run isolated /chroot / LD_LIB_PATH from host system) can be different from base /kit system.

[andyj]

I meant it more to anyone/everyone who comes to the forum with lists of changes that the maintainers need to make to TC. I can see as I read it again how my wording might have seemed like it was directed at someone specific. I apologize for that.

[nick65go]

Dear Roberto,

Please accept my appologies if my wording style looks offending to you.
When I do not have enough info then I assume/speculate some sane defaults.

IMHO virtualbox droped the performance, not the qemu; but case is closed.
If chroot is "good enough", then OK. But you did not said this in the topic title, that you want ISO/cdrom only for X86* and not for arm7.

Have a nice holidays.

[Roberto A. Foglietta]

Hi all,

TinyCore Editor version 0.4.4 runs in TinyCore.

https://github.com/robang74/tinycore-editor/archive/refs/tags/v0.4.4.tar.gz

So, you can compile busybox directly into TC, for example.

Choose "develop" in tinycore/tinycore.conf and follow the quick start in README.txt

Cheers,
-R

[nick65go]

@Roberto: I had a quick look over your scripts. I do not pretend that I understood everything, sorry.
One small suggestion: try to not be a burden (you or the users of your scripts) for the main TC server.

 Please download mirrors.tcz and see all available ftp and http servers, from where *.iso/*tcz*/root*/vmlinuz* could be downloaded. So maybe you can setup (as a better variable maybe) the current $tcrepo in your scripts.(ex:download $tcrepo/$distro/rootfs$ARCH.gz. The TEXT list of mirrors is searched for in "/opt/localmirrors" and "/usr/local/shared/mirrors".

[nick65go]

@Roberto: maybe is a personal taste, but you could offer a variable / parameter, for extra compression of root.gz.in https://github.com/robang74/tinycore-editor/blob/main/tinycore/rootfs.sh you chose -z3 (for 7z)

Code: [Select]

advdef -z3 rootfs.gztc uses -z4 (zopfli, shrink- insane). For your work-in-progress in OK, less time wasted; but good maybe for the final form of scripts. YMMV.

[Roberto A. Foglietta]

you chose -z3 (for 7z)

Code: [Select]

advdef -z3 rootfs.gztc uses -z4 (zopfli, shrink- insane). For your work-in-progress in OK, less time wasted; but good maybe for the final form of scripts. YMMV.

-z3: 1.2 seconds
-z4: 15 seconds

the final size is the same, no any advantage