Suggestion to embrace the yaml format for .info files

[DukeNukem]

Hi everyone,

Switching to YAML for .info files is an interesting idea. While the current format works well for its intended purpose, YAML could bring more consistency and make automated parsing easier for those who need it.

At the same time, it’s worth remembering that .info files are primarily for users installing extensions, so keeping things straightforward and easy to read should remain a priority. If a change is made, it would make sense to approach it gradually, allowing for compatibility with the existing format and ensuring smooth adoption.

[Sashank999]

Hello.

I'm not trying to add more fuel to this argument but I'm in support of using an actual, structured format for creating .tcz.info files. The improvements that immediately come to my mind:

- Easier evaluation of extension submission with submitqc - we can more easily check for missing fields and also add submission details. This can make the submission of updates for existing extensions easier (I myself can write a submitqc patch for fetching .tcz.{info, dep} files for a previous version and placing in the current folder so that you don't need to have them in place).
- Ability to change the rendering format of .info files in an easier way - Apps and tce currently display whatever text the files contains on the server. We can change the rendering format as we can separate the fields and values with proper separators. We can have the changelog listed in chronological order or have the latest change first.

As for the format itself, any format (except XML, please) does the work. CSV, JSON (the GOAT), JSONC, YAML and TOML are a few options that I currently think would work fine.

Moreover, with the alpha testing of TCL v16, I don't think there is no better time for thinking of such a change, if it is actually necessary.


P. S. I also sometimes think of why tce and tce-load are written as bash scripts and not in C (or Rust, for those rustlings). Perhaps there is some reason, such as having editable scripts even after OS boot. In that case, I think we can think of switching to Lua, which is fast and editable. But the main reason I think of switching is performance. Maybe it is a debate for another major release of TCL.

[nick65go]

@Sashank999: Not trying to be the devil's advocate, but regarding tce-load & co, I wish you good-luck... because you will not have 

The basic commands (like tce-load) should work even without extensions loaded, aka in virgin base. And this base does NOT have lua, neither rust etc. (even if LUA is small size). The actual AWK engine is inside busy-box so it is C compiled.

A bump in speed is if using C (compiled) for any/all ash scripts, but curaga does not want it.
Plus is easy to edit/change them on the fly (debugging).
Anyway, any perceived increase in speed MUST first be measured, and if is significant THEN estimate how often is used. (ex: an increase of 50% --keep dreaming -- used only 10% of time is 5%, not impressive). YMMV.

[mocore]

Ability to change the rendering format of .info files in an easier way

see also : "(tcz) info 2 JSON (the GOAT)" @ https://forum.tinycorelinux.net/index.php/topic,27554.0.html
*please test  

I don't think there is no better time for thinking

thunk is cheep ... show me the code ( also the data thnx )


*8 bit mario noises*

I got special powers (wow)
From eating precious flowers

* https://learnxinyminutes.com/awk/

[GNUser]

The basic commands (like tce-load) should work even without extensions loaded, aka in virgin base. And this base does NOT have lua

The lua interpreter is small (around 200 kB in size), so theoretically could be included in base system.

That being said, the TCL developers are (fortunately) very conservative. I wish I could bet that there will be no big changes to TCL's base system or package manager (not even to .info file format). Betting on this would be an easy way to get rich

[mocore]

The lua interpreter is small (around 200 kB in size), so theoretically could be included in base system.

perhaps with luainkernel/lunatik   "info 2 JSON" could be a module 

[Sashank999]

The lua interpreter is small (around 200 kB in size), so theoretically could be included in base system.

That being said, the TCL developers are (fortunately) very conservative. I wish I could bet that there will be no big changes to TCL's base system or package manager (not even to .info file format). Betting on this would be an easy way to get rich

Being conservative has its own challenges. Being resistant to change will lead not only to stability, but to stagnation as well. I think I read somewhere that Robert Shingledecker himself started TCL because the Damn Small Linux maintainer rejected this idea.

Now, I do not want to start a new distro. I would like to see some improvements in TCL distro itself. Maybe packaging format won't change, but IMO we should be having more speed during startup. And for an actually lightweight distro like TCL, the first thing that any user would expect is fast startup. Creating all the symlinks through shell script makes it slow. I personally have the following items onload:

Code: [Select]

Xprogs.tcz
firmware-atheros.tcz
wifi.tcz
xf86-input-libinput.tcz
i2c-6.6.8-tinycore64.tcz
xf86-video-intel.tcz
icewm.tcz
Xorg-7.7-bin.tcz
screen.tcz
firmware-rtl_nic.tcz
firefox.tcz
gnutls38.tcz
elfutils.tcz
libcamera.tcz
alsa-config.tcz
pulseaudio.tcz
xorg-server.tcz
mc.tcz
mylocale.tcz
rxvt.tcz
bash-completion.tcz
vim.tcz
terminus-fonts.tcz
xclip.tcz
ClipIt.tcz
adwaita-icon-theme.tcz
squashfs-tools.tcz
nano.tcz
simple-mtpfs.tcz
notosanstelugu-fonts-ttf.tcz
htop.tcz
man-pages.tcz
git.tcz
icu74.tcz
libsecret.tcz
gnome-keyring.tcz
python3.9.tcz
tk8.6.tcz
brightnessctl.tcz
provides.tcz
notocoloremoji-fonts-ttf.tcz
gnome-terminal.tcz
busybox_extras.tcz
pcmanfm.tcz
libportal-gtk3.tcz
file-roller.tcz
notosansdevanagari-fonts-ttf.tcz
notosans-fonts-ttf.tcz
and it takes around 3 minutes only for my setup to load completely (please suggest any changes that you have if I have done something wrong or suboptimal in my onboot.lst). I want this to change, which is why I want to find some suggestions to optimize it.

[Juanito]

By “onload” do you mean onboot or ondemand?

Perhaps the first place to start would be to remove the recursive deps from your onboot list.

[Sashank999]

By “onload” do you mean onboot or ondemand?

Perhaps the first place to start would be to remove the recursive deps from your onboot list.

I meant onboot. Sorry. I will remove the recursive dependencies, measure execution time and post it here.

Edit: Forgot the mention my laptop's specifications: Intel Core i3 6th generation, 4GB RAM, 512GB SSD, Ethernet and Wi-Fi. I have my bootloader on a USB stick from which I multiboot into TCL. I use a modified Corepure64 initrd, the modification being the execution of an image viewer to show a splash image during boot, hiding other booting logs.

[curaga]

Gnome stuff tends to be heavy, though you don't use full gnome but some apps (at least for full gnome it would be better to use another distro). If you boot with the showapps bootcode (and disable the splash temporarily), you can see what takes the most time.

It may not be anything to do with shell, it may be your usb stick is slow, for example.

[Juanito]

There's also the question of if you're using a backup and, if so, how big it is.

[mocore]

Maybe packaging format won't change, but IMO we should be having more speed during startup.
And for an actually lightweight distro like TCL, the first thing that any user would expect is fast startup.
Creating all the symlinks through shell script makes it slow.
...
and it takes around 3 minutes only for my setup to load completely (please suggest any changes that you have if I have done something wrong or suboptimal in my onboot.lst). I want this to change, which is why I want to find some suggestions to optimize it.

this might be relevant ,
though it appears it would require some testing  ( eg trying to combine different groups of tcz to se what works with your list of tcz / system ect )

"Combining Extensions into Large Extensions" @ https://forum.tinycorelinux.net/index.php/topic,3757.msg19679.html#msg19679

afaik
 a similar method is used by dcore to create "scm packages" (of many Debian packages )

fwiw only i3 i have (not shore what gen) used was laggy / slow  when copying/moving data
( .. after booting to desktop with file-manager , for every distro i tested , im assuming something *by design* with architecture & cache ?? )

[GNUser]

it takes around 3 minutes only for my setup to load completely

Hi Sashank999. A 3-minute boot time suggests there's something unusual about your setup. I have a lot of extensions in my onboot.lst, my laptop is an old clunker (X230 ThinkPad made in 2012), and my boot time is 24 seconds.

Juanito suggested checking the size of your backup (mydata.tgz in tce directory). I do remember that my boot time improved when I moved large files out of mydata.tgz into my persistent directory (/opt).

[Sashank999]

Hello all,
I'm really sorry to disappoint you all by solving this mystery myself. Turns out, I wrote `sudo wifi.sh -a` in my `/opt/bootsync.sh` and as I'm in a different location, it did not detect my home Wi-Fi network which was causing too much variable amounts of delay during boot.
I made it a background job in bootsync.sh and now my system loads up in 29.45 seconds, averaged over 3 boots.

Gnome stuff tends to be heavy, though you don't use full gnome but some apps (at least for full gnome it would be better to use another distro). If you boot with the showapps bootcode (and disable the splash temporarily), you can see what takes the most time.

It may not be anything to do with shell, it may be your usb stick is slow, for example.

Hello curaga.
Only my bootloader (Ventoy, uses GRUB 2 internally) is on the USB stick. TCL is installed on a 10 GiB partition on my laptop's SSD, with persistent /home and /opt.
I also have another boot option that uses the stock Corepure64 initrd, which doesn't have the splash image. I used it and it looks like no tcz is causing too much delay during load.
I use Gnome keyring in order to secure my Github personal access token that I use to push commits to Github for the Core-scripts and fltk_projects repositories. I use Gnome terminal as it directly copies to the clipboard that I can use with Firefox. IIRC, there are 2 clipboards in X and Gnome terminal copies my mouse selections to the clipboard that Firefox also uses and also pastes from it. I found no other alternative so I had to live with it.

There's also the question of if you're using a backup and, if so, how big it is.

Hello Juanito.
Yes, I am using a backup and it is very small. I only included the /opt directory for backup of startup scripts, and its size is
`0.01 MB (12015 bytes)`. I don't think that will affect the boot time at all.

this might be relevant ,
though it appears it would require some testing  ( eg trying to combine different groups of tcz to se what works with your list of tcz / system ect )

"Combining Extensions into Large Extensions" @ https://forum.tinycorelinux.net/index.php/topic,3757.msg19679.html#msg19679

Hello mocore.
I would rather not do that because if any one dependency is updated in the TCZ repo, I would have to restart the process. I let tce-audit do that work for me.

Hi Sashank999. A 3-minute boot time suggests there's something unusual about your setup. I have a lot of extensions in my onboot.lst, my laptop is an old clunker (X230 ThinkPad made in 2012), and my boot time is 24 seconds.

Juanito suggested checking the size of your backup (mydata.tgz in tce directory). I do remember that my boot time improved when I moved large files out of mydata.tgz into my persistent directory (/opt).

Hello GNUser.
I already did that as suggested on this forums by some user on an old thread of mine. Now my backup size is small enough that it does not have any noticeable impact on my boot time.

I thank you all for your time.

[gadget42]

@Sashank999 thank you for sharing your discoveries!