Tiny Core Linux
Tiny Core Base => TCB Talk => Topic started by: Vaguiner on December 28, 2023, 12:42:48 PM
-
Hello. I would like to make a suggestion to start using the yaml format in .info. I personally have no experience with yaml, but I've found that it's the most similar to the current non-standard format and the change would be insignificant.
I'll use http://tinycorelinux.net/14.x/x86_64/tcz/cairo-dev.tcz.info as an example.
The only change would be the description and changelog, in case there are multiple lines.
There are two options:
1. Use a multi-line variable.
It would look like this:
Change-log:|
2012/11/09 first version
2012/11/16 updated 1.10.2 -> 1.12.8
2013/11/05 updated 1.12.8 -> 1.12.16
Or array:
Change-log:
- 2012/11/09 first version
- 2012/11/16 updated 1.10.2 -> 1.12.8
- 2013/11/05 updated 1.12.8 -> 1.12.16
Apparently using spaces or tabs makes no difference, as long as only one is used in a multiline/array variable (using tabs and spaces in the same variable would probably not be correct).
https://codebeautify.org/yaml-parser-online
-
the (current) tcz.info format to meh appears more or less
like the http header format *
- https://www.rfc-editor.org/rfc/rfc9110.html#name-message-context
with the exception of the multilline change-log field you mention
suggestion to start using the yaml format in .info.
I personally have no experience with yaml, but I've found that it's the most similar to the current non-standard format
and the change would be insignificant.
>the change would be insignificant.
for who ? would it be so !?? i wander
if this is the case what advantage would this bring ?
or rather with what(language) and to what purpose are you trying to parse .info files ?
and
consider using awk ;) :P
i cant read the example info with out thinking up some methods to try and handle the multi-line Change-log field case tbh ???
or perhaps for this case its possible * to modify on the fly *
eg include ` sed -e's/Change-log:/Change-log:|/g ` this pipeline before processing tcz..info field ?
*( http header format ftr is also much like tiddlywiki's .tid files https://tiddlywiki.com/static/TiddlerFiles.html )
-
>the change would be insignificant.
for who ? would it be so !?? i wander
Hello. It wouldn't be necessary to adapt all the existing .info files, just the ones that have been updated.
if this is the case what advantage would this bring ?
A standard, without too much effort.
or rather with what(language) and to what purpose are you trying to parse .info files ?
and
consider using awk ;) :P
Actually, I'm not trying to parse the info, at least not now.
or perhaps for this case its possible * to modify on the fly *
eg include ` sed -e's/Change-log:/Change-log:|/g ` this pipeline before processing tcz..info field ?
I don't think it's that easy.
Your example didn't work, but if it did, it probably wouldn't return the same results in the next .info from a different author.
-
hi CardealRusso
thanks for your reply / perspective
>Actually, I'm not trying to parse the info, at least not now.
in that case your subconscious *is* piloting to write one while you are distracted ;D
>I don't think it's that easy.
agreed
all though ... it appears the problem is less the "current non-standard format" and more "current non-standard format's"
if this is the case ( imho it appears so ) * see also : "repology-updater/issues/784# Tiny core linux ~ https://forum.tinycorelinux.net/index.php/topic,26273.0.html "
or perhaps *even more* accurately
the problem is
the current info files are intended for human consumption as plain text
not for programmatic / machine readability & processing
the latter obviously implies "trying to parse the info"
with this in mind
changing "description and changelog, in case there are multiple lines." the format
would (imho) only solve the problem for new/updated info files **
**assuming the *new* format was *somehow(by what process??!!)* enforced
that(imho) in-effect would add one more *known format* to the potential options ....
which reminds me of submit-TCZ script to help check for problems before sending extensions
... some *automated* method to identify the differences in format of info files currently in the repo (that could then be considered as alternative format or be corrected / standardized to a common format )
seams more likely to bring machine readability to info fles / the reop
-
xkcd ftw
https://xkcd.com/927/
-
the current info files are intended for human consumption as plain text
not for programmatic / machine readability & processing
the latter obviously implies "trying to parse the info"
....
seams more likely to bring machine readability to info fles / the reop
Yes, adopting a standard would increase machine readability.
-
... it appears the problem is less the "current non-standard format" and more "current non-standard format's"
if this is the case ( imho it appears so ) * see also : "repology-updater/issues/784# Tiny core linux ~ https://forum.tinycorelinux.net/index.php/topic,26273.0.html "
Sorry, I missed that part.
Indeed.
http://tinycorelinux.net/14.x/x86_64/tcz/alsa-modules-6.1.2-tinycore64.tcz.info
Change-log:
http://tinycorelinux.net/14.x/x86_64/tcz/bash-completion.tcz.info
Change-log: -
http://tinycorelinux.net/14.x/x86_64/tcz/adoptopenjdk-12.tcz.info
Change-log: ----
http://tinycorelinux.net/14.x/x86_64/tcz/88x2bu.tcz.info
Change-log: ----------
http://tinycorelinux.net/14.x/x86_64/tcz/aalib-dev.tcz.info
Change-log: first version
http://tinycorelinux.net/14.x/x86_64/tcz/advcomp.tcz.info
Change-log: 2012/12/02 first version
http://tinycorelinux.net/14.x/x86_64/tcz/aterm.tcz.info
Change-log: first version
2013/10/20
http://tinycorelinux.net/14.x/x86_64/tcz/brave-browser.tcz.info
Change-log: 2021/03/25 first version
...
-
xkcd ftw
https://xkcd.com/927/
fwiw aka "For What It's Worth (Stop, Hey What's That Sound)"
https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish
if the use of embrace in "to embrace the yaml" was not intended as 'Young in cheek' :P?
;) it might be considered `culturally insensitive` to those affected by proprietary licensing
... see also https://en.wikipedia.org/wiki/Halloween_documents
that aside ... its interesting to see the question of machine readability (re)appear again .
-
wrt previous mentioned *known format* and checking format correctness
i just discovered "GNU recutils – human readable, plain text databases "
GNU Recutils is a set of tools and libraries to access human-editable, plain text databases called recfiles. The data is stored as a sequence of records, each record containing an arbitrary number of named fields.
https://www.gnu.org/software/recutils/manual/recutils.html#Invoking-recins
echo "" | recins -f Name -v "Mr Foo" -f Email -v foo@bar.baz
Name: Mr Foo
Email: foo@bar.baz
-
Sorry to inject another format like json, all this format wars.
But see more of Linux tools like ip can output to json.
-
@mocore: Yes I was looking at using GNU Recutils for some tasks related to managing extension info files and it matches the info format very closely, so you could probably generate a Recutils database with just a little more than a "cat *.info". The main extra thing is you need to add a '+' at the start of new lines in field values (see here (http://www.gnu.org/software/recutils/manual/html_node/Fields.html)).
As usual I haven't gotten around to any of it myself and there are more useful projects for me to (not) get around to first...
patrikg: JSON? Much worse, I like the readability of the current Recutils-style.
-
*fanfare*
ftr ...
title : "Script for retrieving selected fields from .tcz.info files" @ https://forum.tinycorelinux.net/index.php/topic,27456.msg176787.html
script : ParseInfoFile.sh
url : https://forum.tinycorelinux.net/index.php?action=dlattach;topic=27456.0;attach=6971
thaughts : outsidethebox
This script was inspired by a thread that talked about handling .info files.
What the planned endgame there is, I still don't know.
I did find these statements a bit disturbing:
... the current info files are intended for human consumption as plain text
not for programmatic / machine readability & processing ...
and
... Yes, adopting a standard would increase machine readability. ...
It sounds like the idea is to make life easier for computers and more difficult
for people creating the .info file.
note : @rich thanks for the script/comments 8)
observation : correlational between disturbing / misapprehensions and posts / scripts in Programming & Scripting - Unofficial
conclusion : more misapprehensions more scripts ;)
-
@CNK
it appears each item in the Free Software Directory (https://directory.fsf.org/wiki/Main_Page) (FSD)
has a .rec file
and the fsd includes a total of "(17041)" items !
idk how many are packaged in each core version/arch repo(s)
so (ftr) it appears that
any one considering tcz-packaging a package included in FSD could "import" existing fields from the projects ./FSD (.rec) file
perhaps with ParseInfoFile.sh ....
... which might be construed as
- making the lives of "people who (attempt to) use computers to produce .tcz extensions" easier
eg : the recutils FSD.rec
https://git.savannah.gnu.org/cgit/recutils.git/tree/FSD
# -*- mode: rec -*-
#
# Information for the Free Software Directory maintained
# by the Free Software Foundation at http://www.fsf.org/directory
%rec: FSD_Entry http://www.jemarch.net/downloads/FSD.rec
Title: GNU Recutils
Description:
+ GNU Recutils is a set of tools and libraries to access human-editable
+ text-based databases called recfiles. The GNU recutils suite
+ comprises a Texinfo manual describing the rec format, a C library
+ (librec) providing a rich set of functions to access rec files, a set
+ of C utilities (recinf, recsel, recins, recset and recfix) that can be
+ used in shell scripts, and an Emacs mode (rec-mode).
GNU: yes
Homepage: http://www.gnu.org/software/recutils
PublicVCSCheckout: git clone git://git.sv.gnu.org/recutils.git
License: GPLv3PLUS
License: GFDLv21PLUS
Maintainer: Jose E. Marchesi <jemarch@gnu.org>
Version: 1.7
MaturityLevel: Production
DevelList: http://www.gnu.org/mailman/listinfo/bug-recutils
HelpList: http://www.gnu.org/mailman/listinfo/help-recutils
InterfaceStyle: CommandLine
InterfaceStyle: Console
# End of FSD
https://git.savannah.gnu.org/cgit/recutils.git/tree/etc/FSD.rec
# FSD.rec - Record descriptor for FSD entries
#
# This file contains the record descriptor for entries in the Free
# Software Directory.
#
# The canonical location of this file is
# http://www.jemarch.net/downloads/FSD.rec
# Copyright (C) 2010, 2020, 2022 Jose E. Marchesi
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
%rec: FSD_Entry
%mandatory: Title
%type: Title line
%type: GNU bool
%type: MaturityLevel enum
+ Undefined Planning PreAlpha Alpha Beta
+ Production Mature Orphaned
%type: License enum
+ GPLv2 GPLv2PLUS GPLv3 GPLv3PLUS
+ GFDLv21PLUS
%type: InterfaceStyle enum
+ CommandLine Console Daemon XWindow Web Email
# End of FSD.rec
-
Isn't the info file for the person installing the extension?
-
Hi NewUser
Yes, it's for the user installing the extension.
-
Hi everyone,
Switching to YAML for .info files is an interesting idea. While the current format works well for its intended purpose, YAML could bring more consistency and make automated parsing easier for those who need it.
At the same time, it’s worth remembering that .info files are primarily for users installing extensions, so keeping things straightforward and easy to read should remain a priority. If a change is made, it would make sense to approach it gradually, allowing for compatibility with the existing format and ensuring smooth adoption.
-
Hello.
I'm not trying to add more fuel to this argument but I'm in support of using an actual, structured format for creating .tcz.info files. The improvements that immediately come to my mind:
- Easier evaluation of extension submission with submitqc - we can more easily check for missing fields and also add submission details. This can make the submission of updates for existing extensions easier (I myself can write a submitqc patch for fetching .tcz.{info, dep} files for a previous version and placing in the current folder so that you don't need to have them in place).
- Ability to change the rendering format of .info files in an easier way - Apps and tce currently display whatever text the files contains on the server. We can change the rendering format as we can separate the fields and values with proper separators. We can have the changelog listed in chronological order or have the latest change first.
As for the format itself, any format (except XML, please) does the work. CSV, JSON (the GOAT), JSONC, YAML and TOML are a few options that I currently think would work fine.
Moreover, with the alpha testing of TCL v16, I don't think there is no better time for thinking of such a change, if it is actually necessary.
P. S. I also sometimes think of why tce and tce-load are written as bash scripts and not in C (or Rust, for those rustlings). Perhaps there is some reason, such as having editable scripts even after OS boot. In that case, I think we can think of switching to Lua, which is fast and editable. But the main reason I think of switching is performance. Maybe it is a debate for another major release of TCL.
-
@Sashank999: Not trying to be the devil's advocate, but regarding tce-load & co, I wish you good-luck... because you will not have :)
The basic commands (like tce-load) should work even without extensions loaded, aka in virgin base. And this base does NOT have lua, neither rust etc. (even if LUA is small size). The actual AWK engine is inside busy-box so it is C compiled.
A bump in speed is if using C (compiled) for any/all ash scripts, but curaga does not want it.
Plus is easy to edit/change them on the fly (debugging).
Anyway, any perceived increase in speed MUST first be measured, and if is significant THEN estimate how often is used. (ex: an increase of 50% --keep dreaming -- used only 10% of time is 5%, not impressive). YMMV.
-
Ability to change the rendering format of .info files in an easier way
see also : "(tcz) info 2 JSON (the GOAT)" @ https://forum.tinycorelinux.net/index.php/topic,27554.0.html
*please test ??? :o
I don't think there is no better time for thinking
thunk is cheep ... show me the code ( also the data thnx )
*8 bit mario noises*
I got special powers (wow)
From eating precious flowers
* https://learnxinyminutes.com/awk/
-
The basic commands (like tce-load) should work even without extensions loaded, aka in virgin base. And this base does NOT have lua
The lua interpreter is small (around 200 kB in size), so theoretically could be included in base system.
That being said, the TCL developers are (fortunately) very conservative. I wish I could bet that there will be no big changes to TCL's base system or package manager (not even to .info file format). Betting on this would be an easy way to get rich ;D
-
The lua interpreter is small (around 200 kB in size), so theoretically could be included in base system.
perhaps with luainkernel/lunatik "info 2 JSON" could be a module ???
-
The lua interpreter is small (around 200 kB in size), so theoretically could be included in base system.
That being said, the TCL developers are (fortunately) very conservative. I wish I could bet that there will be no big changes to TCL's base system or package manager (not even to .info file format). Betting on this would be an easy way to get rich ;D
Being conservative has its own challenges. Being resistant to change will lead not only to stability, but to stagnation as well. I think I read somewhere that Robert Shingledecker himself started TCL because the Damn Small Linux maintainer rejected this idea.
Now, I do not want to start a new distro. I would like to see some improvements in TCL distro itself. Maybe packaging format won't change, but IMO we should be having more speed during startup. And for an actually lightweight distro like TCL, the first thing that any user would expect is fast startup. Creating all the symlinks through shell script makes it slow. I personally have the following items onload:
Xprogs.tcz
firmware-atheros.tcz
wifi.tcz
xf86-input-libinput.tcz
i2c-6.6.8-tinycore64.tcz
xf86-video-intel.tcz
icewm.tcz
Xorg-7.7-bin.tcz
screen.tcz
firmware-rtl_nic.tcz
firefox.tcz
gnutls38.tcz
elfutils.tcz
libcamera.tcz
alsa-config.tcz
pulseaudio.tcz
xorg-server.tcz
mc.tcz
mylocale.tcz
rxvt.tcz
bash-completion.tcz
vim.tcz
terminus-fonts.tcz
xclip.tcz
ClipIt.tcz
adwaita-icon-theme.tcz
squashfs-tools.tcz
nano.tcz
simple-mtpfs.tcz
notosanstelugu-fonts-ttf.tcz
htop.tcz
man-pages.tcz
git.tcz
icu74.tcz
libsecret.tcz
gnome-keyring.tcz
python3.9.tcz
tk8.6.tcz
brightnessctl.tcz
provides.tcz
notocoloremoji-fonts-ttf.tcz
gnome-terminal.tcz
busybox_extras.tcz
pcmanfm.tcz
libportal-gtk3.tcz
file-roller.tcz
notosansdevanagari-fonts-ttf.tcz
notosans-fonts-ttf.tcz
and it takes around 3 minutes only for my setup to load completely (please suggest any changes that you have if I have done something wrong or suboptimal in my onboot.lst). I want this to change, which is why I want to find some suggestions to optimize it.
-
By “onload” do you mean onboot or ondemand?
Perhaps the first place to start would be to remove the recursive deps from your onboot list.
-
By “onload” do you mean onboot or ondemand?
Perhaps the first place to start would be to remove the recursive deps from your onboot list.
I meant onboot. Sorry. I will remove the recursive dependencies, measure execution time and post it here.
Edit: Forgot the mention my laptop's specifications: Intel Core i3 6th generation, 4GB RAM, 512GB SSD, Ethernet and Wi-Fi. I have my bootloader on a USB stick from which I multiboot into TCL. I use a modified Corepure64 initrd, the modification being the execution of an image viewer to show a splash image during boot, hiding other booting logs.
-
Gnome stuff tends to be heavy, though you don't use full gnome but some apps (at least for full gnome it would be better to use another distro). If you boot with the showapps bootcode (and disable the splash temporarily), you can see what takes the most time.
It may not be anything to do with shell, it may be your usb stick is slow, for example.
-
There's also the question of if you're using a backup and, if so, how big it is.
-
Maybe packaging format won't change, but IMO we should be having more speed during startup.
And for an actually lightweight distro like TCL, the first thing that any user would expect is fast startup.
Creating all the symlinks through shell script makes it slow.
...
and it takes around 3 minutes only for my setup to load completely (please suggest any changes that you have if I have done something wrong or suboptimal in my onboot.lst). I want this to change, which is why I want to find some suggestions to optimize it.
this might be relevant ,
though it appears it would require some testing ( eg trying to combine different groups of tcz to se what works with your list of tcz / system ect )
"Combining Extensions into Large Extensions" @ https://forum.tinycorelinux.net/index.php/topic,3757.msg19679.html#msg19679
afaik
a similar method is used by dcore to create "scm packages" (of many Debian packages )
fwiw only i3 i have (not shore what gen) used was laggy / slow when copying/moving data
( .. after booting to desktop with file-manager , for every distro i tested , im assuming something *by design* with architecture & cache ?? )
-
it takes around 3 minutes only for my setup to load completely
Hi Sashank999. A 3-minute boot time suggests there's something unusual about your setup. I have a lot of extensions in my onboot.lst, my laptop is an old clunker (X230 ThinkPad made in 2012), and my boot time is 24 seconds.
Juanito suggested checking the size of your backup (mydata.tgz in tce directory). I do remember that my boot time improved when I moved large files out of mydata.tgz into my persistent directory (/opt).
-
Hello all,
I'm really sorry to disappoint you all by solving this mystery myself. Turns out, I wrote `sudo wifi.sh -a` in my `/opt/bootsync.sh` and as I'm in a different location, it did not detect my home Wi-Fi network which was causing too much variable amounts of delay during boot.
I made it a background job in bootsync.sh and now my system loads up in 29.45 seconds, averaged over 3 boots.
Gnome stuff tends to be heavy, though you don't use full gnome but some apps (at least for full gnome it would be better to use another distro). If you boot with the showapps bootcode (and disable the splash temporarily), you can see what takes the most time.
It may not be anything to do with shell, it may be your usb stick is slow, for example.
Hello curaga.
Only my bootloader (Ventoy, uses GRUB 2 internally) is on the USB stick. TCL is installed on a 10 GiB partition on my laptop's SSD, with persistent /home and /opt.
I also have another boot option that uses the stock Corepure64 initrd, which doesn't have the splash image. I used it and it looks like no tcz is causing too much delay during load.
I use Gnome keyring in order to secure my Github personal access token that I use to push commits to Github for the Core-scripts and fltk_projects repositories. I use Gnome terminal as it directly copies to the clipboard that I can use with Firefox. IIRC, there are 2 clipboards in X and Gnome terminal copies my mouse selections to the clipboard that Firefox also uses and also pastes from it. I found no other alternative so I had to live with it.
There's also the question of if you're using a backup and, if so, how big it is.
Hello Juanito.
Yes, I am using a backup and it is very small. I only included the /opt directory for backup of startup scripts, and its size is
`0.01 MB (12015 bytes)`. I don't think that will affect the boot time at all.
this might be relevant ,
though it appears it would require some testing ( eg trying to combine different groups of tcz to se what works with your list of tcz / system ect )
"Combining Extensions into Large Extensions" @ https://forum.tinycorelinux.net/index.php/topic,3757.msg19679.html#msg19679
Hello mocore.
I would rather not do that because if any one dependency is updated in the TCZ repo, I would have to restart the process. I let tce-audit do that work for me.
Hi Sashank999. A 3-minute boot time suggests there's something unusual about your setup. I have a lot of extensions in my onboot.lst, my laptop is an old clunker (X230 ThinkPad made in 2012), and my boot time is 24 seconds.
Juanito suggested checking the size of your backup (mydata.tgz in tce directory). I do remember that my boot time improved when I moved large files out of mydata.tgz into my persistent directory (/opt).
Hello GNUser.
I already did that as suggested on this forums by some user on an old thread of mine. Now my backup size is small enough that it does not have any noticeable impact on my boot time.
I thank you all for your time.
-
@Sashank999 thank you for sharing your discoveries!