WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: tce-load compiled in C for faster boot?  (Read 2325 times)

Offline Roberto A. Foglietta

  • Full Member
  • ***
  • Posts: 212
    • personal home page
tce-load compiled in C for faster boot?
« on: August 19, 2021, 01:50:58 PM »
Does anyone take in consideration to write the tce-load in C for a faster boot especially when may TCZ are involved?

Offline hiro

  • Hero Member
  • *****
  • Posts: 1217
Re: tce-load compiled in C for faster boot?
« Reply #1 on: August 19, 2021, 02:29:16 PM »
i don't think that script is the main bottleneck for the delays.
measure first...

Offline nick65go

  • Hero Member
  • *****
  • Posts: 799
Re: tce-load compiled in C for faster boot?
« Reply #2 on: August 20, 2021, 01:49:19 AM »
@roberto: I think the big delay for "tce-load" script is because it calls an internal function
Code: [Select]
app_exists() {
    [ -f "$2/$1" ] && [ -f "$2/$1".md5.txt ] && (cd "$2" && md5sum -cs "$1".md5.txt)
}
so md5 check for EACH tcz could be  the main delay.

What else, logically can be? The main process is to build a full list of tcz (from recursive *.dep) and mount them all sequentially.
Can you mount them in parallel (in background?), with an algorithm half of half of half (for more than 4 tczs)?
The order of tcz should not matter if in the final a (possible) /usr/local/tce.installed/$app/xx.sh is run, to cover all asynchronous tcz loaded.
« Last Edit: August 20, 2021, 02:05:52 AM by nick65go »

Offline Roberto A. Foglietta

  • Full Member
  • ***
  • Posts: 212
    • personal home page
Re: tce-load compiled in C for faster boot?
« Reply #3 on: August 20, 2021, 02:39:18 AM »
@roberto: I think the big delay for "tce-load" script is because it calls an internal function
Code: [Select]
app_exists() {
    [ -f "$2/$1" ] && [ -f "$2/$1".md5.txt ] && (cd "$2" && md5sum -cs "$1".md5.txt)
}
so md5 check for EACH tcz could be  the main delay.

That check works only when a TCZ is going to be downloaded, not a boot time: tested.

However, thank you for the interesting point. I makes me able to solve another problem related to my meta-packages.
« Last Edit: August 20, 2021, 02:54:38 AM by Roberto A. Foglietta »

Offline Roberto A. Foglietta

  • Full Member
  • ***
  • Posts: 212
    • personal home page
Re: tce-load compiled in C for faster boot?
« Reply #4 on: August 20, 2021, 02:58:42 AM »
About fetch_app() function: should not download the .dep file, also?

Offline Roberto A. Foglietta

  • Full Member
  • ***
  • Posts: 212
    • personal home page
Re: tce-load compiled in C for faster boot?
« Reply #5 on: August 20, 2021, 03:21:20 AM »
The first line avoid the case in which the TCZ is just installed but not available where supposed to be

Code: [Select]
app_exists() {
[ -e "$TCEINSTALLED/${1/.tcz/}" ] && return 0
[ -f "$2/$1" ] && [ -f "$2/$1".md5.txt ] && (cd "$2" && md5sum -cs "$1".md5.txt)
}

Please evaluate to apply this change. Thank you, R.

Offline nick65go

  • Hero Member
  • *****
  • Posts: 799
Re: tce-load compiled in C for faster boot?
« Reply #6 on: August 20, 2021, 04:01:20 AM »
@roberto: maybe a specific smaller new script "onboot-tce-load" can be made, for few KB on disk size.
because the generic tce-load has a lot of not-necesary checks, which are already TRUE on booting (nothing was already mounted):
-check for "-d /tmp/tcloop/..  || sudo /bin/mkdir -p /tmp/tcloop/...
awk -v appname .. / tmp / tcloop / ... exit 1 }  " / etc / mtab"
« Last Edit: August 20, 2021, 04:04:55 AM by nick65go »

Offline Roberto A. Foglietta

  • Full Member
  • ***
  • Posts: 212
    • personal home page
Re: tce-load compiled in C for faster boot?
« Reply #7 on: August 20, 2021, 04:24:35 AM »
Modified version in quote does not trigger the forum issue
Code: [Select]
app_exists() {
[ "$INSTALL" == "TRUE" -a -e "$TCEINSTALLED/${1/.tcz/}" ] && return 0
[ -f "$2/$1" ] && [ -f "$2/$1".md5.txt ] && (cd "$2" && md5sum -cs "$1".md5.txt)
}

Please evaluate to apply this change. Thank you, R.

This version is even more specific: do not download an installed app if the aim of the download is to install it otherwise it takes the download as a specific user request and satisfy it.

In the attachment because forum issue
« Last Edit: August 20, 2021, 04:27:51 AM by Roberto A. Foglietta »

Offline nick65go

  • Hero Member
  • *****
  • Posts: 799
Re: tce-load compiled in C for faster boot?
« Reply #8 on: August 20, 2021, 04:35:10 AM »
@roberto: regarding a specific smaller new script "onboot-tce-load", the old one / generic "tce-load" including just the rest on options (except $ONBOOT option)
It does NOT need to check if a tcz has a md5 associated. Neither to wget-fetch anything from any server; just to build the recursive dependencies and mount them. The main golden engine, recursive_scan_dep() can be an external subroutine (in /etc/init.d/tc-functions). IMMV.
PS: even the awk engine recursive_scan_dep() can be a little faster, because removing wget: " rm -f depfile ; w.g.e.t. -c -P "
« Last Edit: August 20, 2021, 05:02:43 AM by nick65go »