tce-load patch to support local mirror in filesystem

[GNUser]

Currently, tce-load can only fetch extensions using wget. Therefore, /opt/tcemirror must contain a URL to http(s) or ftp server.

But what if the TC machine has access to a local mirror attached to its filesystem? It would seem silly for the machine to run http sever just to serve a mirror to itself.

I created a simple patch to tce-load. With this patch, tce-load examines the first character of the MIRROR variable. If the first character is / then tce-load uses cp to fetch the extension(s). Otherwise, tce-load uses wget as usual.

For example, if /opt/tcemirror contains something like this:

Code: [Select]

http://repo.tinycorelinux.net/Then tce-load uses wget as usual.

On the other hand, if /opt/tcemirror contains something like this:

Code: [Select]

/mnt/mirror/tinycorelinux/Then tce-load uses cp.

Here is the patch:

Code: [Select]

--- tce-load-old 2019-07-08 08:57:49.155847229 -0400
+++ tce-load-new 2019-07-08 10:12:52.121940000 -0400
@@ -7,7 +7,7 @@
 checknotroot
 PROG_NAME=$(basename $0)
 KERNELVER=$(uname -r)
-unset WGET INSTALL COPYINSTALL BOOTING ONDEMAND DOWNLOAD_ONLY LOAD_ONLY SUPPRESS
+unset GET INSTALL COPYINSTALL BOOTING ONDEMAND DOWNLOAD_ONLY LOAD_ONLY SUPPRESS
 FORCE="n"  # Overwrite system files default to no. Use -f to force overwrite.
 SAVED_DIR=`pwd`
 
@@ -49,7 +49,7 @@
 while getopts wilcbosft: OPTION
 do
  case ${OPTION} in
- w) WGET=TRUE ;;
+ w) GET=TRUE ;;
  i) INSTALL=TRUE ;;
  l) LOAD_ONLY=TRUE ;;
  c) COPYINSTALL=TRUE ;;
@@ -62,7 +62,7 @@
  esac
 done
 shift `expr $OPTIND - 1`
-[ -z "$1" ] || ( [ -z "$WGET" ] && [ -z "$INSTALL" ] ) && abort
+[ -z "$1" ] || ( [ -z "$GET" ] && [ -z "$INSTALL" ] ) && abort
 
 app_exists() {
  [ -f "$2/$1" ] && [ -f "$2/$1".md5.txt ] && (cd "$2" && md5sum -cs "$1".md5.txt)
@@ -70,8 +70,13 @@
 
 fetch_app() {
  echo "Downloading: $1"
- wget -cq "$MIRROR"/"$1".md5.txt 2>/dev/null
- wget -c "$MIRROR"/"$1"
+ if [ "$MIRROR_IS_PATH" ]; then
+ cp "$MIRROR"/"$1".md5.txt ./
+ cp "$MIRROR"/"$1" ./
+ else
+ wget -cq "$MIRROR"/"$1".md5.txt 2>/dev/null
+ wget -c "$MIRROR"/"$1"
+ fi
  md5sum -c "$1".md5.txt
  if [ "$?" != 0 ]; then
  echo "Error on $1"
@@ -158,7 +163,7 @@
 }
 
 recursive_scan_dep() {
- echo -e "$@"|awk '
+ echo -e "$@"|awk -v mirror_is_path="$MIRROR_IS_PATH" '
  function recursive_scan(name, optional, mirror, _, depfile, line, i) {
  gsub(/[\t ]+/, "", name)
  if (name) {
@@ -176,8 +181,12 @@
  IRANGE[name"#1"]=IDX
  depfile=optional"/"name".dep"
  if (mirror && (system("test ! -f "depfile) == 0 || system("test ! -f "optional"/"name) == 0))
- if (system("rm -f "depfile"; wget -c -P "optional" "mirror"/"name".dep 2>/dev/null") == 0 && ! SUPPRESS)
- system("echo "name".dep OK 1>&2")
+ if (mirror_is_path == "TRUE")
+ if (system("rm -f "depfile"; cp "mirror"/"name".dep ./"optional"/ 2>/dev/null") == 0 && ! SUPPRESS)
+ system("echo "name".dep OK 1>&2")
+ else
+ if (system("rm -f "depfile"; wget -c -P "optional" "mirror"/"name".dep 2>/dev/null") == 0 && ! SUPPRESS)
+ system("echo "name".dep OK 1>&2")
  MARK[name]=2
  if (mirror || system("test -f "optional"/"name) == 0) {
  while (getline line < depfile > 0)
@@ -215,7 +224,7 @@
 [ -f /etc/sysconfig/showapps ] && SHOWAPPS=TRUE && SUPPRESS=TRUE
 #  Check for download only
 [ -z "$INSTALL" ] && DOWNLOAD_ONLY=1
-[ -z "$WGET" ] && [ "$INSTALL" ] && LOAD_ONLY=1
+[ -z "$GET" ] && [ "$INSTALL" ] && LOAD_ONLY=1
 
 OPTIONAL="`realpath $TCEDIR`/optional"
 TARGETSLOCAL=""
@@ -245,7 +254,7 @@
  fi
 fi
 
-if [ "$WGET" ]; then
+if [ "$GET" ]; then
  if app_exists "$EXTENSION" "$FROMWHERE"; then
  echo "$APPNAME is already downloaded."
  else
@@ -263,6 +272,7 @@
 
 if [ "$TARGETSFETCH" ]; then
  getMirror
+ [ "$(echo $MIRROR | cut -c1)" = "/" ] && MIRROR_IS_PATH=TRUE
  TARGETSFETCH="`echo -e $TARGETSFETCH | awk '/\w/ {print $1" . '"$MIRROR"'"}'`"
  recursive_scan_dep "$TARGETSFETCH" | while read F; do
  { test "${F%% *}" = "@" && EXTENSION="${F#@ }" && SKIP="" || test "$SKIP"; } && continue
I hope it is useful.

Cheers,
-Bruno

[GNUser]

Although tce-load works with my patch, having an absolute path in /opt/tcemirror breaks other things. It definitely breaks tce-ab, for instance.

I found several executables in /usr/bin/ that call wget (provides.sh, tce-audit, tce-fetch.sh, tce-update, version). I think all of these would need to be looked at and potentially patched in order for an absolute path in /opt/tcemirror to not break anything. Alas, things are never as simple as they seem

[curaga]

Why not symlink optional/ into /mnt/mirror/tinycorelinux/10.x/x86? Granted, it doesn't let you use the App browser to view info files, but it lets "tce-load -i" work. And if it's on a separate disk, it may not work on boot.

It's such a special case though, that I think it would not have many users.

[GNUser]

Thank you. Excellent points.

If you don't think the flexibility of using a path in /opt/tcemirror would benefit a lot of users, I'll just drop it instead of patching all the affected files. I'm glad I posted the initial patch before rolling up my sleeves and doing more.

[Rich]

Hi GNUser

... But what if the TC machine has access to a local mirror attached to its filesystem? It would seem silly for the machine to run http sever just to serve a mirror to itself ...

It  might not be so silly. There are some pretty lightweight servers available:
http://tinycorelinux.net/10.x/x86/tcz/busybox-httpd.tcz.info   about 20K it has no dependencies
http://tinycorelinux.net/10.x/x86/tcz/lighttpd.tcz.info              about 1.6Meg including dependencies
http://tinycorelinux.net/10.x/x86/tcz/nginx.tcz.info                 about 1.9Meg including dependencies
The  wget  requests won't even go out on the wire. The kernel will route it all internally through the local loopback (127.0.0.1).

[GNUser]

20K for an http server?! Busybox is amazing.

[hiro]

200 lines here
http://www.acme.com/software/micro_httpd/