Tiny Core Linux

General TC => Programming & Scripting - Unofficial => Topic started by: Rich on July 08, 2019, 01:08:02 PM

Title: Script to download extensions+dependencies on other Linux distros
Post by: Rich on July 08, 2019, 01:08:02 PM
This is my attempt to try to ease a situation that comes up on a regular basis:
I can't get the extension(s) I need because I can't get my wireless connection up.

After downloading the script make it executable:
Code: [Select]
sudo chmod 777 FetchExt.sh
Adjust these 3 variables in the script to match your requirements:
Code: [Select]
# Repository to download from
ADDR="http://repo.tinycorelinux.net"

# Tinycore version
TC="10.x"

# Processor architecture, current options are  x86  x86_64  armv6  armv7
ARCH="x86"

Place the script in its own directory to keep all downloads in one place. To run it, go to that directory and:
Code: [Select]
./FetchExt.sh ExtensionNameNote the leading  dot  and  slash. They are required because the directory is not in your systems PATH.

This should run on any Linux machine that has an Internet connection. I tried to avoid any Tinycore specific references and kept
command options as simple as possible. Dependencies are resolved by reducing the  .tree  file to a sorted list of unique entries.
Files are downloaded to the directory you are in when you invoke the script.

I think the script is fairly well documented if you want to see what's going on. Comments and criticism are welcomed.

FetchExt.sh ExtensionName  without the  .tcz  will download the extension+dependencies including .dep and .md5 files
FetchExt.sh info  will download a list  of available extensions and attempt to display it using the  less  command in a separate terminal.
                          If that fails, just open  info.lst  with your favorite file viewer.

This is what the  ./Log.txt  file it generates looks like:
Code: [Select]
Mon Jul  8 00:55:48 UTC 2019
Processing http://repo.tinycorelinux.net/10.x/x86/tcz/wifi.tcz.tree
libiw.tcz downloaded.
libiw.tcz.md5.txt downloaded.
libnl.tcz downloaded.
libnl.tcz.md5.txt downloaded.
ncursesw.tcz downloaded.
ncursesw.tcz.md5.txt downloaded.
openssl.tcz downloaded.
openssl.tcz.md5.txt downloaded.
readline.tcz downloaded.
readline.tcz.md5.txt downloaded.
readline.tcz.dep downloaded.
wifi.tcz downloaded.
wifi.tcz.md5.txt downloaded.
wifi.tcz.dep downloaded.
wireless-4.19.10-tinycore.tcz downloaded.
wireless-4.19.10-tinycore.tcz.md5.txt downloaded.
wireless_tools.tcz downloaded.
wireless_tools.tcz.md5.txt downloaded.
wireless_tools.tcz.dep downloaded.
wpa_supplicant.tcz downloaded.
wpa_supplicant.tcz.md5.txt downloaded.
wpa_supplicant.tcz.dep downloaded.

Mon Jul  8 15:06:48 UTC 2019
Processing http://repo.tinycorelinux.net/10.x/x86/tcz/wireless-rtl8822be-4.19.10-tinycore.tcz.tree
firmware-rtlwifi.tcz downloaded.
firmware-rtlwifi.tcz.md5.txt downloaded.
wireless-4.19.10-tinycore.tcz already downloaded.
wireless-rtl8822be-4.19.10-tinycore.tcz downloaded.
wireless-rtl8822be-4.19.10-tinycore.tcz.md5.txt downloaded.
wireless-rtl8822be-4.19.10-tinycore.tcz.dep downloaded.

The line after each timestamp is what you add to your  onboot.lst  file, in this case  wireless-rtl8822be-4.19.10-tinycore.tcz
and  wifi,tcz.

Then copy  *.tcz*  from your download directory to your  tce/optional directory.


The script is attached for downloading.

A newer version of the script can be found here:
http://forum.tinycorelinux.net/index.php/topic,23034.msg164745.html#msg164745

    [EDIT]: Updated attachment.  Rich
    [EDIT]: Added some clarification to my post.  Rich
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 05, 2019, 04:17:08 PM
Sorry, very noobish of me, but I have no clue how to run the script. My internet-able computer is running Kali Linux (which I also don't know how to use properly), and I tried navigating through CLI to various folders, specifically, where "FetchExt.sh" is located, and I tried just commandline execute of "FetchExt.sh ExtensionName", and I tried "nano FetchExt.sh" and I guess maybe I forgot to sudo some of these? I can try that next.

But maybe I am supposed to do something else with the file? How do I download TC.tcz, using FetchExt.sh ?
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on December 05, 2019, 04:52:08 PM
Hi doctoranonymous
The first thing you need to do after downloading the script is to make it executable. Go to the folder with the script and execute:
Code: [Select]
sudo chmod 777 FetchExt.sh
To fetch nano for example:
Code: [Select]
./FetchExt.sh nanoNote the leading dot and slash. I recommend placing  FetchExt.sh  in its own directory and running it there. This way you don't
clutter up one of your other directories with extensions.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 05, 2019, 09:41:40 PM
Several things, Rich. Thank you for being awesome; (a) helpful, (b) prompt reply, (c) not condescending or elitist. I have noticed a lot of really bad info on the internet, not because it is wrong, but because it tells and shows HOW to do a thing right, but explains nothing about the WHY of the HOW.

You are being a good teacher here, hence "moderator" status. Now, I have probably used the chmod command in ten or twenty other "recipes" and still don't know quite what it does. I have probably even read the man pages, but I think, if this works (I haven't tried it yet), I will forever remember "chmod" as "That instruction that makes a bash shell script, ".sh", executable". Now, the "777" is a bit more opaque and abstruse, and, quite frankly arbitrary? But I can't argue; I am a biologist, and our science probably looks crazy to you guys, and your science, too!

OK one last thing (that will probably make you pull your hair out, and whisper-muttering-yelling "NOOBIE!"), is Tannenbaum. Not, tis the season, "Oh Tannenbaum", but Arthur Tanenbaum, Minix, and Linux. I have been reading his Minix book, because I want to know about what Linus knew when he invented the stuff we are hacking away, diligently, daily, upon. And so, the file system structure (ambiguous terms here? yes. one meaning is e.g. minix, ext4, FAT, and another , meaning is the "Unix tree",  or "directory structure"? I sometimes wish I was better with the distinctions and finer minutiae of "programmer-speak"?). So, I was thinking about directories and those "holdover", "standard-folders", from the standard UNIX type OSes, and how, I am a silly windows freak (until 2016), so I just make a folder, anywhere, when I want one. In theory, this is incorrect for Linux? Or perhaps, just a "cluttery" type attitude?

I read somewhere that certain of these "standard folders", in a Linux system could be placed in different partition, and personal files kept there, so that, if wiping and installing a new OS, these personal "work files" will be retained in the new OS? Now the Tiny core "extensions" seem like a similar idea; (1) keep a clean boot, (2) don't mix up the superfluous with the essential, etc. ( I still "sort of" read the tiny core book... bad student sometimes.).

So, as per your instructions, and my predilection for clutter, I will make a FetchedExts directory, in root (though in truth, I am sometimes perplexed that "/" is root and "/root/" is root? Oh well, I guess we can't all make sense, all the time, and computer programmers are no exception to this rule?)

"Keep a clean boot, be careful not to clutter root, you don't need a Kali TAIL, to onion-out the plain suits"~Adapted/stolen from B. Dylan, by yours truly, just now ...
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 05, 2019, 09:55:00 PM
OK, I get this :

root@localhost:/FetchedExts# ./FetchExt.sh TC
./FetchExt.sh: 53: [: 8: unexpected operator
http://repo.tinycorelinux.net/10.x/x86/tcz/TC.tcz.tree  not found.

root@localhost:/FetchedExts# ./FetchExt.sh nano
./FetchExt.sh: 107: [: 0: unexpected operator
nano.tcz: OK
./FetchExt.sh: 107: [: 8: unexpected operator
ncurses5.tcz: OK

synopsis: seems to have fetched nano, but not TC? I will try ( flailing? in vain?) to troubleshoot it?

Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 05, 2019, 10:01:22 PM
Ahhhh, armv6 repo, not x86 repo !!! Maybe?

I think I have to alter the FetchExt.sh script to tell it where to go (and how to get there?) Maybe, then, re-chmod it?
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 05, 2019, 10:14:01 PM
So, I altered the code very slightly, went back and deleted nano (cuz its x86 and 10.x), and saved my altered code under the name FetchExt2.sh.

Here is the mild alteration:

Code: [Select]
"


# Repository to download from
ADDR="http://repo.tinycorelinux.net"

# Tinycore version
TC="9.x"

# Processor architecture, current options are  x86  x86_64  armv6  armv7
ARCH="armv6"

# This is where the extensions get downloaded from.
URL="$ADDR/$TC/$ARCH/tcz/"

EXT="$1"
TYPE=".tcz.tree"
if [ "$EXT" = "info" ]
then
TYPE=".lst"
fi

FILE=""

"

So, I'll chmod the fetchext2 and then run it, as per your previous instructions. Thanks again!

    [EDIT]: Added code tags.  Rich
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 05, 2019, 10:23:12 PM
So it is running now, and seems to be working; stuff is downloading. But still getting "[: 0/8: unexpected operator" messages?

Also, here is the directory tree for TC, which is basically superfluous for anyone else reading this, but completes and satisfies my questions. haich tee tee pee ://repo.tinycorelinux.net/9.x/armv6/tcz/TC.tcz.tree

Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 05, 2019, 11:03:05 PM
And, final loose ends (for other noobs like me), bash vs sh. The unexpected operator error message is because of different syntax between bash and sh shell scripting languages. I think, maybe "==" vs "=" is a necessary distinction; but also the "shebang" (whatever that is?) , i.e. #/bin/ vs /bin/ ?

As I said, I'm a noob, and this is obvious to some people, but its nice to search around a bit, find answers, and share freely. Cheers, all!
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 06, 2019, 05:14:29 AM
And .... Shebang! Now I have knowledge. haich tee tee pee ess ://en.wikipedia.org/wiki/Shebang_(Unix)
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on December 06, 2019, 05:43:15 AM
Hi doctoranonymous
... You are being a good teacher here, hence "moderator" status. Now, I have probably used the chmod command in ten or twenty other "recipes" and still don't know quite what it does. I have probably even read the man pages, but I think, if this works (I haven't tried it yet), I will forever remember "chmod" as "That instruction that makes a bash shell script, ".sh", executable". Now, the "777" is a bit more opaque and abstruse, and, quite frankly arbitrary? But I can't argue; I am a biologist, and our science probably looks crazy to you guys, and your science, too!
Short answer: chmod sets who has permission to read, write, and execute a file. 777 is a shotgun approach that lets anyone
                      run the script.

Longer answer: The following directory listing has been spread out so I could place titles above some fields.
Code: [Select]
    U    G    O         User    Group
-  rwx  rwx  rwx   1     tc     staff     3386 Jul  8 16:05 FetchExt.sh
You 'll see that U (tc), G (staff), and O (everyone else) have permission to read, write, and execute the file. Those are what
the 3 digits in  777  represent. r, w, and x have values like this:
Code: [Select]
4 2 1
r w x
So to give read and write permission it's  r+w  which equals  4+2  which equals  6.

The point of this was to set the  O (everyone else)  field to  7  to avoid dealing with users and groups on other Linux systems.

Quote
... in root (though in truth, I am sometimes perplexed that "/" is root and "/root/" is root? Oh well, I guess we can't all make sense, all the time, and computer programmers are no exception to this rule?) ...
The  /  is the root directory below which all directories get attached.  The  /root/  is a directory named root which happens to be
the  home  directory for the user  root (i.e superuser).
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on December 06, 2019, 06:31:50 AM
Hi doctoranonymous
And, final loose ends (for other noobs like me), bash vs sh. The unexpected operator error message is because of different syntax between bash and sh shell scripting languages. I think, maybe "==" vs "=" is a necessary distinction; ...
I tried the script under  bash  but could not reproduce your error messages. I do agree there were 2 places I should have
used  ==  instead of  =.  I have attached a corrected script in my original post. Please download it and let me know if the
error messages go away.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 06, 2019, 03:15:06 PM
about "==" vs "=", I wasn't being critical, but thanks for fixing anyway. I was going to try fixing it, on my end, for my own purposes, but I was basically "coding in my sleep" last night. Got some ideas, tried them out, read a lot, and somehow found the right answers. But as for commenting intelligently or tactfully, I did not.

My last few short posts were at exhaustion's limit, and I didn't have the energy to dive into a study of "shell scripting" of "bourne" versus "bash". (still don't have the energy, today. Maybe tomorrow?).

Thanks for fixing anyway; other noobs like me will see this and avoid silly mistakes.

Also, after I posted "/" vs "/root/" , I went tried it on my laptop Kali linux (GUI) system, and found that "/home/" is mixed up in it too. I read a good description of why this is, recently, but forgot. Basically, its a historical artifact of how Unix developed and how it turned into Linux, etc. I think maybe wikipedia, too, has a good "rundown" on what these directories have been used for historically, and how they are used now (good they do, because I'll be darned if know!).

I will dl it and try it. Can't remember if Kali is Bash or Bourne (Oh, I think it is Bash; it always yells that at me, because I am a noob and can't get the commands right! hahaha!), but will provide that info too, in my reply. Thanks again, not just for the code/computer fix, but for the "brain patch", i.e. wetware-brain, i.e. education.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on December 06, 2019, 05:11:55 PM
Hi doctoranonymous
about "==" vs "=", I wasn't being critical, but thanks for fixing anyway. I was going to try fixing it, on my end, for my own purposes, but I was basically "coding in my sleep" last night. Got some ideas, tried them out, read a lot, and somehow found the right answers. But as for commenting intelligently or tactfully, I did not. ...

 ... Thanks for fixing anyway; other noobs like me will see this and avoid silly mistakes. ...
My script performed 4 tests for equality, with 2 coded as  =  and the other 2 coded as  == , so I'd say you were right to
question my silly mistakes. :)
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: doctoranonymous on December 07, 2019, 02:02:52 AM
Looks good here, too.

Code: [Select]
root@localhost:/FetchExt2Bdeldls# ./FetchExtMod.sh nano
nano.tcz: OK
ncurses5.tcz: OK

I should probably mention here, for any other beginners trying to follow this thread, that I am working with piCore, 9.x, for the raspberry pi zero, and trying to get TC desktop extension, offline (not all my pi zeros are zero W; I have one zero W, and about five zero-plains). Bottom line? Success. This code snippet will work for you (if you know what you want)!

Eventually a cluster computer is envisioned, and I'd like to load piCore into RAM, resident, and then unplug the SD card and boot another pi zero with the same card. I was doing this, to save money on cards.

After the past two days of internet scouring, I have found some other methods, and tried a few, but I'll leave that in-depth discussion for another post, where it won't be so off-topic. I navigated to this string from a cluster computer topic, and I forgot that other people reading this don't necessarily know where I came from and what I'm doing ! (Like so many things, in "real life").

In short, I have also looked at ethernet gadget for several months (on and off; busy elsewhere), but stumbled across usbboot (github; official rpi site), and some discussions on USB host boot, vs. USB device boot. Its a long story and I don't understand it yet, but 2 more options, potentially SD-cardless!

One thing, if you are thinking of USB boot methods, be careful of the OTP, One Time Programming bits, because you can flip your pi one way, and then not get it back! I luckily, for a novice, have not done this, but suffice it to say I will procxeed to USB host/device boot methods with appropriate caution.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: xor on December 22, 2021, 01:47:55 AM
I'm asking to understand the working logic!?

How do we call 32-bit dependencies on 64.bit!?
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Juanito on December 22, 2021, 01:51:00 AM
Everything is explained in the first post of this thread.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 03, 2022, 10:10:32 PM
I've attached an updated copy of  FetchExt.sh  to this post.

You can now enter multiple extensions in one command and it no longer matters whether or not you include
 .tcz  in the extension name:
Code: [Select]
tc@E310:~/Scripting/FetchExtensions/tmp/debug$ ./FetchExt.sh 9vx xt_geoip_LE_IPv4 9vx.tcz xyzzy 915resolution.tcz xyzzy.tcz
..........
tc@E310:~/Scripting/FetchExtensions/tmp/debug$
I replaced excess screen output with dots to indicate progress since the  Log.txt  file has a record of all activity anyway.

This is the log file from the above command:
Code: [Select]
Sun Apr  3 22:34:50 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
Processing 9vx.tcz
9vx.tcz downloaded.
9vx.tcz.md5.txt downloaded.

Sun Apr  3 22:34:55 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
Processing xt_geoip_LE_IPv4.tcz
iptables.tcz downloaded.
iptables.tcz.md5.txt downloaded.
iptables.tcz.dep downloaded.
netfilter-4.19.10-tinycore.tcz downloaded.
netfilter-4.19.10-tinycore.tcz.md5.txt downloaded.
netfilter-4.19.10-tinycore64.tcz downloaded.
netfilter-4.19.10-tinycore64.tcz.md5.txt downloaded.
xt_geoip_LE_IPv4.tcz downloaded.
xt_geoip_LE_IPv4.tcz.md5.txt downloaded.
xt_geoip_LE_IPv4.tcz.dep downloaded.
xtables-addons-4.19.10-tinycore.tcz downloaded.
xtables-addons-4.19.10-tinycore.tcz.md5.txt downloaded.
xtables-addons-4.19.10-tinycore.tcz.dep downloaded.
  Error:  xtables-addons-4.19.10-tinycore64.tcz download failed.
  Error:  xtables-addons-4.19.10-tinycore64.tcz.md5.txt download failed.

Sun Apr  3 22:35:03 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
Processing 9vx.tcz
9vx.tcz already downloaded.

Sun Apr  3 22:35:04 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
  Error:  Processing xyzzy.tcz failed.

Sun Apr  3 22:35:04 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
Processing 915resolution.tcz
915resolution.tcz downloaded.
915resolution.tcz.md5.txt downloaded.

Sun Apr  3 22:35:05 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
  Error:  Processing xyzzy.tcz failed.
The lines that begin with  Processing  refer to the extensions from the command line. That's what you add to
your onboot.lst  file,
The lines after that give the results for that extension and all dependencies.
Errors are indented for better visibility.

I've also attached a  ShowErrors.sh  script. It parses the Log.txt script created by FetchExt.sh and outputs a
formatted list of any errors found:
Code: [Select]
tc@E310:~/Scripting/FetchExtensions/tmp/debug$ ./ShowErrors.sh

Sun Apr  3 22:34:55 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
Processing xt_geoip_LE_IPv4.tcz
Error:  xtables-addons-4.19.10-tinycore64.tcz download failed.
Error:  xtables-addons-4.19.10-tinycore64.tcz.md5.txt download failed.

Sun Apr  3 22:35:04 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
Error:  Processing xyzzy.tcz failed.

Sun Apr  3 22:35:05 UTC 2022
http://repo.tinycorelinux.net/10.x/x86/tcz/
Error:  Processing xyzzy.tcz failed.

tc@E310:~/Scripting/FetchExtensions/tmp/debug$
The  xtables-addons  error shows that the 64 bit module extension is missing from the TC10 x86 repository.
The  Error:  Processing  message means the extension you requested from the command line does not exist.

The script now creates a list of kernel versions in the repo. If a kernel extension is needed, the script attempts to
download all kernel versions of that extension. Since the OS will load the correct version, it's just simpler that way.
PiCore sometimes has multiple versions. X86 supports core and core64, so we download both versions.

Remember to make the script make it executable:
Code: [Select]
sudo chmod 777 FetchExt.sh
Remember to edit the version and architecture variables to match your system:
Code: [Select]
# User variables ***************************************************************

# Website to download from.
ADDR="http://repo.tinycorelinux.net"

# Tinycore version. Versions always end in  .x , there are no minor version digits.
TC="10.x"

# Processor architecture, current options are  x86  x86_64  armv6  armv7  armv7l  aarch64
ARCH="x86"

# End of user variables ********************************************************

I think the script is fairly well documented if you want to see what's going on. Comments and criticism are welcome.

    [EDIT]: Attachments removed. They can be found later in this thread.  Rich
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: GNUser on April 06, 2022, 01:01:48 PM
This is great, Rich. Thank you.

Just two comments:

1. == is a bashism, which leads to "unexpected operator" errors with the  /bin/sh  shebang (see https://mywiki.wooledge.org/Bashism). Changing every  [ "foo" == "bar" ]  in the script to  [ "foo" = "bar" ]  solves the problem.

2. The three  chown  steps require superuser privileges. It seems the two options are either to run the entire script as superuser or to change the three  chown  commands to  sudo chown  . I prefer doing as little as possible as superuser, so I went with the second option. The downside to my approach is that it is slightly less portable since not every user on every OS has  sudo  installed.

I will keep the script handy. I was using the old version to help me upgrade from one TCL version to the next, but this new version is even better :)
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 06, 2022, 02:45:28 PM
Hi GNUser
... == is a bashism, which leads to "unexpected operator" errors ...
doctoranonymous reported the same issue in reply #8. I was not able to reproduce the issue (reply #11)
http://forum.tinycorelinux.net/index.php/topic,23034.msg146361.html#msg146361
I did a little searching and I think the correct syntaxes are:
[ "$1" = "info" ]  for strings.
[ $? -eq 0 ]  for integers (equal).
[ $? -ne 0 ]  for integers (not equal).

Could I ask you to change the tests of  $?  to the appropriate integer tests and see if that also fixes the error?
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: GNUser on April 06, 2022, 06:02:52 PM
Hi, Rich. Yes, using the integer syntax also fixes the error.

In your script you can use either the string comparison syntax:
[ "$var" = "string" ]
[ "$var" != "string" ]

Or the integer comparison syntax:
[ $var -eq 0 ]
[ $var -ne 0 ]

The reason you can use either in this script is because the $? variable will always be an integer (return code of the last command). Integers can be compared using integer comparison or string comparison (the shell implicitly converts integers to strings when performing a string comparison). The only thing that doesn't work is the == operator because, although it means the same thing as = in this context, it only exists in bash.

By the way, the integer comparisons would fail if the variable happens to hold a string (including empty string) or if the variable is undefined. For this reason, string comparison operators are generally preferable because they are less likely to error out. I generally only use integer comparison operators when I need one of these operators:
-gt (greater than)
-lt  (less than)
-ge (greater than or equal to)
-le (less than or equal to)
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 06, 2022, 08:43:09 PM
Hi GNUser
Hi, Rich. Yes, using the integer syntax also fixes the error. ...
Thanks for confirming that.

Quote
... integer comparisons would fail if the variable happens to hold a string (including empty string) or if the variable is undefined. ...
If my intent is to compare integers, that's how I would want it to behave, because those are all bugs. I think I'll
stick to  -eq  and  -ne  for integers.

... It seems the two options are either to run the entire script as superuser or to change the three  chown  commands to  sudo chown ...
I'm basically in agreement with you. That still leaves systems that request a password for sudo. That could include
systems that request a password for every sudo and those that use a timeout before requesting the password again.

For those situations I'm inclined to tell the user to run the script as root:
Forum error, please see attachment.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: gadget42 on April 06, 2022, 10:15:04 PM
re: bash

reminded of recent:
https://rachelbythebay.com/w/2022/04/05/pipe/

and which referenced previous:
https://rachelbythebay.com/w/2013/08/01/script/

sharing is caring
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: gadget42 on April 06, 2022, 11:37:06 PM
one more(re: bash):
https://rachelbythebay.com/w/2022/04/06/text/

"running empty files"...successfully!?!

sharing is caring
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: GNUser on April 07, 2022, 06:19:56 AM
Hi, Rich. I took the liberty to update the script with the following minor changes:

- Removed the == bashism
- Used integer comparison where appropriate (as per your preference)
- Added sudo handling
- Added UID user variable

The reason for adding the UID variable is that some TCL users (such as yours truly) use the user=somebody boot code (in my case it's user=bruno). So while user tc (UID 1001) does exist on my TCL box, my username on TCL is actually bruno (UID 1000). On my foreign Linux box my UID is also 1000. So for me, having the downloaded extensions owned by UID 1000 is much more convenient.

See attached.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 07, 2022, 07:21:21 AM
Hi GNUser
I already incorporated the integer and sudo changes, as well as a usage message, extremely basic command
line validation, and some other cleanup.

I will roll in your UID addition. Environmental variables use all uppercase, so I'll probably rename that.
I will include your moving chown to a function so I use it in a few other places.
Although the  use_sudo  is not necessary, I'll include it because it's probably good form.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 07, 2022, 06:01:03 PM
Hi GNUser
Here are new copies of both scripts. I added your changes, more error checking, and a help (-h) screen.
I did some cleanup on the ShowErrors.sh script too.

    [EDIT]: Attachments removed. They can be found later in this thread.  Rich
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: gadget42 on April 08, 2022, 03:31:41 AM
re: my earlier semi-related(or not) posts...sigh...one more...but Rob Pike! ftw!

https://rachelbythebay.com/w/2022/04/07/empty/

sharing is caring

20220408-0535am-modified: added "this is not the first mention of Rob Pike on the forum"(go figure...said tongue-in-cheek...lol)
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: GNUser on April 08, 2022, 06:09:53 AM
Hi, Rich. Looks great and works perfectly.

The only fault I can find is purely cosmetic: There are stray -es in Log.txt:

Forum error. See attached.

P.S. The issue is that the behavior of echo's options is notoriously inconsistent across different implementations. printf is a lot more consistent. Solution is in the attachment.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 08, 2022, 06:32:22 AM
Hi GNUser
... The only fault I can find is purely cosmetic: There are stray -es in Log.txt: ...
Unfortunately, it's not just cosmetic. The ShowErrors.sh  script is basically a state machine and uses the blank
lines as a reset mechanism.

I used echo because it's simple. It didn't occur to me that some versions didn't support escape sequences. I also used
it because it's likely available on all systems. I don't know if that's true of printf.

While it is a good fix, I do want to keep the number of commands required as small and simple as possible just to
minimize the odds of weird stuff like this happening.

Since I was just looking for some extra linefeeds, I'll add a few extra echos in place of the  \n  sequences.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: GNUser on April 08, 2022, 06:46:22 AM
I do want to keep the number of commands required as small and simple as possible just to
minimize the odds of weird stuff like this happening. Since I was just looking for some extra linefeeds, I'll add a few extra echos in place of the  \n  sequences.
I think  printf  is just as universal as  echo  but I agree with minimizing the number of required commands. Better to just use basic  echo  everywhere than use  echo  in some places and  printf  in others. Eliminating echo's  -e  option will do the trick.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 08, 2022, 07:51:31 PM
Hi GNUser
OK, fixed the echo commands in FetchExt.sh.

I also fixed a couple issues in ShowErrors.sh:
The read command was swallowing the 2 leading spaces from the lines beginning with Error:.
If the first entry in the Log.txt file was an error, its time stamp was ignored.

    [EDIT]: Attachments removed. They can be found later in this thread.  Rich
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: GNUser on April 09, 2022, 05:20:55 AM
Hi, Rich. FetchExt.sh works perfectly now (tested in Devuan where /bin/sh is a symlink to dash). The script has already become indispensable to me--it horrifies me to think of doing a fresh TCL install or a TCL upgrade without it. Thank you so much for putting it together!

P.S. I did not test ShowErrors.sh. FetchExt.sh was written by Rich and tested by GNUser, so I don't expect there will be any errors to show ;D

-----
An echo postmortem:

I can understand a shell programmer having to be mindful of avoiding bashisms when writing a script for a Bourne shell (a.k.a. /bin/sh or sh) implementation. What I will never understand is why the different Bourne shell implementations can't agree to be compatible. echo is a shell built-in, so the issue here was that in TCL's version of sh (BusyBox ash),  echo  needs the  -e  flag to interpret escape sequences. In Devuan's version of sh (dash),  echo  does not have the  -e  option because it interprets escape sequences by default:

Forum error. See attached.

You'd be hard-pressed to find someone who loves shell scripting more than I do, but this kind of pitfall (i.e., ones resulting from compatibility-busting bickering between different  sh  implementations) sure is frustrating.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 10, 2022, 02:37:11 PM
Hi GNUser
... P.S. I did not test ShowErrors.sh. FetchExt.sh was written by Rich and tested by GNUser, ...
Then I hope you don't mind I included this at the top of the script:
Code: [Select]
#!/bin/sh
# Script to download an extension and its dependencies written by Richard A. Rost July 7,2019
# Special thanks go out to GNUser for his time and effort spent testing and providing
# helpful suggestions and feedback.
#

Quote
... so I don't expect there will be any errors to show ...
Always expect the unexpected. We have had the following issues in the repo, all of which will flag an error:
Missing  .md5.txt  file.
Incorrect md5sum in the  .md5.txt  file.
Listed dependency not in the repo.

Which brings me to my final update. I failed to provide the user with a yay or nay when the script finishes. :-[

No errors:
Code: [Select]
tc@E310:~/Scripting/FetchExtensions/tmp/debug$ ./FetchExt.sh PicFormat abook ace-of-penguins acl
......................
OK


Errors:
Code: [Select]
tc@E310:~/Scripting/FetchExtensions/tmp/debug$ ./FetchExt.sh PicFormat abook xyzzy ace-of-penguins acl
......................
One or more errors occurred. See Log.txt after timestamp:
Sun Apr 10 16:43:16 UTC 2022

Seems there's still no xyzzy extension in the repo.

The latest copies are attached here. I will remove the older versions and point the original post to here later on.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: GNUser on April 11, 2022, 05:15:15 AM
Hi, Rich. The acknowledgement is fine, but it was my pleasure to help. 99% of the time the two of us are involved, the help flows in the other direction...

It's amazing that you found a way to make this script even better!
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: Rich on April 11, 2022, 02:00:13 PM
Hi GNUser
I really appreciate your help. The goal of this update was to make the script as painless to use as possible. Thanks to
your efforts, I think we may have accomplished that.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: zharr on May 18, 2022, 08:26:30 PM
So I didn't know this script existed and made my own script, including flashing and other setup, essentially so I can use one command to set up a repeatable image including all dependencies, wifi & ssh, custom program sources, etc.
It is less a ready-to-use script and more of a setup-once-and-forget kinda thing if you expect to distribute something that uses a tinyCore image or just want your setup to be repeatable.
Note: It's made for piCore, so you HAVE to edit your sources/kernel versions and extensions beforehand, and remove some custom RaspberryPi-specific config.
I posted the script here:
http://forum.tinycorelinux.net/index.php/topic,25760.0.html
Includes a custom wifi script so it connects to the best available known wifi.
With this I can set up a new SD card with everything I need and work on it over SSH within a minute, no matter where I am (office, home, etc.), without needing a keyboard and monitor.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: tacpilot on May 23, 2022, 09:06:59 AM
I had to split this across multiple posts because of the server error issue plaguing us all.

Though sed supports using most anything as a separator, unless you have some instance that
requires using something different, I would stick to using "/". This will keep your coding habits
inline with standard REGEX syntax across languages, and makes code easier to read when
used in a long line of command piping.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: tacpilot on May 23, 2022, 09:07:43 AM
Because of the extended features of bash, I so much prefer coding in bash over sh. However,
portability with shared scripts becomes and issue. With /bin/sh being linked to numerous shell
interpreter ie.. ash,dash, busybox , ... , the conformity guidelines revolve around being POSIX
compliant.

Coding habits become most important in collaborative environments, and accepted standards
become demanded. Its just good habits to get into.
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: tacpilot on May 23, 2022, 09:08:34 AM
There are numerous tools out there to help check for such things. Some you can run locally
and some sites you can post to.

Here is one such site that will check your code...
https://www.shellcheck.net/

Hope This Helps
Title: Re: Script to download extensions+dependencies on other Linux distros
Post by: zharr on May 23, 2022, 06:48:10 PM
Thank you very much tacpilot - as you might have guessed I am not that proficient in shell programming.
Though that means I can't use [[ ]] either, as it is not part of posix sh, right? Would you really recommend me to switch that?
I expect this script to be run on a Linux host, though even busybox supports [[ ]], so it should not be a big deal to use it right?
But yeah, making it compatible with tinyCore sounds like a good idea long term, so even people only using tinyCore can use it.
Actually sounds like a decent idea for my project, for people that want to build custom images for my embedded project but don't use linux, I can provide a tinyCore image to boot to that can be used to set up the embedded piCore image - genius!