Tiny Core Linux
Off-Topic => Off-Topic - Tiny Tux's Corner => Topic started by: jazzbiker on July 18, 2023, 07:06:33 PM
-
Greetings!
I have some script. It do everything it must do. But as for me it looks like a heap of junk. What can be done to make it look a little bit better?
TNAME=${1##*/}
BNAME=${2##*/}
TDIR=${1%$TNAME}
depends-on ${TDIR}.do..${TNAME}
JOBS=${JOBS:-2}
LOGS=$(lua -e "for i = 1, $JOBS do print(('$TNAME.%d.log'):format(i)) end")
RDIR=$(pwd)
test -n "$TDIR" && cd $TDIR
export MAP_DIR=$(pwd)
REDO_DIRPREFIX=
for LOG in $LOGS
do
redo -l $LOG -m $TNAME $BNAME &
done
wait
{ echo 'return {'; cat $LOGS; echo '}'; } > $TNAME.log
cd $RDIR
lua log2map.lua $1.log > $3
Thanks in advance!
-
There's not too much you can do about it, bash are ugly by nature. Beauty is not the purpose but to just work.
I even ask a member of the TinyCore team here if it wouldn't be interesting to migrate some corescripts to Nim. Nim is a high-level programming language that goes very well with TinyCore; Result will be native binaries, fast, small (some binaries may be smaller than some bash scripts) and easily readable source code.
function run_redo(name, base) {
local logs=( "$name.%d.log" )
for log in "${logs[@]}"; do
redo -l "$log" -m "$name" "$base" &
done
wait
}
function main() {
local name=$1
local base=$2
local output=$3
run_redo "$name" "$base"
echo "return {$(cat "${logs[@]}")}" > "$output"
}
main "$@"/\ this is a bash script btw
-
Hi CardealRusso,
Thanks for reply! Yes, bash looks like overshooting for such chunk. I even try to make such pieces dash-able. So enclosing some code into the function may improve the outlook, agreed. But Your version misses some ugly details, which will damage the nice picture when added (
Nim is compiled into C, so I don't understand what may be its advantages.
Thanks again and best regards!
-
After removing some boilerplate with traveling through directories my ugly script looks like
TNAME=${1##*/}
TDIR=${1%$TNAME}
depends-on ${TDIR}.do..${TNAME}
JOBS=${JOBS:-2}
LOGS=$(lua -e "for i = 1, $JOBS do print(('$1.%d.log'):format(i)) end")
export MAP_DIR=$(test -n "$TDIR" && cd $TDIR; pwd)
for LOG in $LOGS
do
redo -l $LOG -m $1 $2 &
done
wait
{ echo 'return {'; cat $LOGS; echo '}'; } > $1.log
lua log2map.lua $1.log > $3
which seems to be slightly less ugly.
-
Hi, Andrey. I have hundreds of shell scripts. If they look like junk, then I won't understand them or know how to use them if I haven't used them in a while.
Class A recommendations:
- Use a main function
- Group related steps into functions
- Use function names that include a verb and clearly describe what the function does, so that comments become superfluous
- Put script name, version, purpose, and usage example at top of script
- Maximize simplicity and portability by avoiding bashisms (https://mywiki.wooledge.org/Bashism) unless you absolutely need bash
- Keep user variables and internal script variables (i.e., ones user should not mess with) separate
- Use clear variable names that include a noun
- After your script is done, check it with checkbashisms (https://github.com/duggan/shlint/blob/master/lib/checkbashisms) and shellcheck (https://www.shellcheck.net/)
Class B recommendations:
- Put main function at the top
- Use Allman braces style because it is trivial to use correctly (only rules are "each brace on its own line", "brace pairs equally indented")
I'm not sure what your script does, so I can't give the script or its functions good names. Here is a first attempt at cleaning it up. Note that this is rough pseudocode that has several obvious errors and doesn't actually work (plus, its stated purpose and usage example are completely made up)--it's just to illustrate the principles:
#!/bin/sh
# dependency-checker v1.0
# Purpose: Find all of an extension's dependencies
# Usage example: $ dependency-checker brave-browser
# user variables:
log=/home/andrey/logs/dependency-checker.logs
main()
{
run_redo "$name" "$base"
echo "return {$(cat "${logs[@]}")}" > "$output"
}
run_redo()
{
local logs=( "$name.%d.log" )
for log in "${logs[@]}"; do
redo -l "$log" -m "$name" "$base" &
done
wait
}
# internal variables:
TNAME=${1##*/}
BNAME=${2##*/}
TDIR=${1%$TNAME}
main "$@"
I try my hardest to make my shell scripts beautiful. The Pragmatic Programmer (https://www.amazon.com/Pragmatic-Programmer-journey-mastery-Anniversary/dp/0135957052/ref=sr_1_1?keywords=the+pragmatic+programmer&sr=8-1) is the book that I have found to be the most helpful in this regard.
-
Hi Bruno,
Thanks for the masterclass! As far a s I will be able I'll learn it. Some questions arose, caused with my bad knowledge of shell:
1)
"${logs[@]}"
- what's this? CardealRusso also used this construct in his example.
2) logs are declared as local
run_redo()
{
local logs=( "$name.%d.log" )
...
but are used in main():
main()
{
run_redo "$name" "$base"
echo "return {$(cat "${logs[@]}")}" > "$output"
}
What does "local" means in shell?
Thank You for the time spent and efforts!
-
Hi, Andrey. It seems CardealRusso is a bash guy. Bash is not my cup of tea, but "different strokes for different folks" :)
1) "${foo}" is the safer (better handling of spaces) equivalent of ${var}
$var is shorthand for ${var}
So far, so good.
"${foo[@]}" is a bashism that means "all elements of the array foo". POSIX shells (e.g., dash and busybox ash) do not support arrays.
2) local is not POSIX-defined
In shell, variables are global by default. If you want a variable to be local to a function, some shells allow the local varname (or the even less portable local varname=value) construct.
To avoid breakage and maximize portability, I'd avoid using bashisms and POSIX-undefined keywords in shell scripts.
Refs:
https://tldp.org/LDP/abs/html/arrays.html
https://mywiki.wooledge.org/Bashism (see "Arrays" and "Builtins")
----------
P.S. When I was learning shell, I found that these two excellent, succinct pages gave me the most bang for my buck:
https://mywiki.wooledge.org/BashPitfalls
https://mywiki.wooledge.org/Bashism
If you are extremely interested in shell, there is this exhaustive resource (caveats: time consuming, bash-specific):
https://tldp.org/LDP/abs/html/
-
I even ask a member of the TinyCore team here if it wouldn't be interesting to migrate some corescripts to Nim. Nim is a high-level programming language that goes very well with TinyCore; Result will be native binaries, fast, small (some binaries may be smaller than some bash scripts) and easily readable source code.
Have You done this for yourself and have practical results?
I expect shells are heavy legacy load from ancient times. I use for my personal purposes Lua as scripting language. I've started it as a kind of exercise. And was astonished that common tasks written with the interpreted language are accomplished faster than the same tasks done by specialized utilities written in compiled language - C. I have no explanaitions, simply got this as the fact.
In my opinion if You are interested in some replacement for shell, it must not be compiled language, it should be dynamically typed interpreted with garbage collector. Otherwise - C can do everything.
-
common tasks written with the [Lua] interpreted language are accomplished faster than the same tasks done by specialized utilities written in compiled language - C.
I'm astonished by this. I looked at Lua a few years ago when you clued me in. Although shell (system administration) and awk (text munching) can handle all my household's programming needs, I'll take a look at Lua again.
-
...it must not be compiled language...
I totally agree with you. To be honest, I hadn't considered Lua.
Usually linux distributions use python, but it seems that it would be inappropriate to integrate the python runtime into tinycore (since python is the same size as tinycore).
I considered Nim for its syntax. I strongly believe that concise and readable code greatly facilitates community development. Currently, CoreScripts bash scripts are neither easy to read nor user friendly (though understandable).
but yes, you are correct about lua, it is a very good and light option.
And you are also correct in requesting examples. I'll try to formulate an example of what it would look like, for example (it could be another one) tce-update in nim or lua (although I don't have a good command of lua)
-
Hi jazzbiker
... logs are declared as local ... but are used in main(): ...
The busybox ash shell includes local as one of its built-ins. Enter help in a terminal:
tc@E310:~$ help
Built-in commands:
------------------
. : [ [[ alias bg break cd chdir command continue echo eval exec
exit export false fg getopts hash help history jobs kill let
local printf pwd read readonly return set shift source test times
trap true type ulimit umask unalias unset wait
tc@E310:~$
Here is an example of local:
#!/bin/sh
Var=5
TestLocal()
{
# This echos the global Var.
echo "2 Var=$Var"
# This overrides the global Var with an empty local Var.
local Var
echo "3 Var=$Var"
# This sets the local Var to 3.
Var=3
echo "4 Var=$Var"
}
echo "1 Var=$Var"
TestLocal
echo "5 Var=$Var"
This is the result:
tc@E310:~/split$ ./LocalDemo
1 Var=5
2 Var=5
3 Var=
4 Var=3
5 Var=5
tc@E310:~/split$
-
Currently, CoreScripts bash scripts are neither easy to read nor user friendly (though understandable).
Core scripts have one huge advantage - they work properly ) I don't want to propose repair what's working right. For TinyCore phenomenal stability is absolutely unique. Some features are accessible only in TinyCore.
And even if put the question - what I would like to rewrite in Core? Frankly speaking - nothing. Extending preferably be done in some new ways or new tools, but the Core is wonderful! Stability is easy to destroy but impossible to repair.
If talk about another languages, for example Lua, some things may be taken into consideration. For example I wrote the Lua version of the initial script. Here it is:
#!/usr/bin/env lua
local TDir, TName = arg[1]:match("(.-)([^/]*)$")
------------------------------------------------
-- Auxillary function to run complex commands --
------------------------------------------------
local run = function(cmd)
assert(os.execute(table.concat(cmd)))
end
run{"depends-on ", TDir, ".do..", TName}
------------------------------
-- Gathering necessary data --
------------------------------
local jobs = tonumber(os.getenv("JOBS")) or 2
local map_dir = assert(io.popen("cd " .. TDir .. "./; pwd")):read()
-----------------------
-- Storing log names --
-----------------------
local log_names = {}
for i = 1, jobs do
log_names[#log_names + 1] = string.format("%s.%d.log", arg[1], i)
end
------------------------------------------
-- Creating and running complex command --
------------------------------------------
local cmd = {}
for i, name in ipairs(log_names) do
cmd[#cmd + 1] = string.format("redo -l %s -m %s %s & ", name, arg[1], arg[2])
end
cmd[#cmd + 1] = "wait"
run(cmd)
----------------------------------------------
-- Concatenation of partial logs in one log --
----------------------------------------------
local log_name = arg[1] .. ".log"
local flog = io.open(log_name, "w")
flog:write("return {\n")
for i, name in ipairs(log_names) do
flog:write(io.open(name):read("a"))
end
flog:write("}\n")
flog:close()
---------------------------------
-- Loading the processing code --
---------------------------------
log2map = assert(loadfile("log2map.lua"))
------------------------------------
-- Emulating MAP_DIR env variable --
------------------------------------
local getenv_orig = os.getenv
os.getenv = function(name)
if name == "MAP_DIR" then
return map_dir
end
return getenv_orig(name)
end
------------------------
-- Redirecting stdout --
------------------------
assert(io.output(arg[3]))
-----------------------
-- Processing itself --
-----------------------
log2map(log_name)
As You can see it is slightly bigger ) Still for me such approach is more comfortable, because everything is clear and all aspects of code are accessible and changeable. No limitations of what can be done - absolutely everything. Which is wrong for shell. Some things are absolutely impossible to do with shell.
-
Core scripts have one huge advantage - they work properly ) I don't want to propose repair what's working right. For TinyCore phenomenal stability is absolutely unique. Some features are accessible only in TinyCore.
Wow, well said! Some TCL scripts are not very pretty but they work perfectly and I wouldn't touch them.
I use my TCL-powered laptop and wireless router daily and my overall impression of TCL is that, by minimizing size and complexity, it is the most stable and easy-to-understand GNU/Linux distribution that exists.
P.S. My *nix journey was Debian -> Arch Linux -> OpenBSD -> Devuan -> TCL. I still manage a Devuan machine in my household (wife's laptop) but I, personally, am 100% TCL. Once you're used to TCL, the complexity and size of other distros is just not acceptable.
-
The busybox ash shell includes local as one of its built-ins. Enter help in a terminal:
Thanks for explanations and examples, Rich! It works as expected. But in GNUsers example it was looking strange and I was not sure whether it works in shell the same way as in another languages. But GNUser wrote that it is not POSIX feature and for better portability it must not be relied upon.
Best regards!
-
Core scripts have one huge advantage - they work properly
I agree. Unfortunately it is sad to see that the latest commits on such scripts go back 10 or more years.
I'm not very familiar with Lua, but it's a surprisingly easy language. Unfortunately, the beginning is interesting but when the lua script ends it is noticeable that its syntax is no longer modern. The constant use of "ends" and other redundancies makes lua outdated.
In case you're curious, I started making a standalone version of tce-size (https://github.com/tinycorelinux/Core-scripts/blob/master/usr/bin/tce-size) in lua, as a demo. The LOC seems unnecessarily large, but keep in mind that there is no dependency. The script is also incomplete, but is partially working. You can test from https://www.jdoodle.com/execute-lua-online/ and inserting the package name in the argument.
function GetRemoteContent(url)
local isGzipped = url:sub(-3) == ".gz"
local wgetCmd = string.format("wget -q -O- %s", url)
if isGzipped then
wgetCmd = string.format("%s | gunzip", wgetCmd)
end
local handle = io.popen(wgetCmd)
local content = handle:read("*a")
handle:close()
if content and #content > 0 then
return content
else
print(string.format("Failed to download content from '%s'.", url))
return nil
end
end
function getFileSize(sizelistContent, filename)
for line in sizelistContent:gmatch("[^\r\n]+") do
local file, size = line:match("(%S+)%s+(%d+)")
if file == filename then
return tonumber(size)
end
end
return 0
end
function FileExist(filename)
local path = "/etc/sysconfig/tcedir/optional/" .. filename
local file = io.open(path, "r")
if file then
file:close()
return true
else
return false
end
end
local app = arg[1]
if not app then
print("Specify extension in command line:")
print("Example: /usr/bin/tce-size firefox.tcz.")
return
end
if not app:match("%.tcz$") then
app = app .. ".tcz"
end
local sizelistURL = "http://tinycorelinux.net/14.x/x86_64/tcz/sizelist.gz"
local treeURL = string.format("http://tinycorelinux.net/14.x/x86_64/tcz/%s.tree", app)
local sizelistContent = GetRemoteContent(sizelistURL)
local treeContent = GetRemoteContent(treeURL)
local totalSizeBytes = 0
local totalSizeNonExistentBytes = 0
local fileList = {}
for filename in treeContent:gmatch("%s*(.-)%s*\n") do
if filename:sub(-4) == ".tcz" then
local sizeBytes = getFileSize(sizelistContent, filename)
totalSizeBytes = totalSizeBytes + sizeBytes
table.insert(fileList, {filename = filename, sizeBytes = sizeBytes})
end
end
local function formatSizeMB(sizeBytes)
return string.format("%.2f MB", sizeBytes / (1024 * 1024))
end
table.sort(fileList, function(a, b) return a.sizeBytes > b.sizeBytes end)
print(" Filename Size (bytes), Size (MB)")
print("------------------------------------------------------------------")
for _, fileData in ipairs(fileList) do
local filename = fileData.filename
local sizeBytes = fileData.sizeBytes
local fileExists = FileExist(filename)
if not fileExists then
filename = "+" .. filename
totalSizeNonExistentBytes = totalSizeNonExistentBytes + sizeBytes
end
print(string.format(" %-40s %12d, %15s", filename, sizeBytes, formatSizeMB(sizeBytes)))
end
print("\n Total size (bytes) " .. string.format("%12d, %15s", totalSizeBytes, formatSizeMB(totalSizeBytes)))
print(" + Indicates need to download " .. string.format("%12d, %15s", totalSizeNonExistentBytes, formatSizeMB(totalSizeNonExistentBytes)))
-
Unfortunately it is sad to see that the latest commits on such scripts go back 10 or more years.
Why is it sad? Since the scripts still work perfectly on the current version of busybox ash (TCL's default shell), what would be the point of changing them? I think the age of the commits speaks to the quality of the scripts and the stability of TCL's goals and design.
-
Sorry, maybe I didn't make myself clear. My point is simply modernization. The discussion I propose is precisely the extent to which this modernization will violate tinycore's "keep everything small" principles.
I don't think that the long age of something functional is reason enough for it to stop in time.
Maybe I'm being a little extreme in making this comparison, but I can make Windows XP work perfectly.
-
Hm, how different approaches are! For me the code which is in the age of 10 years and works well is 10 times better than the code in age of 1 year ) Like a wine, I guess. The sign and proof that it was mastered perfectly.
By the way, CardealRusso, I completely don't understand what is modern syntax??? Probably it means that I'm outdated ) along with Lua )
Thanks for tce-size script! I will gladly test it later. And some Core scripts may be a little bit sluggish. As we've seen not long ago when GNUser proposed his version of update-everything script, which appeared to be 10 times faster, if I am not mistaken. Maybe some performance bottlenecks exist and may be made more comfortable at least.
-
TCL is stable, fast, small, understandable, easily extendable, runs on old machines, runs on new machines. Extensions (applications) are updated when necessary to fix breakage or provide security fixes. New applications become available when users find new uses for TCL and are generous enough to submit new extensions for the repository.
The above is what I care about. If the TCL infrastructure is able to support the above--and it is--then the age of the infrastructure makes little difference to me. (If anything, I'd say older infrastructure is better because it has had time to prove its stability. Would you rather drive on a 10 year-old bridge or on a 1 week-old bridge?)
But I speak only for myself, not for the project. I'm not a TCL developer, just a grateful user+contributor.
-
TCL is stable, fast, small, understandable, easily extendable, runs on old machines, runs on new machines.
I've made my point clear, now maybe it's time to make yours clear. You don't make clear what you are arguing against. I just suggested a modernization, but nothing specific. I initially suggested NIM, but soon abandoned this idea in favor of lua, which I also abandoned because I thought it was outdated.
As for lua, it is available in all TinyCore repositories, the package and its dependencies in total barely reach 1MB. What do you necessarily mean? That if TCL migrated to LUA, it would be done wrong and break the system?
That those 500KB would be too much to support? I need you to enlighten me so that I can also enlighten you. My point about the migration is a syntax that facilitates the contribution of other users.
Bash scripts do work but they are also extremely delicate, maybe that's why everyone is afraid to make any changes. While it's working, it's very easy to break a bash script with one or two changes.
The tce-update for example, I was trying to find a piece of code to ignore non-existent extensions from the repository, so that the update wouldn't be interrupted by things like "mylocale.tcz", but I got so lost that I gave up.
I completely don't understand what is modern syntax???
Perhaps this is subjective. By "modern" in this context of scripts and small programs, I necessarily mean you typing less to achieve the same without sacrificing code readability for future and third party maintenance.
It is clearly impossible to achieve all these points at once, but some manage to achieve this balance. Perhaps there are more, but of the ones I've tested, the ones that achieve this are: Python, Nim, and Node. Today I checked that Lua was on this list until recently, he clearly left recently.
-
Hi CardealRusso
... The constant use of "ends" and other redundancies makes lua outdated. ...
I'm not familiar with Lua programming, but assuming the syntax
you provided is correct, I would describe it as:
The consistent use of "ends".
Don't all languages have some way of marking where a function ends?
How about where if, while, for, case, and other conditional clauses end?
Ash/bash uses }, fi, done, done, and esac for these.
C uses } but allows ; for one liners.
-
It is clearly impossible to achieve all these points at once, but some manage to achieve this balance. Perhaps there are more, but of the ones I've tested, the ones that achieve this are: Python, Nim, and Node. Today I checked that Lua was on this list until recently, he clearly left recently.
What is Node? Is this the language name?
-
Don't all languages have some way of marking where a function ends?
Certainly. I assume it's a must.... That can be anything from "e n d" to a simple "<return>" or, actually, nothing (assuming it already has a line break)
What is Node? Is this the language name?
Sorry, are you asking a genuine question or being sarcastic?
Assuming this is a genuine question, anyone who has mastered the basics of JS understands that interpreters are easier to speak than JS itself. After all, if I suggest the JS, I have to immediately suggest the interpreter...
Which could be the 1mb QuickJS or the 40mb node (plus many others of course, this was just to demonstrate the difference in size)
Assuming you're being sarcastic:
(https://i.imgur.com/ev8GtO8.png)
-
No, I was not sarcastic. Don't do too many assumptions.
-
I've made my point clear, now maybe it's time to make yours clear. You don't make clear what you are arguing against.
I'm arguing against making drastic changes to TCL's infrastructure because of potential stability issues. As it is (i.e., powered by shell scripts including some that are a decade old), TCL is stable, does absolutely everything I need it to do (and well), and has qualities that make it the only distro that I want to use and contribute to.
The TCL developers do not discriminate against new technologies and will accept your contributions if they are good. Good luck!
I don't have anything else to add to this discussion. Happy hacking!
-
Hi CardealRusso
Don't all languages have some way of marking where a function ends?
Certainly. I assume it's a must.... That can be anything from "e n d" ...
Didn't you just criticize The constant use of "ends" in a previous post?
... to a simple "<return>" or, actually, nothing (assuming it already has a line break) ...
I take it you mean a blank line. That would mean no blank lines inside
of functions, and for or while loops, blocks of code in if statements, etc.
-
Didn't you just criticize The constant use of "ends" in a previous post?
Hello Rich.
Please allow me to explain what I'm referring to with some examples.
Python:
for a in range(3):
for b in range(3):
for c in range(3):
for d in range(3):
println("Hello, World!")
Nim:
for a in 0..3:
for b in 0..3:
for c in 0..3:
for d in 0..3:
echo "Hello, World!"
Lua:
for a = 1, 3 do
for b = 1, 3 do
for c = 1, 3 do
for d = 1, 3 do
println("Hello, World!")
end
end
end
end
But that's just a point I make, I don't mean that Lua is totally bad for that. In fact, it looks like it has the potential to be the best option, as you have to follow some basic rules of "script yes. not the compiled language" as well as a small binary.
Following these limitations, it seems to me that the second viable option would be QuickJS (which, by the way, the package present in the x64 repository is badly organized, since the libraries are not dependencies of the binaries, there should be "-bin" with qjs only, "-libs" with libraries and "-tools" with qjsc)
-
Hi CardealRusso
I see. So rather than using braces or keywords, they
determine scope through indentation.
I prefer properly indented braces or keywords on their
own lines myself. I don't mind a little extra typing.
-
I'm arguing against making drastic changes to TCL's infrastructure because of potential stability issues. As it is
>(i.e., powered by shell scripts including some that are a decade old),<
TCL is stable, does absolutely everything I need it to do (and well), and has qualities that make it the only distro that I want to use and contribute to.
wrt above quote spesificly bold section i was reminded of a post on the web titled "More shell, less egg"
previously mentioned @ https://forum.tinycorelinux.net/index.php/topic,22762.msg142399.html
http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/ || https://web.archive.org/web/20130209012208/http://www.leancrew.com/all-this/2011/12/more-shell-less-egg/
Knuth wrote his program in WEB, a literate programming system of his own devising that used Pascal as its programming language.
His program used a clever, purpose-built data structure for keeping track of the words and frequency counts; and the article interleaved with it presented the program lucidly.
McIlroy’s review started with an appreciation of Knuth’s presentation and the literate programming technique in general.
He discussed the cleverness of the data structure and Knuth’s implementation,
pointed out a bug or two, and made suggestions as to how the article could be improved.
And then he calmly and clearly eviscerated the very foundation of Knuth’s program.
What people remember about his review is that McIlroy wrote a six-command shell pipeline
that was a complete (and bug-free) replacement for Knuth’s 10+ pages of Pascal.
-
Hi mocore,
You propose the McIlroy's script as an example of the true formatting? Then my initial script is absolutely ok ;-)
-
For those using piCore, micropython is available for scripting.
The adaptation of "indentation languages" is incredible frustrating at first. It also will start the argument:
Which is correct
1) Indent with tabs
2) Indent with spaces
I'm a <tab> indenter, with my editor set for a tabsize of 4.
-
For those using piCore, micropython is available for scripting.
I haven't found it among arm tczs. It is included in the rootfs?
-
Hi jazzbiker
It is included in the rootfs as /usr/bin/micropython.
-
Hi Paul_123
... I'm a <tab> indenter, with my editor set for a tabsize of 4.
Same here.
-
micropython is available for scripting.
The package is 300kb. This is amazing.
Jesus Christ, really amazing. I wonder why it is not present in the other repository. I've just created the 64-bit tcz and will be sending out the email.
https://i.imgur.com/6K45sci.png
Hi jazzbiker
It is included in the rootfs as /usr/bin/micropython.
Could be included in the others ports.
-
Hi jazzbiker
It is included in the rootfs as /usr/bin/micropython.
Thanks, Rich!
It is bigger than Lua.
-
Depends on the features frozen into the main binary, and libraries included. Micropython is even getting asyncio support.
Since I use Micropython/Circuitpython on baremetal too, I choose to stay python.
-
Depends on the features frozen into the main binary, and libraries included. Micropython is even getting asyncio support.
after reading mention of micro python on this topic
i hapened to scrool past geophile/marcel:"A modern shell " @ https://github.com/geophile/marcel#scripting
marcel provides a Python API, allowing Python to be used as the scripting language.
While Python is sometimes considered to already be a scripting language, it isn't really.
Executing shell commands from Python code is cumbersome.
You've got to use os.system, or subprocess.Popen, and write some additional code to do the integration.
Marcel provides a Python module, marcel.api, which brings shell commands into Python in a much cleaner way.
For example, to list file names and sizes in /home/jao:
from marcel.api import *
for file, size in ls('/home/jao') | map(lambda f: (f, f.size)):
print(f'{file.name}: {size}')
intresting
-
Hi jazzbiker
It is included in the rootfs as /usr/bin/micropython.
piCore has used micropython during the boot process for many years.
/usr/bin/tce-bootload is a micropython script called by tce-setup.
I think this was changed because, on the original Raspberry Pi's, this was the biggest bottleneck in the boot process.
-
Hi jazzbiker
It is included in the rootfs as /usr/bin/micropython.
piCore has used micropython during the boot process for many years.
/usr/bin/tce-bootload is a micropython script called by tce-setup.
I think this was changed because, on the original Raspberry Pi's, this was the biggest bottleneck in the boot process.
Hi Greg Erskine,
Thanks for the information. piCore's /usr/bin/tce-bootload actively uses os.fork() and os.waitforpid(). Definitely micropython's os module gives better control over background processes than shell.
I don't write python, please correct me if I am wrong - os module is imported and it means that it is not compiled into the micropython binary, right?
-
Yes, os is an external library from here https://github.com/micropython/micropython-lib We only include a few of the micropython libs. It’s enough for basic scripting.
Mounting extensions is a heavy IO process, but it does significantly improve the time loading extensions during boot by forking up to the number of cpu cores. Micropython support for python’s threading module is still fairly new and limited.
-
The adaptation of "indentation languages"
Same here.
Hi. I will not insist on such changes anymore as it is clear that the disapproval is due to the learning curve and middle age effects; Regardless of Lua or Python (and I would not like to take that responsibility if it was assigned to me). I would just really like to know why not integrate micropython with the other ports for symmetry purposes?
Although not insisting, I would like to leave some comparisons I made before giving up that may be interesting; First, Lua is extremely fast.
I honestly don't know if this speed would benefit TinyCore in any way, but in basic synthetic tests, lua managed, again, to surprise me.
One such test I did was a counter up to 1 billion; C finished in 2 seconds, Lua in 9, Micropython (custom) 47 seconds, Micropython (standard) 1:24, Python took about 3 minutes and bash never finished (I canceled about 5 minutes in because I thought it would never finish). https://pastebin.com/raw/DmEQB95d
About MicroPython, I started doing, again, what supposedly could be something like tce-size. I had a little knowledge about python, and in these last hours I learned a lot. Here is an (incomplete) example of tce-size for python: https://pastebin.com/7wLZDmVa
As I mentioned before, the idea is to use a programming language that can be more easily "mastered" by newbies, so that way, community development can maybe move along a bit more. And with that I allow myself to analyze small portions of the three codes (fragmented and unordered, which might look a little messy)
Getting the list and the tree.
SH:
tce-size:
tce-fetch.sh sizelist.gz || exit 1
touch sizelist
tce-fetch.sh "$app".tree >/dev/null 2>&1 || echo "$app" > "$app".tree
tce-fetch.sh:
if [ "$1" == "-O" ]; then
shift
wget -cq -O- "$MIRROR"/"${1//-KERNEL.tcz/-${KERNELVER}.tcz}" <-- (I confess that I still don't understand how this works)
else
F="${1//-KERNEL.tcz/-${KERNELVER}.tcz}"
[ -f "$F" ] && rm -f "$F"
wget -cq "$MIRROR"/"$F"
fiMPY:
def GetRemoteTextFile(filename):
response = urequests.get(f"{BuildMirror()}/tcz/{filename}")
if response.status_code == 200:
return response.text
else:
print(f"Error downloading file {filename}. Status code: {response.status_code}")
sys.exit(1)
...
sizelist = GetRemoteTextFile("sizelist")
packageTree = GetRemoteTextFile(f"{package}.tree")
Dealing with the .tcz or lack thereof in the package name:
sh:
app=${app%.tcz}
app="$app.tcz"
MPY:
if not package.endswith(".tcz"):
package += ".tcz"
Handling dependencies:
SH:
for F in `sed 's/ //g' "$app".tree | sort -f | uniq`; do
size=`grep -w "^$F" sizelist | awk '{print $2}'`
case $size in
[0-9]*)
sizemb=`Fixed3Div $size $M`
if [ -f "$localtce"/optional/"$F" ]; then
totalsize_installed=$(($totalsize_installed + $size))
echo -n " "
printf "%-40s" $F
printf " %10d, %6.2f MB\n" $size $sizemb
else
echo -n "+ "
totalsize_needed=$(($totalsize_needed + $size))
printf "%-40s %10d, %6.2f MB\n" $F $size $sizemb
fi
;;
*)
printf "%-40s Error, not found \n" $F
;;
esac
done
}
MPY (incomplete):
# Create a dictionary mapping the file name to its size in bytes
size_dict = {line.split()[0]: int(line.split()[1]) for line in sizelist.splitlines()}
# Print the results directly from the list comprehension
print("\n".join(f"{filename:<40} {size_in_bytes:<13} {bytes_to_mb(size_in_bytes):<8.2f}" for filename, size_in_bytes in size_dict.items() if filename in packageTree))
It could be simpler, remembering the lua:
for treeline in packageTree.splitlines():
for sizeline in sizelist.splitlines():
currentSizeLine = sizeline.split(" ")
if treeline.strip() == currentSizeLine[0]:
filename = currentSizeLine[0]
size_in_bytes = int(currentSizeLine[1])
size_in_mb = convert_bytes_to_mb(size_in_bytes)
print(f"{filename:<40} {size_in_bytes:<13} {size_in_mb:<8}")
Just replacing shelling outs with built-in functions, as well as not using regex (or something similar to regex) makes the code easier to understand. This way, half of the code is understandable just by reading it, eliminating (not much) the need to understand things like "\" or "\\" or "\\\" or "\\\\/////" or maybe "||||||||" or rather, why not "\o/\o/\o/\o/\o/"
-
I will not insist on such changes anymore...I would just really like to know why not integrate micropython with the other ports for symmetry purposes?
Hi CardealRusso. This is easy to answer. It's because no user has stepped up to submit a micropython extension for the other ports. Many users insist on changes and ask for explanations, but few submit extensions and create pull requests.
it is clear that the disapproval is due ... middle age effects
Hi Rich. Do the forum rules allow this?
-
submit a micropython extension for the other ports.
HI. Apparently there is no micropython extension in any version of tinycore, it's included directly in rootfs. as mentioned in
It is included in the rootfs as /usr/bin/micropython.
Hi Rich. Do the forum rules allow this?
Sorry if I sounded rude. In my culture, old age is impossible to be used as any form of offense, it's a blessing to reach such an age and be healthy.
We believe, however, that there is a way of acting that is inherited long before we became homosapiens; Old people don't learn, they teach.
It does not mean that old people are incapable of learning, but that they are simply less likely to learn because they feel they are in the position of life teachers.
-
Hi GNUser
... it is clear that the disapproval is due ... middle age effects
Hi Rich. Do the forum rules allow this?
This was obviously not directed at me since at 64 I'm well
past middle aged, so I'll allow it. ;D
-
Haha. Touché!
It seems beyond middle age it is possible to have both smarts and charm ;)
-
Hi CardealRusso
... Old people don't learn, they teach. ...
I try to do both. See reply #27. Two days ago I knew nothing
about Python. So I did a search on Python syntax and learned
it uses indentation to determine scope. :o
Seriously, I learn new things all of the time. If I didn't, I wouldn't
be able to answer some of the questions that get ask.
-
Hi Core guys,
Thanks You all, discussion was quite meaningful - at least for me. I've got:
1. Nice advices on making shell scripts more readable.
2. piCore secrets unveiled.
3. Closer view of micropython - it appeared to be the very nice tool for system scripting. With proper modules especially.
4. Ability to distinguish outdated programming languages from uptodate ones and knowledge of the modern syntax.
@Rich, I still haven't caught what are the middle age problems, but I want to ask You to mark this thread as solved.
-
Hi jazzbiker
... Rich, I still haven't caught what are the middle age problems, ...
Start at reply #42:
https://forum.tinycorelinux.net/index.php/topic,26332.msg169759.html#msg169759
and read the 4 posts that follow.
... mark this thread as solved.
Done.