WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: [Solved] Asking for recomendations on the shell script formatting.  (Read 10202 times)

Online Paul_123

  • Administrator
  • Hero Member
  • *****
  • Posts: 1270
Re: Asking for recomendations on the shell script formatting.
« Reply #30 on: July 20, 2023, 10:30:45 AM »
For those using piCore, micropython is available for scripting.

The adaptation of "indentation languages" is incredible frustrating at first.   It also will start the argument:

Which is correct
1) Indent with tabs
2) Indent with spaces

I'm a <tab> indenter, with my editor set for a tabsize of 4.




Offline jazzbiker

  • Hero Member
  • *****
  • Posts: 934
Re: Asking for recomendations on the shell script formatting.
« Reply #31 on: July 20, 2023, 10:42:12 AM »
For those using piCore, micropython is available for scripting.

I haven't found it among arm tczs. It is included in the rootfs?

Online Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11705
Re: Asking for recomendations on the shell script formatting.
« Reply #32 on: July 20, 2023, 11:28:02 AM »
Hi jazzbiker
It is included in the rootfs as  /usr/bin/micropython.

Online Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11705
Re: Asking for recomendations on the shell script formatting.
« Reply #33 on: July 20, 2023, 11:31:07 AM »
Hi Paul_123
... I'm a <tab> indenter, with my editor set for a tabsize of 4.
Same here.

Offline CardealRusso

  • Full Member
  • ***
  • Posts: 181
Re: Asking for recomendations on the shell script formatting.
« Reply #34 on: July 20, 2023, 11:38:17 AM »
micropython is available for scripting.
The package is 300kb. This is amazing.
Jesus Christ, really amazing. I wonder why it is not present in the other repository. I've just created the 64-bit tcz and will be sending out the email.
https://i.imgur.com/6K45sci.png

Hi jazzbiker
It is included in the rootfs as  /usr/bin/micropython.
Could be included in the others ports.
« Last Edit: July 20, 2023, 11:54:21 AM by CardealRusso »

Offline jazzbiker

  • Hero Member
  • *****
  • Posts: 934
Re: Asking for recomendations on the shell script formatting.
« Reply #35 on: July 20, 2023, 11:50:37 AM »
Hi jazzbiker
It is included in the rootfs as  /usr/bin/micropython.
Thanks, Rich!
It is bigger than Lua.

Online Paul_123

  • Administrator
  • Hero Member
  • *****
  • Posts: 1270
Re: Asking for recomendations on the shell script formatting.
« Reply #36 on: July 20, 2023, 01:24:38 PM »
Depends on the features frozen into the main binary, and libraries included.  Micropython is even getting asyncio support.

Since I use Micropython/Circuitpython on baremetal too, I choose to stay python.



Offline mocore

  • Hero Member
  • *****
  • Posts: 672
  • ~.~
Re: Asking for recomendations on the shell script formatting.
« Reply #37 on: July 20, 2023, 02:02:37 PM »
Depends on the features frozen into the main binary, and libraries included.  Micropython is even getting asyncio support.

after reading mention of micro python on this topic
i hapened to scrool past geophile/marcel:"A modern shell "  @ https://github.com/geophile/marcel#scripting



Quote
marcel provides a Python API, allowing Python to be used as the scripting language.
While Python is sometimes considered to already be a scripting language, it isn't really.
Executing shell commands from Python code is cumbersome.
You've got to use os.system, or subprocess.Popen, and write some additional code to do the integration.

Marcel provides a Python module, marcel.api, which brings shell commands into Python in a much cleaner way.
For example, to list file names and sizes in /home/jao:
Code: [Select]
from marcel.api import *

for file, size in ls('/home/jao') | map(lambda f: (f, f.size)):
    print(f'{file.name}: {size}')

intresting

Offline Greg Erskine

  • Sr. Member
  • ****
  • Posts: 404
Re: Asking for recomendations on the shell script formatting.
« Reply #38 on: July 20, 2023, 05:41:28 PM »
Hi jazzbiker
It is included in the rootfs as  /usr/bin/micropython.

piCore has used micropython during the boot process for many years.

/usr/bin/tce-bootload is a micropython script called by tce-setup.

I think this was changed because, on the original Raspberry Pi's, this was the biggest bottleneck in the boot process.

Offline jazzbiker

  • Hero Member
  • *****
  • Posts: 934
Re: Asking for recomendations on the shell script formatting.
« Reply #39 on: July 20, 2023, 09:37:40 PM »
Hi jazzbiker
It is included in the rootfs as  /usr/bin/micropython.

piCore has used micropython during the boot process for many years.

/usr/bin/tce-bootload is a micropython script called by tce-setup.

I think this was changed because, on the original Raspberry Pi's, this was the biggest bottleneck in the boot process.

Hi Greg Erskine,

Thanks for the information. piCore's /usr/bin/tce-bootload actively uses os.fork() and os.waitforpid(). Definitely micropython's os module gives better control over background processes than shell.
I don't write python, please correct me if I am wrong - os module is imported and it means that it is not compiled into the micropython binary, right?

Online Paul_123

  • Administrator
  • Hero Member
  • *****
  • Posts: 1270
Re: Asking for recomendations on the shell script formatting.
« Reply #40 on: July 20, 2023, 10:00:15 PM »
Yes, os is an external library from here https://github.com/micropython/micropython-lib  We only include a few of the micropython libs. It’s enough for basic scripting.

Mounting extensions is a heavy IO process, but it does significantly improve the time loading extensions during boot by forking up to the number of cpu cores.   Micropython support for python’s threading module is still fairly new and limited.

Offline CardealRusso

  • Full Member
  • ***
  • Posts: 181
Re: Asking for recomendations on the shell script formatting.
« Reply #41 on: July 21, 2023, 05:25:14 PM »
The adaptation of "indentation languages"
Same here.
Hi. I will not insist on such changes anymore as it is clear that the disapproval is due to the learning curve and middle age effects; Regardless of Lua or Python (and I would not like to take that responsibility if it was assigned to me). I would just really like to know why not integrate micropython with the other ports for symmetry purposes?

Although not insisting, I would like to leave some comparisons I made before giving up that may be interesting; First, Lua is extremely fast.
I honestly don't know if this speed would benefit TinyCore in any way, but in basic synthetic tests, lua managed, again, to surprise me.
One such test I did was a counter up to 1 billion; C finished in 2 seconds, Lua in 9, Micropython (custom) 47 seconds, Micropython (standard) 1:24, Python took about 3 minutes and bash never finished (I canceled about 5 minutes in because I thought it would never finish). https://pastebin.com/raw/DmEQB95d

About MicroPython, I started doing, again, what supposedly could be something like tce-size. I had a little knowledge about python,  and in these last hours I learned a lot. Here is an (incomplete) example of tce-size for python:  https://pastebin.com/7wLZDmVa
As I mentioned before, the idea is to use a programming language that can be more easily "mastered" by newbies, so that way, community development can maybe move along a bit more. And with that I allow myself to analyze small portions of the three codes (fragmented and unordered, which might look a little messy)

Getting the list and the tree.
SH:
Code: [Select]
tce-size:
tce-fetch.sh sizelist.gz || exit 1
touch sizelist
tce-fetch.sh "$app".tree >/dev/null 2>&1 || echo "$app" > "$app".tree
tce-fetch.sh:
if [ "$1" == "-O" ]; then
shift
wget -cq -O- "$MIRROR"/"${1//-KERNEL.tcz/-${KERNELVER}.tcz}" <-- (I confess that I still don't understand how this works)
else
F="${1//-KERNEL.tcz/-${KERNELVER}.tcz}"
[ -f "$F" ] && rm -f "$F"
wget -cq "$MIRROR"/"$F"
fi
MPY:
Code: [Select]
def GetRemoteTextFile(filename):
    response = urequests.get(f"{BuildMirror()}/tcz/{filename}")
    if response.status_code == 200:
        return response.text
    else:
        print(f"Error downloading file {filename}. Status code: {response.status_code}")
        sys.exit(1)
...
sizelist = GetRemoteTextFile("sizelist")
packageTree = GetRemoteTextFile(f"{package}.tree")
Dealing with the .tcz or lack thereof in the package name:
sh:
Code: [Select]
   app=${app%.tcz}
   app="$app.tcz"
MPY:
Code: [Select]
if not package.endswith(".tcz"):
    package += ".tcz"

Handling dependencies:
SH:
Code: [Select]
for F in `sed 's/ //g' "$app".tree | sort -f | uniq`; do
      size=`grep -w "^$F" sizelist  | awk '{print $2}'`
      case $size in
      [0-9]*)
sizemb=`Fixed3Div $size $M`
if [ -f "$localtce"/optional/"$F" ]; then
totalsize_installed=$(($totalsize_installed + $size))
echo -n "  "
printf "%-40s" $F
printf " %10d, %6.2f MB\n" $size $sizemb
else
echo -n "+ "
totalsize_needed=$(($totalsize_needed + $size))
printf "%-40s %10d, %6.2f MB\n" $F $size $sizemb
fi
      ;;
      *)
printf "%-40s Error, not found \n" $F
      ;;
      esac
done
}

MPY (incomplete):
Code: [Select]
# Create a dictionary mapping the file name to its size in bytes
size_dict = {line.split()[0]: int(line.split()[1]) for line in sizelist.splitlines()}
 
# Print the results directly from the list comprehension
print("\n".join(f"{filename:<40} {size_in_bytes:<13} {bytes_to_mb(size_in_bytes):<8.2f}" for filename, size_in_bytes in size_dict.items() if filename in packageTree))
It could be simpler, remembering the lua:
Code: [Select]
for treeline in packageTree.splitlines():
    for sizeline in sizelist.splitlines():
        currentSizeLine = sizeline.split(" ")
        if treeline.strip() == currentSizeLine[0]:
            filename = currentSizeLine[0]
            size_in_bytes = int(currentSizeLine[1])
            size_in_mb = convert_bytes_to_mb(size_in_bytes)
            print(f"{filename:<40} {size_in_bytes:<13} {size_in_mb:<8}")

Just replacing shelling outs with built-in functions, as well as not using regex (or something similar to regex) makes the code easier to understand. This way, half of the code is understandable just by reading it, eliminating (not much) the need to understand things like "\" or "\\" or "\\\" or "\\\\/////" or maybe "||||||||" or rather, why not "\o/\o/\o/\o/\o/"

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1531
Re: Asking for recomendations on the shell script formatting.
« Reply #42 on: July 21, 2023, 07:10:34 PM »
I will not insist on such changes anymore...I would just really like to know why not integrate micropython with the other ports for symmetry purposes?
Hi CardealRusso. This is easy to answer. It's because no user has stepped up to submit a micropython extension for the other ports. Many users insist on changes and ask for explanations, but few submit extensions and create pull requests.

it is clear that the disapproval is due ... middle age effects
Hi Rich. Do the forum rules allow this?
« Last Edit: July 21, 2023, 07:17:28 PM by GNUser »

Offline CardealRusso

  • Full Member
  • ***
  • Posts: 181
Re: Asking for recomendations on the shell script formatting.
« Reply #43 on: July 21, 2023, 07:31:29 PM »
submit a micropython extension for the other ports.
HI. Apparently there is no micropython extension in any version of tinycore, it's included directly in rootfs. as mentioned in
It is included in the rootfs as  /usr/bin/micropython.

Hi Rich. Do the forum rules allow this?
Sorry if I sounded rude. In my culture, old age is impossible to be used as any form of offense, it's a blessing to reach such an age and be healthy.

We believe, however, that there is a way of acting that is inherited long before we became homosapiens; Old people don't learn, they teach.

It does not mean that old people are incapable of learning, but that they are simply less likely to learn because they feel they are in the position of life teachers.

Online Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11705
Re: Asking for recomendations on the shell script formatting.
« Reply #44 on: July 21, 2023, 08:08:18 PM »
Hi GNUser
...
it is clear that the disapproval is due ... middle age effects
Hi Rich. Do the forum rules allow this?
This was obviously not directed at me since at 64 I'm well
past middle aged, so I'll allow it.  ;D