tce with comments + explicit path busybox applets, in future Tc14?

[nick65go]

Hi, /usr/bin/tce depends on scripts /usr/bin/{search.sh, provides.sh, select, tce-size}.
I explicitly replaced all the busybox commands with their full path (/bin, /usr/bin), by that I eliminated the need for

Code: [Select]

. /etc/init.d/tc-functions && useBusybox
There is now no need to call /source tc-config (18497 bytes) by 4 times;
yes, I know was already in RAM, but I least I documented who call whom, which minimum busybox applets are in need for each script, etc. The speed could not be lower than original scripts.

The new scripts remain very small (less than 4KB -- as page cache): tce (2753 Bytes), tce-size (2499 bytes), select (1353 bytes), provides.sh (945 bytes), search.sh (921 bytes).

Also I commented each script, in its top, with which busybox was explicitly used. Sample:

Code: [Select]

# /usr/bin/tce call scripts from /usr/bin/{search.sh, provides.sh, select, tce-size}
# only used: /usr/bin/id, /bin/mktemp

# /usr/bin/search.sh only used: /bin/gunzip, /bin/date, /bin/awk

# provides.sh only used tc-functions:getMirror, /usr/bin/wget, /usr/bin/zsync, /bin/awk

# /usr/bin/select only used /bin/awk

# /usr/bin/tce-size only used: /bin/sed, /usr/bin/sort, /usr/bin/uniq,
# /bin/grep, /usr/bin/awk, /bin/gunzip, /bin/date, /bin/touch
Is any interest to have something like these (comments + explicit path busybox applets) in future Tc14?

EDIT: for provides.sh I did a substitution like:

Code: [Select]

# getMirror from tc-functions
getBuild() {
ARCH=`uname -m`
case ${ARCH} in
    armv6l) echo "armv6" ;;
    armv7l) echo "armv7" ;;
    aarch64) echo "aarch64" ;;
    i686)   echo "x86" ;;
    x86_64) [ -f /lib/ld-linux-x86-64.so.2 ] && echo "x86_64" || echo "x86" ;;
    *)      echo "x86" ;;
esac
}
BUILD=$(getBuild)

verid=$(grep '^VERSION_ID=' /etc/os-release)
verid=${verid#*=}
getMajorVer=${verid%.*}

read MIRROR < /opt/tcemirror
MIRROR="${MIRROR%/}/$(getMajorVer).x/$BUILD/tcz"and for tce one like:

Code: [Select]

# checknotroot from tc-functions
 if [ `/usr/bin/id -u` -eq 0 ]; then
   echo "Don't run this as root." >&2
   exit 1
 fito avoid full tc-config sourcing for just few bytes functions.

[hiro]

the point of PATH is to not burden a programmer or user with absolute paths.
i'd rather make the computer do something useful for me than micromanage such things.

[nick65go]

@Hiro: Agree it! I know the role of PATH.
The point of the exercise (learning / fun) was to understand the main logic, why few of busybox functions would need to be sourced separably by usebusybox(). I think even GNU tools will behave similar, taken into account these functions do not use many parameters (one or two) in these specific scripts.

These TC scripts are 2 .. 10 years old, and will stay like this for many years maybe. Then, there are only few scripts (less than 30?) so will be one time job, to do not need reverse-engineering /understand it again, without documentation.
PS: the specific TC scripts make TC unique. So "porting" them to any other linux base, can be beneficial, to quickly grab (from a mirror) any tcz+deps and run them, from any arbitrary folders, etc.

[hiro]

well, you have to draw a line somewhere.

whether it's complexity in the code, numer of lines, long lines, wrapped lines, documentation, long names, absolute names, unneeded verbosity, syntactical noise, they all comes with a cost. you need a balance.

every extra character needs a little extra time for a human to parse (even if it's the same for a computer).

i think this goes too far. but that's just my opinion

[GNUser]

to quickly grab (from a mirror) any tcz+deps

Hi nick65go. For this specific task, Rich wrote a nice script that can be used on any linux distro. The script is called FetchExt.sh and can be downloaded from here.

[nick65go]

Thank you GNUser for info and Rich for script, I will look at it.

Like Hiro said, I have my balance, I draw the line (in the sand, but it evolves) about linux.
The 90% of my "needs" are covered by with fat-pigs like Xorg-3D + Firefox + LibreOffice + Vlc; and some light-weight tools (7Zip + Xfe + flwm + aterm). OK, I could  shrink a little to gnumeric + abiword +mpv/mplayer, etc.

Then my focus shifted to security, which in never guaranteed, increase the cost, lower the speed execution.

So here began the try with VM (qemu) and low resources. Like take a "secured" kernel + modules (Alpinelinux, Ubuntu), add a [static] shell (toybox, busybox), a "compatible" libc / musl, and then happy use whatever apps are available as a shameless consumer.

One small problem was the package-manager for recursive dependencies. Most are [fast] pigs, or drag useless huge dependencies. Without care the final result will be  as bloated as a win10. Purpose defeated as complexity means less security.

So, the new [naive] idea is to create a Virtual machine on-the-fly [all resources/pkg/tcz/deb are already present in the host, compacted], like Xen but much lighter, or like a single app restricted in a pseudo-container when internet connection is need [f**king Firefox piggy, I am looking at damn you]. So here is one of the TC strong points: applications with light dependency.

[hiro]

"So here began the try with VM (qemu) and low resources. Like take a "secured" kernel + modules (Alpinelinux, Ubuntu)"

why not run qemu on tc, it seems more lightweight than alpine or ubuntu. i'm sure ubuntu is less secure bec. of all the preinstalled bloat.

"So, the new [naive] idea is to create a Virtual machine on-the-fly [all resources/pkg/tcz/deb are already present in the host, compacted], like Xen but much lighter, or like a single app restricted in a pseudo-container when internet connection is need [f**king Firefox piggy, I am looking at damn you]. So here is one of the TC strong points: applications with light dependency."

i'm not sure i understand. i guess you're making linux namespaces manually on tc?

[nick65go]

@Hiro: To clarify, I indeed run qemu from TC, for now [trust the trustee].

 -yes, the kernel is now from tc14 (in host and in guest machines), has all virtio drv. I used kernel from Alpinelinux to see all virtio drv dependency [lsmod /modprobe in guest] because they split out every possible module. And modules are SHA signed [yeah, with alpine keys, seen with modinfo]. The idea for a future guest-kernel build with ONLY the modules needed in qemu (size down from 5-7 MB to 2-3 MB ?). Toybox has one kernel ~= 2MB, but with old-drv.

- Ubuntu kernel only for host, in stand-by, tempting because has M$ secure-boot keys [signed].

- I did not understand very well Xen from wikipedia, or from their site/documentation; but I fetched the idea of using "templates" (like qemu hdd /snapshots) to build VMs on-the-fly. Plus it needs some CPU flags which my APU maybe does not have etc.

- Long story make short, I do not have much experience / knowledge with kernel (even after reading 100 of pages, too much info, maybe less brain / time / motivation).

-the virtual container LCX, docker, whatever seams complicated (more resource /libs, PAM kernel modules etc) versus layman qemu and/or  chroot.
 

[hiro]

The idea for a future guest-kernel build with ONLY the modules needed in qemu (size down from 5-7 MB to 2-3 MB ?). Toybox has one kernel ~= 2MB, but with old-drv.

so you're thinking of excluding unused modules for security?

- Ubuntu kernel only for host, in stand-by, tempting because has M$ secure-boot keys [signed].

so you want to allow microsoft to control what punk operating system you're allowed to boot? if you care about your privacy i wouldn't trust microsoft with anything. don't let them control you.

- I did not understand very well Xen from wikipedia, or from their site/documentation; but I fetched the idea of using "templates" (like qemu hdd /snapshots) to build VMs on-the-fly. Plus it needs some CPU flags which my APU maybe does not have etc.

The main thing to note about Xen is that now is that in practice it's a Type2 Hypervisor like everything else now.
The main value is actually XCP-NG's management layer and GUI (XOA). It helps a lot if you have a whole fleet of hypervisors and need to remote-manage, migrate, failover, the VMs on them. Very good even for less experienced people or who doesn't understand the complicated qemu commandline syntax or the libvirt XML configuration complexities.

- Long story make short, I do not have much experience / knowledge with kernel (even after reading 100 of pages, too much info, maybe less brain / time / motivation).

maybe just browse through all those options a bit in make menuconfig to get an overview of all the modules and where they are in that hierarchy.

-the virtual container LCX, docker, whatever seams complicated (more resource /libs, PAM kernel modules etc) versus layman qemu and/or  chroot.

Here I fully agree, virtualization is much more involved on those lower layers, but it's such a clean abstraction that in practice it's much much simpler to manage. Both as developer and as user.
And bec. of some cpu extensions like vt-x and also paravirtualization it's not even much slower.
It's also easier to get right, but there's no way to eve get it properly secure, there's always side-channels no matter what you do.

[nick65go]

@Hiro: Hi, thanks for the feed-back!
1. H: "so you're thinking of excluding unused modules for security?"
 Yes, simplicity (eliminating not-used things) for me means less attack surface [less bugs], so more security.

2. H: "i wouldn't trust Microsoft with anything. don't let them control you"
 Yes, fully agree! May I be forgiven, please; errare humanum est.

3. H: "if you have a whole fleet of hypervisors.."
 NO. I am just a simple person with a computer, fooling around. I can understand the complicated qemu command-line syntax.

4. H: "there's no way to even get it properly secure".
 Right! So simplification again, qemu stays. Too much (to learn /test) on my plates, for the time been.It is supposed to be fun, not part-time job.
As primitive protection I keep my private files on separate partition, hopping to not be mounted/ read by not-authorized software.

[hiro]

Yes, simplicity (eliminating not-used things) for me means less attack surface [less bugs], so more security.

generally, i agreed.
whether it's effective and worth it for kernel modules specifically, i'm just 50/50 about that. practically my lazyness decides in this draw...

NO. I am just a simple person with a computer, fooling around. I can understand the complicated qemu command-line syntax.

then i see no benefit in xen. if you know qemu stick with that, there's no reason to switch to anything else.

As primitive protection I keep my private files on separate partition, hopping to not be mounted/ read by not-authorized software.

yep! i do the same.

like, safety is somewhat related also to the duration of the exposure. hard to beat the security gains if you can afford unmounting, then unloading the crypto keys, or even better, physically disconnecting a disk. and only connecting it in the rare case where you need it.

if you have *multiple* disks, you can connect just a small subset of disks at a time. the more granular you go with rarely used data, the more secure is everything else (because it raises the probability that it's unavailable during an attack).

not much computer engineering involved. just end-user behavior.