Tiny Core Linux

Tiny Core Base => TCB Talk => Topic started by: TT on January 04, 2021, 06:02:56 PM

Title: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 04, 2021, 06:02:56 PM
How tc-config could be modified, so it loads packages online from locally defined httplist?

Kinda httplist=file:///… whereas file:///… has a content of: 192.168.0.1/…

Or would it be smarter to introduce a 'filelist' parameter to get_app with "FILE" following?
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: andyj on January 04, 2021, 07:37:24 PM
How would this be different than onboot.lst? You can use the lst= boot command parameter to use a different list file than onboot.lst.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 05, 2021, 04:16:10 AM
Uhm-m-m, andyj… Have you read, actually, what my posting was about?
whereas file:///… has a content of: 192.168.0.1/…
How on earth could you have THAT in onboot.lst?
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: andyj on January 05, 2021, 08:06:16 AM
I use httplist with PXE booting, but that is what it is for. You want a file local to the computer with a list of extensions to be loaded. That is what onboot.lst is. Maybe what you want is "include" capability, or the ability to have more than one file of extensions, but that is different from httplist and not what httplist is designed to do.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 05, 2021, 08:22:49 AM
None of the above (well, "load local packages, then 'include' online ones, or vice-versa" might be a development). The starting post is succinct just enough to get a brewing idea.

httplist= is (points to) (almost) a typical *.lst file. No more no less.

The whole point is to have it local, while get the list of packages (in it) load online, but I'm just repeating my first post.

It's not about the design or what is for and what was not, it's about augmenting tc-config to embrace a new functionality.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: curaga on January 05, 2021, 12:29:36 PM
Such a design, local list that is looped and each entry loaded over the net, would encourage wastefulness and place additional load on the mirrors. As such it won't be accepted.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 05, 2021, 01:18:10 PM
With all due respect (and, in fact, it had been unspoken my own statement. You broke the silence first (just kidding)):

Seriously?… Well, perhaps. May I lay out few open questions? (for good surely exaggerated)

No responsible admins/users out there? No nginx for local ISPs? At home/work? Writing open source Linux was an unworthy effort, let's no shell scripts submit for modifying (*)? PXE booting is not a problem anymore? (I didn't get the meaning of the 'list that is looped')

Not trying to prop an example, really - but SliTaz boots with 0.8 Mb (!) ISO a full-blown desktop using the latter tech (which is really heavier than 'including' a singled-out module on a tiny system "upped" from what not necessary be a TCL mirror), and no sense of unhealthy competition?

*. Tiny Core Linux - Leo's Notes (https://leo.leung.xyz/wiki/Tiny_Core_Linux)
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: Rich on January 05, 2021, 01:53:24 PM
Hi TT
... (I didn't get the meaning of the 'list that is looped') ...
I think curaga may be referring to extensions being loop mounted. If you look in  /usr/local/bin/  you will find it full of links
back to  /tmp/tcloop/...  which is where extensions get mounted by  tce-load.  When you access one of those links in
/usr/local/bin/  it reads that data from the mounted extension.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 05, 2021, 02:04:40 PM
Ah, thanks, you all guys are lighting fast if there was a question!

P.S. it just crossed my mind that I might have been understood two-fold: a) "get [download] packages online"; b) "get packages online [http-analogue of nfsmount]". Mine was "a)". Well, let's increase the complexity of the question at hands ))
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: andyj on January 05, 2021, 03:06:47 PM
Yes, there is a gap here. Perhaps instead of saying "change Y to do X" maybe say "I would like to be able to do X. What functionality already exists that will let me do that or something similar that might suffice." TC can load extensions locally or from a remote location either during boot or from the command line, so we are not getting what it is that you are trying to do that can not already be done.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 05, 2021, 03:37:55 PM
If such a thing could already be done, may you provide a particular example, please?:

0) small-sized boot media, just enough for the latest released TC x86 ISO 11.1 (not Core, not Plus, just Tiny + Xvesa), so, there was no way onboot to have, let say, a Palemoon browser from a mirror.
1) we are not talking about a post-boot scenario, it shall [down]load-mount-and-loop a package during the boot process, like any "usual" local package would-be already present on the ISO, however not (online only); we are not talking about changing any code but manipulate kernel-line configuration parameters;
2) yes, we could modify a kernel boot parameter(s);
3) please, outline in configuration code you are getting this done.

*) we do not have an access to the external resources (e.g. cannot modify a web-site, and no touching of the internal network)
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 05, 2021, 04:51:23 PM
A slightly different question was recently bubbling around, without a straightforward solution. I'm quoting it here since it seems to me remotely related.
Seems to me that no question can be stupid; you're asking how to accomplish something which isn't (correct me if I'm wrong other folks) possible - but the reason you're asking this is to solve a situation you're interested in;  getting games.

First off - how does TC handle the TCE App Listing;  well, the /usr/bin/tce-fetch.sh is a good place to look.  Here it pairs the current kernel, the current TC version, and the current mirror all together to form the url which is used to fetch things like:

  • the app list itself (( seen in /tmp/tags.db ))
  • an individual app's description, dependencies, file listings, etc

note - the current mirror is stored in /opt/tcemirror

Unfortunately, the suite of package management scripts are all geared around the currently running combination of kernel architecture, and the TC version ( which can be found in /usr/share/doc/tc/release.txt  )  Moreover, if you could point at an earlier TC repo, seems quite likely to me that literal binary contents of the executables would be incompatible from an ABI standpoint;  there might be cases where this isn't true - I am not an expert at all.

But - as to your goal - getting games.  You have many options.  One can request new (or renewed) packages in the appropriate forum for that (( which is http://forum.tinycorelinux.net/index.php/board,46.0.html (http://forum.tinycorelinux.net/index.php/board,46.0.html) )) -  one can download the source to games which provide their source and build directly - one can run precompiled games, so long as the underlying dependencies are met.  (( for example - I've used a handful of the Humble Bundle games on TC 11.1 x86_64 ))

So - your question wasn't stupid;  I hope my answer helps give you the next direction to go in for this...

Good luck,
Lexi
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: andyj on January 05, 2021, 06:15:26 PM
By small-sized media, do you mean CD-sized (700 MB), or 20-year-old flash drive (32 MB)?

What does this mean? Can you add a computer that can serve what you need?
... no touching of the internal network

Are you trying to configure/build servers, workstations (desktops or kiosks), or both?

There are a lot of people here with a lot of experience solving a lot of problems. We just need to understand (or better yet grok) what your specific problem is first. Maybe I'm old and have been doing this for too many decades, but I'm just not getting what you are trying to accomplish.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: curaga on January 06, 2021, 03:10:16 AM
It's pretty clear to me. They want to skimp on local storage, as well as not run their own servers (so they can't use NFS or http/tftp). Which would download the listed extensions from the mirrors on every boot, externalising costs but keeping the benefits.

It's irrelevant what the actual technical details are, if it's embedded on a BIOS flash chip, a builtin MMC chip or a small usb stick inside the case. The network use case is abusive.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on January 06, 2021, 04:12:06 AM
Hi curaga and andyj,

I appreciate you're back. As to the above example it is intentionally exaggerated (well, the post I'm answering now exaggerated it even more) because I started questioning my English, that my first succinct posting was yet not enough to convey what I wanted to ask… Even if I give reference(s), they are not being read… I don't know, what's going on, is this how the beginning of 2021 looks like?

At least I've got a confirmation I could still write in English. It took 12 (twelve) postings to this point and time. Now back to business.

Please, don't judge the models you're constructing in your mind that are largely disconnected from what is on the table.

I may accept [they] "not run their own servers" (in this particular thread the talk is about a local storage), "download the listed extensions from the mirrors on every boot" (it seems all the words from my reply #6 (http://forum.tinycorelinux.net/index.php/topic,24681.msg156679.html#msg156679) got swallowed by a black hole), but I cannot take [they] "want to skimp on local storage" (I am paying my bills), "externalising costs but keeping the benefits" (again my reply #6 (http://forum.tinycorelinux.net/index.php/topic,24681.msg156679.html#msg156679)) which is going too far in the cosmos.

Well, I never thought discussing a technology is to be abusive.

Any fresh thoughts? If SliTaz fatally abuses the network use case, please, go ahead and kill the thread, I'll survive. It means I'll still stay with TC.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on September 18, 2022, 03:19:54 PM
Hi guys, may I get a more balanced view near one-and-a-half year later? ;D
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on November 05, 2022, 02:07:34 AM
;bump; I'm really sorry for that…
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: gadget42 on November 05, 2022, 06:54:47 AM
With all due respect (and, in fact, it had been unspoken my own statement. You broke the silence first (just kidding)):

Seriously?… Well, perhaps. May I lay out few open questions? (for good surely exaggerated)

No responsible admins/users out there? No nginx for local ISPs? At home/work? Writing open source Linux was an unworthy effort, let's no shell scripts submit for modifying (*)? PXE booting is not a problem anymore? (I didn't get the meaning of the 'list that is looped')

Not trying to prop an example, really - but SliTaz boots with 0.8 Mb (!) ISO a full-blown desktop using the latter tech (which is really heavier than 'including' a singled-out module on a tiny system "upped" from what not necessary be a TCL mirror), and no sense of unhealthy competition?

*. Tiny Core Linux - Leo's Notes (https://leo.leung.xyz/wiki/Tiny_Core_Linux)
the Leo's Notes website is interesting, thanks for the reference.
i want to understand your question more but need concrete instructions to replicate your experiments.
i have SliTaz 2.0, 3.0, 4.0, and Rolling Core 5 in 1
i have any/all TCL
what do i do next?
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on November 05, 2022, 03:07:11 PM
Hi, gadget42,

Thank you for noticing my thread (I did my homework) - and congratulations! you are the first in more than one and a half years to really have read into it, picking up the Leo's Notes website.
First thing first, my opening message back then was as follows:

How tc-config could be modified, so it loads packages online from locally defined httplist?

Kinda httplist=file:///… whereas file:///… has a content of: 192.168.0.1/…

Or would it be smarter to introduce a 'filelist' parameter to get_app with "FILE" following?

So, I had to refresh my memory before continuing. The idea was born out because of − if one or two or few modules could not fit my memory card − how could one get a left-behind module(s) (or, indeed, all) onboard but from a web server (e.g. one replicating the TCL structure) over http(s) instead of a local storage (i.e. instead of file:///…), and have it mounted the same way SliTaz had already mastered.

If my 'refreshed' memory serves me well, it was about (perhaps for sure) modifying initrd, which mounts local modules accordingly to the onboot.lst (was it get_app function in initrd?), but my proposal was to add a filelist parameter  (a "httplist") instructing get_app loading them ("files") ("modules") from the Internet instead (I may be mistaken in small details, but a system which can "mount", shall be able to "wget" or "curl"). I had already looked at the code and offered the TCL maintainers to seize this opportunity, which… well, was not received wholeheartedly.

I couldn't go back to that question now at the same level of curiosity as back then, time flies, but hopefully have clarified my first post.

In short: unpack initrd; modify the code around get_app ; accordingly to something like onboot.lst , define a filelist parameter  (a "httplist") instructing get_app loading them ("files") ("modules") from the Internet instead; type in your "httplist", it should look something like:

192.168.0.1/…/gnumeric.tcl
10.0.0.1/…/galculator.tcl
tinycorelinux.net/…/palemoon.tcl
etc.

and test that your system [downloads and] mounts the local [and downloaded] modules as instructed upon booting up.

Best,
TT

P.S. One may even envisage mounting modules without even downloading them, that is, mounting them over http(s) but that was not my original idea.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: Rich on November 05, 2022, 05:46:28 PM
Hi TT
... 192.168.0.1/…/gnumeric.tcl
10.0.0.1/…/galculator.tcl ...
That's OK. You are supplying the data from your machines on your LAN.

Quote
... tinycorelinux.net/…/palemoon.tcl
etc. ...
This is not OK. Tinycore and the sites that mirror our repository provide access to extensions
at no cost to you. This service is not an unlimited resource and should not be squandered.
These sites have a fixed bandwidth and possibly monthly data caps associated with them. The
problem is not just how inconsiderate it would be for you to abuse this resource, but for how
disruptive it could be if everyone else decided to do the same thing. You are talking about
downloading the same extensions over and over every time you reboot you machine.

Quote
... P.S. One may even envisage mounting modules without even downloading them, that is, mounting them over http(s) but that was not my original idea.
Even worse. When you start the program it gets downloaded into RAM. If you exit the program and
start it again later on, it may no longer be cached in RAM and get downloaded again. If the program
uses overlays, they'll get downloaded every time they need to get swapped in again.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on November 06, 2022, 02:37:13 AM
Hi Rich

Just to be clear, since I have stated it from the outset − I am not to be associated with, nor advocating "squandering ... Tinycore and the sites that mirror [TCL] repository" which "have a fixed bandwidth and possibly monthly data caps associated with them". Presumption of innocence is a cornerstone of justice (in a broad sense) − why would anyone put a purported blame on me for what I am not having any powers over??! I am not a father of every Internet child. And, speaking equally, TCL is not a father of each and every Internet mirror either, for this scenario policies indeed do exist.

For the other part ("if everyone else decided to do the same thing"), the best explanation is probably the following: https://en.wikipedia.org/wiki/Association_fallacy#. Why, indeed, would "everyone else decided to do the same thing"? Is out there a sort of biblical synchronization event? What about in the worst inconsiderable case, setting ... well, data (or daily speed) caps? Isn't TCL somebody's open source project (I mean [at least initially] voluntary commitment, or is that truly expensive in your economy to run such a project [I'm not an expert on this])? I am just trying to be reasonable and serious, please, get me in the right way. How one could stop modifying as little as a shell script? I have replied to the user with just examples for better understanding.

And the SliTaz guys shall be definitely pilloried as the worst abusers of their own infrastructure: Internet with gPXE (http://boot.slitaz.org/).

Finally, my "P.S." section was just a thoughtful phrase, if there exists a sort of NFS over http(s), which does not get downloaded every time they need to get swapped in again (edit: I'm not getting into technicalities here and it actually may get 'swapped in' again).

I think, I had already explained it all in the above posts, there should be no need for more explanation entropy and making the Heat death of the universe (https://en.wikipedia.org/wiki/Heat_death_of_the_universe) any closer. The moderators also had a say. There could be either heeding the progress or abandoning it altogether.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: Rich on November 06, 2022, 07:41:59 AM
Hi TT
... I am not to be associated with, nor advocating "squandering ...
That's awfully defensive. All I did was quote your example and state  "This is not OK.".
I then explained why I felt it is not OK.

Quote
... blame on me for what I am not having any powers over??! I am not a father of every Internet child. ...
I understand. You want to push for someone to create this option, and still be able to say
it's not your fault other users are using what you pushed for.

Quote
... TCL is not a father of each and every Internet mirror either ...
That is correct. However, just as we don't want to encourage excess traffic on our
servers, we also don't want to place further demands on the mirrors that graciously
host our distro.

Quote
... Why, indeed, would "everyone else decided to do the same thing"? ...
Figure of speech. The point being we are not adding a configuration option that could
needlessly generate extra server traffic.

Quote
... What about in the worst inconsiderable case, setting ... well, data (or daily speed) caps? ...
We would still be dealing with extra traffic. Besides, the best way to handle data and/or
speed caps is to not create a need for them in the first place.

Quote
... And the SliTaz guys shall be definitely pilloried as the worst abusers of their own infrastructure: Internet with gPXE (http://boot.slitaz.org/). ...
What the SliTaz guys do with their distro is their business.

Or maybe I misunderstood your point. Does the point you were trying to make
go something like this:
    Bobbys mom: OK Bobby, it's 10PM, time to go to bed.
    Bobby: Already? But Charlies mom lets him stay up until 1 AM.
And maybe you think this exchange ends like this:
    Bobbys mom: Alright, since Charlies mom says it's OK you can stay up too.

Quote
... I am just trying to be reasonable and serious, please, get me in the right way. ...
Maybe get an online Google drive? I think you can get about 15 Gigabytes for free. Mount
the drive using  bootlocal.sh  and then you can load extra extensions like this:
Code: [Select]
tce-load -i /mnt/Path/To/Google/Drive/ExtensionName.tcz
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: TT on November 06, 2022, 09:46:44 AM
Hi Rich

I have made my points. You have made yours. I respect what other people are saying, because they're independent, even if personal allusions may appear unfair and hurtful to me. I am not an antagonist in any way to you, or the TCL guys here. This is all about technology, not about the messenger. Let me stop here, however, firmly reject your "you want … still be able to say it's not your fault" and "maybe you think", as nobody can seriously claim reading other people's minds. If you guys don't want to place further demands on the mirrors that graciously host [TCL] distro, that group thinking is wonderful, however it was about if I want to utilize my own mirror, would you still don't want to place any demands on me utilizing my own infrastructure (this is just one scenario). Why pushing forward a mischievous attitude first, I could hardly understand. This is not going to help in life. I have got your other points right about the TCL ideology, and may assure you, you have my understanding. This could frankly have been stated next to my first message along with the alternative GDrive suggestion. There is no shame to stand by the TinyCore philosophy, I only appeal to you guys to give it the benefit of a doubt or a periodical review, and not to have it frozen in time.
Title: Re: How to modify tc-config for: define httplist local - get packages onboot online?
Post by: Rich on November 06, 2022, 10:46:37 PM
Hi TT
... along with the alternative GDrive suggestion. ...
If you are interested, I posted instructions on how to mount a Google drive here:
https://forum.tinycorelinux.net/index.php/topic,25996.0.html