WIKI RECOVERY

[mocore]

before this thread drifts OT  AF

We near the finish line, my friends!  As soon as logins and permissions are done at the wiki, I believe my to-do list is finally cleared.

  thanks for your own (& others) herculean efforts to enshore the user facing tc infrastructure remains operational

as your todo list has depleted  could i direct your (future) atention ( wrt  infrastructure )
to this topic/request  "could we have a web-site index.html of extensions "- https://forum.tinycorelinux.net/index.php/topic,22843.msg142792.html#msg142792 -

idk if you remember tc/3.x used to have a html extention viewer
0) http://tinycorelinux.net/3.x/tcz/
and for a time there was packages.tinycorelinux.net if i remember this was sql db based aproach
1) https://web.archive.org/web/20160222021718/http://packages.tinycorelinux.net/

and also : http://tinycorelinux.net/9.x/x86_64/tcz/index.html
appears afair to be the result of this topic https://forum.tinycorelinux.net/index.php/topic,22016.0.html

 wrt of this idea in genral ,
 the data required to built such things ,
is the  repo url layout & repo menta data info.lst/gz ect , plus the extention meta data (.info)

as noted hear: https://forum.tinycorelinux.net/index.php/topic,14237.msg80232.html#msg80232 perhaps ?
if this was automagicly packaged in extention's for each release
 eg repo-data and tcz-data ,
that could/would encoraged other similary minded users to hack/play with and experenent with

wrt :  repo url layout 
the info files for (hyperthetical extention) repo-data and tcz-data would be the perfect place to explain this most hidden part of TC

this is infomation i only found through pokeing around and finding the ibiblio mirror supported ftp
( this afaik is no longer the case =[  i could not login to ibiblio with filezilla , and most browsers have droped native ftp suport )
afaik no other way exists to *see* /  browse the repo file/directory structure !!

ftr

... i have (somewhere) some awk/bash script containg a function to itterate over each release "for_each_release()"
https://forum.tinycorelinux.net/index.php/topic,18767.0.html
and build alist of release urls and then pass them to some other function eg: do_somthing_tcz()
which then looped over each extention and out put a list of tcz url's for the release
which where then passed  to another function which built the table 


... and have previously made some modifications to html reop viewer to add search
https://forum.tinycorelinux.net/index.php/topic,16270.msg96568.html#msg96568

... if posible i would be intrested in progresing some  extetion browser  solution , but it appears it would be best achived with the asitance consideration of persons with server access ,...
eg perhaps packages.tinycorelinux could be reinstated & based on plain text & awk/shell scripts


thanks !!

[Rich]

Hi mocore

before this thread drifts OT ...

The off topic posts have been split into their own thread called Z54 and
moved to the  Tiny Core Lounge.

[CentralWare]

before this thread drifts OT  AF

@mocore: I'm not certain the purpose you're after, but we already have a number of text based files found in the repo itself which may suit your purpose.
Bare in mind they're a part of the repository as opposed to the forum.

tcz/info.lst is basically a file listing of TCZs
tcz/sizelist is the above plus file lengths
tcz/tags.db is a g-zipped version of info plus tag words
tcz/provides.db is similar to an INI file with file listings for each extension
tcz/md5.db is, as you guessed it, the signatures for each file
^{* most of the items above are available in g-zip form, some are gz only.}

I think there's even a file with filename : description but honestly, I can't recall if it's something we built here or something that's in the TCL repo.

As for the File / Directory structure:
[VERSION]/[PLATFORM]/[CONTENT]
There's really nothing hidden, you just follow the general layout as shown above which I believe became the TCL standard around v5.x
Example: http://distro.ibiblio.org/tinycorelinux/13.x/x86/tcz/
Which would be VERSION=13, PLATFORM=x86 (or x86_64, armv6, armv7) and CONTENT=tcz (extensions)

[CentralWare]

Hi CentralWare
I just did a check on members who were last active within the last 730 days (2 years)
and I came up with a count of 1910 unique members.

The 3K users was an SQL query (something like SELECT * from MEMBERS where LAST_LOGIN < [October 1, 2020 or such])
As of October 14, manually, I see 1902 (63 full pages @ 30 users per page and on page 64, there's 12 more that fit the bill of the past 730 days) in Admin -> member list
Mind you, SQL query probably includes unverified (email) users, banned users, etc. -- I cannot be sure as I didn't research too deeply.

[Rich]

Hi CentralWare

... (63 full pages @ 30 users per page and on page 64, there's 12 more that fit the bill of the past 730 days) in Admin -> member list ...

Yup, that's exactly what I did.

[CNK]

this is infomation i only found through pokeing around and finding the ibiblio mirror supported ftp
( this afaik is no longer the case =[  i could not login to ibiblio with filezilla , and most browsers have droped native ftp suport )
afaik no other way exists to *see* /  browse the repo file/directory structure !!

The mirrors.dotsrc.org mirror still allows FTP connections. FTP still works well in Dillo and Links, plus you can always use the command-line FTP client in inetutils. The MC file manager also does FTP.

ftp://mirrors.dotsrc.org/tinycorelinux/

... i have (somewhere) some awk/bash script containg a function to itterate over each release "for_each_release()"
https://forum.tinycorelinux.net/index.php/topic,18767.0.html
and build alist of release urls and then pass them to some other function eg: do_somthing_tcz()
which then looped over each extention and out put a list of tcz url's for the release
which where then passed  to another function which built the table 


... and have previously made some modifications to html reop viewer to add search
https://forum.tinycorelinux.net/index.php/topic,16270.msg96568.html#msg96568

... if posible i would be intrested in progresing some  extetion browser  solution , but it appears it would be best achived with the asitance consideration of persons with server access ,...
eg perhaps packages.tinycorelinux could be reinstated & based on plain text & awk/shell scripts

I don't really like the massive table approach myself, the plain-text list of program names in the current repo is better in terms of download time and reliable page-text searching without turning up lots of false-positives from bits of descriptions (including when using browsers without Javascript support).

But I do think that a separate web interface with similar functionality to the extension browsers would be very useful. Basically something like packages.debian.org. Besides being less work to browse from outside of (the desired version of) Tiny Core, this would also avoid things like me thinking that an extension has no dependencies because I mis-typed ".dep" in the URL and got a 404 - which has happened to me a few times.

As it happens, I made a vague start on this last night (partly just because my internet wasn't working), and mocked up an extension-webpage template. But if that does go anywhere, I'm not really interested in working with others on it. It would run on its own (free) VPS and although I'd publish the code I'm currently thinking of doing it in C, so that may not be convenient for other people looking to make modifications.

The general concept would be a static site generator which makes per-extension web pages and paginated keywords-based lists of extensions. It would use a local mirror of the repo to generate these, but the generated HTML would all be kept in a separate location to the extensions. Search might be done using a PHP script, or later on it might be well-enough indexed by search engines that they could be used (eg. I can search man pages fairly reliably using Duck Duck Go with searches like "top site:man7.org/linux/man-pages/").

Note that it's perfectly likely that nothing will come of this, though I have done a couple of similar things (online product catalogues) from scratch before. They're never pretty though.

[nick65go]

I am looking forward (from the shadow) for the final results on "index.html" of TCZs.

FYI: It is (/was) NOT a problem for a TC /linux user. It is maybe for TC to gain better visibility (if this is a desirable goal) from a potential non-unix user.

Ex: a win10+ user may use "latest" firefox browser, which eliminated FTP protocol. So no dillo, no elink etc. Such user will find "difficult" to "evaluate" TC catalogs (split in few text files) for program versions, total dependency requirements, what is new, etc. The non-linux could filter/ sort for similar /equivalent appls for music /video / whatever.

I understand that this (tce browser) is a trivial task for a linux user. Maybe not an attractive option for a win10 user who would like to test TC capabilities from another OS, eventually in a qemu/virtualbox machine before fully jumping in linux world.
PS: I think it was mentioned in this forum there were TCZ without proper descriptions, or tags, etc. Maybe now they are fully synchronized. A tabular list (Excel or gnumeric) of tcz will show the missing pieces, allow global helicopter picture.

[CentralWare]

Ex: a win10+ user may use "latest" firefox browser, which eliminated FTP protocol. So no dillo, no elink etc. Such user will find "difficult" to "evaluate" TC catalogs (split in few text files) for program versions, total dependency requirements, what is new, etc. The non-linux could filter/ sort for similar /equivalent appls for music /video / whatever.

Honestly, I cannot envision a life-long Windows user resorting to a web browser to sort/filter extensions as their first choice considering the typical Windows user is going to experiment with Tiny Core using the TinyCore.iso with desktop and the App Browser has a search/filter system that does the job rather nicely.  The average newcomer isn't likely going to use index.html as their starting point and the descriptors for filename.tcz.info are not guaranteed to list all of the features or lack of features you've mentioned (like browsers that do or do not support FTP or ones that have the latest CA certificate support compared to those who otherwise lack in that area, etc.)

Additionally, consider what's being compared: Firefox (web browser) and FTP protocol.
Firefox and Chrome no longer support FTP internally (I'm not sure about their counterparts of Opera, Microsoft, etc.)
However, it's far from difficult to install an actual FTP(S) application to handle ftp:// related links for those which still exist.
How an indexed content page of TCZ extensions would help with that...  I just don't see it.  Maybe you can shed some light?

Assuming: Your running Tiny Core Linux v5.x or newer on reasonably supported hardware with desktop(#2) or shell(#3)...
1. Navigate your web browser of choice, within TCL, to http://tinycorelinux.net/12.x/armv7/tcz/
    The web browser just shows you a list of extensions (when they stopped indexing I do not know, I don't have access to the www site.)
    a. To create a new indexing system would require me to clone a good chunk of the repo in order to obtain the content being displayed.
    b. Due to #2 and #3 I don't currently see justification to spend the resources/hours but I'm not closed-minded if it's properly debated.
2. Open the desktop extension manager
    Search for the word browser and tell me what's missing that index.html would provide
3. Open a terminal window and type in tce-ab <ENTER>
    Search for the word browser and tell me what's missing that index.html would provide

Please understand that #2 and #3 are using "tags" found in the text file I mentioned in an earlier message.  Results are not database driven.
When you select an extension from #2 or #3 a more thorough description is given, but far from complete.  I've read manuals (man pages) that aren't remotely complete!
If we resurrected index.html pretty-indexing, the same details you see from #2 and #3 would simply be available in web format.  It would not become "more" informative.

[gadget42]

...snip...
If we resurrected index.html pretty-indexing, the same details you see from #2 and #3 would simply be available in web format.  It would not become "more" informative.

IMHHHO, @CentralWare is correct. Nothing is to be gained other than perhaps a different "look" via format/rendering/etc(definitely NOT worth the efforts involved).

[nick65go]

How an indexed content page of TCZ extensions would help with that...  I just don't see it.  Maybe you can shed some light?
..
    b. Due to #2 and #3 I don't currently see justification to spend the resources/hours but I'm not closed-minded if it's properly debated.

Maybe my use of TC is unique and it does not worth to generalize it. But, please allow me to explain it.At home I use (almost) exclusively linux (TC13_x64, Alpinelinux, etc). This means I use it in the week-ends and sometimes in the evening on my personal laptop. However at day-work in the office and in some evenings I use the employer laptop, which has win10.
So most of the time I am forced to use win10. And on win10 I have huge restrictions: no rights to install any programs (but I manage to have firefox manually functional; and a qemu --slow without acceleration drivers), many web-sites are forbiden.
[do not hit me with "8 h/zi must be dedicated to paying employer" in a not-IT field  ].

So, in my spare time (yes I am productive enough to have spare time at work) I can browse internet, and copy, sort, filter etc, to "complete" the TC with programs from other distro, which can run in chroot (because diff libs versions, x32 on x64 etc). Ex: latest 7zip_20.21, WIMlib compression. I can grab them from Archlinux, Alpinelinux, etc.

FYI: as a long time TC user (from 2009?) I have no problem from INSIDE TinyCore environment. My "challenge" (and maybe for others win users) is to "study" / compare TC with other distro, from OUTSIDE TinyCore environment.

Summary: I have no request from TC team, I can manage myself to customize the TC kit / tools. I just wanted to provide us (TC users) how many others (on win10, or other linux distro) could search for TC capabilities. Because in the end not all users will biuld/rebuild daily their linux, some users just need to use applications (me included). It is not fair (for TC team) that me/others to systematically ask for apps/tcz (ex: libreCAD?) not present in TC.

[mocore]

wrt : WIKI RECOVERY

its good to see wiki links posted ,.. just saw this one https://wiki.tinycorelinux.net/doku.php?id=wiki:creating_meta-extensions
while scrolling the most recent post's
 
...the wiki page links to http://wiki.tinycorelinux.net/Creating+Extensions ( currently 404 ) 
this is working wiki url https://wiki.tinycorelinux.net/doku.php?id=wiki:creating_extensions

i guess some redirect would be simpler than fixing all the old style links  in the forum & wiki ?


wrt : tc repo data format

...snip...
If we resurrected index.html pretty-indexing, the same details you see from #2 and #3 would simply be available in web format. 
It would not become "more" informative.

wrt > It would not become "more" informative.

perhaps this misses the point , which imho is IF the (repo/extension) data (paths/format)

that any thing ( from tce-ab to the "app browser" , or random user scripts ) uses to list/view/ect
- tcl release versions 
- extensions / updates

 were more documented/available(eg:offline / downloadable as extension)
that would de-facto be more informative
... perhaps they were documented/mentioned in the wiki   


much like this post and attached file makes it possible to view understand and reason about
wiki page's archived at different times   
 https://forum.tinycorelinux.net/index.php/topic,25957.msg166495.html#msg166495  datapages.zip

*thinking out loud* it imho it would be amazing if
a wiki archive like datapages.zip could be created and included with 
- the release files or extensions
- coreplus

The mirrors.dotsrc.org mirror still allows FTP connections.

thanks !!

FYI: as a long time TC user (from 2009?) I have no problem from INSIDE TinyCore environment. My "challenge" (and maybe for others win users) is to "study" / compare TC with other distro, from OUTSIDE TinyCore environment.

i think this is an important point , tc can be your desktop , but could also be used in vm
in some other environment perhaps even in *gasp* a  OFFLINE  setting
hence the need for
https://forum.tinycorelinux.net/index.php/topic,23034.0.html 'Script to download extensions+dependencies on other Linux distros'
 

[CentralWare]

its good to see wiki links posted ,.. just saw this one https://wiki.tinycorelinux.net/doku.php?id=wiki:creating_meta-extensions while scrolling the most recent post's
 ...the wiki page links to http://wiki.tinycorelinux.net/Creating+Extensions ( currently 404 ) 

Thank you for this post; the page has been flagged for editing and will be corrected.

[CentralWare]

FYI: as a long time TC user (from 2009?) I have no problem from INSIDE TinyCore environment. My "challenge" (and maybe for others win users) is to "study" / compare TC with other distro, from OUTSIDE TinyCore environment.

Here's the proposed compromise:

• The project has to be hosted on wiki/forum as I do not have access to www, repo or DNS and I hate bugging other people for medial tasks.
• The data files WILL be stale (see schedule below) - this mainly due to #1 but not likely a big deal
• The interface will be basic at first, bells and whistles can be added later (I have a business to run, this is a side assist! )

• Since this will be browser driven, you will select VERSION and PLATFORM from drop-downs
• This an extension browser, ALL files will not be shown, only TCZs and their associated items
• Browsers have search functions built in - for the moment that's going to be sufficient as a search feature.  (CTRL+F)
• I may incorporate a mirror drop-down so if you click on a file/dep/etc. it's flagged from the mirror

Schedule*

• Versions 0.0 - 9.x will update once per month (until that version is no longer in the repo or not supported?)
• Versions 10.x - 12.x will update weekly (few, if any changes expected.)
• Version 13.x (most recent 'stable') will update every couple days
• Version 14.x (most recent 'beta') will update every couple days

<sup>* The above list of versions obviously changes as newer releases take place.
* The above list of versions will start with v5.x, older versions may not be incorporated.</sup>

[mocore]

...the wiki page links to http://wiki.tinycorelinux.net/Creating+Extensions ( currently 404 ) 

Thank you for this post; the page has been flagged for editing and will be corrected.

ftr im happy to edit/correct that link on the wiki if that's whats required ..however


some method

to redirect 

requests for URL http://wiki.tinycorelinux.net/$page_name

to this URL https://wiki.tinycorelinux.net/doku.php?id=wiki:$page_name

would fix all links , internal forum/wiki  and historuc links from the web ( cant be edited )

---

• The project has to be hosted on wiki/forum as I do not have access to www, repo or DNS and I hate bugging other people for medial tasks.

my mistake i had thought *you* had posted this "1.x and 2.x releases to be removed by year end " - https://forum.tinycorelinux.net/index.php/topic,25955.msg166475.html#msg166475
and that it was part of server recovery maintenance

The interface will be basic at first, bells and whistles can be added later (I have a business to run, this is a side assist! )

while this would be nice to have
 it appears my suggestion was a little  misdirected/misplaced 
and perhaps better progressed in  Programming & Scripting - Unofficial  https://forum.tinycorelinux.net/index.php/board,36.0.html

ftr fwiw my general sentiment on this tc community /  infrastructure , more or less summed up succinctly in this quote i found
in these form =o

-Please GPL the backend CGI scripts that Tiny Core uses.  This would allow the community to make suggestions/improvements and better understand how everything works.  This would also allow people who setup their own tcz repository to run the CGI's on their own server so that their Tiny Core systems wouldn't need to have direct internet access for the appbrowser to work.   Plus, in my opinion anyway, opening up the backend code of a GPL project is the right thing to do (that's just me though, and is coming from a card carrying member of the Free Software Foundation..  )

[/quote]
via https://forum.tinycorelinux.net/index.php/topic,25302.msg163376.html

[CentralWare]

ftr im happy to edit/correct that link on the wiki if that's whats required ..however
some method to redirect 
requests for URL http://wiki.tinycorelinux.net/$page_name
to this URL https://wiki.tinycorelinux.net/doku.php?id=wiki:$page_name
would fix all links , internal forum/wiki  and historuc links from the web ( cant be edited )

@mocore: There's pros and cons here.
First and foremost, my apologies for responding so late; I've been out of state for a while and thus slammed once I returned.  I should be back in the saddle sometime Sunday.

Regarding the $page_name suggestion above, one of the senior admins had emphasized the avoidance of rewriting source code for the wiki and forum directly due to the likeliness of software upgrades in the future potentially breaking the upgrade OR breaking the updates we make today, so pushing a 404 based redirect onto doku.php as suggested would potentially fall into that category.
Additionally, the ?id=wiki portion of the URL cannot be assumed as it's a container ("id=wiki" is a set of pages, not the heart of the wiki site itself - you'd want to review the docs for DokuWiki to more familiarize yourself of their navigation.)  The id=wiki container was likely created back when id=dCore was implemented as the admins look as though they wanted to separate the two and thus where I assume id=wiki was born.