WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: Servers for piCore?  (Read 3699 times)

Offline CentralWare

  • Administrator
  • Hero Member
  • *****
  • Posts: 1652
Servers for piCore?
« on: February 15, 2022, 11:54:14 AM »
With the advent of Pi4, using a RasPi as a server doesn't sound unreasonable to me, but from the looks of it, most of the dependencies for NFSd (portmap, rfc, etc) are unavailable and God knows somewhere in the archives there will be an x86/64 list of sources which would save hours of hunting, compiling (missing dep error), rinse - lather - repeat.  If anyone has access to said list(s) I don't mind doing the leg-work.
NOTE: I have not hunted in pre-11.x armvX versions to see if one or more of these already exist.

Extensions requested (in this area):
NFS (nfs_utils)
AoE (aoe_tools, vblade) -- I think I still have the dependency list on this set
SMB (Samba 4.x is listed, confirmation to its functionality requested for those who've used it.)
FTP (I'm going to compile busybox's FTPd for now, but it's far from being a "complete" server - suggestions requested)
SMTP/POP3/IMAP
Apache
MySQL (MariaDB)
PHPSubversion (SVN), CVS, etc.

Thanks everyone!
Over 90% of all computer problems can be traced back to the interface between the keyboard and the chair

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11178
Re: Servers for piCore?
« Reply #1 on: February 15, 2022, 07:38:35 PM »
Hi centralware
nfs-utils is available:
http://tinycorelinux.net/13.x/aarch64/tcz/nfs-utils.tcz.info

A search for  "samba mysql maria nfs aoe vblade apache"  turned up this in the source directories:
Code: [Select]
http://tinycorelinux.net/10.x/x86_64/tcz/src
drwxrwxr-x 2 root staff 4096 Jan 24  2019 apache

http://tinycorelinux.net/10.x/x86/tcz/src
drwxrwxr-x 2 root staff 4096 Feb 10  2019 nfs-utils

http://tinycorelinux.net/11.x/x86_64/tcz/src
drwxrwxr-x 2 root staff 4096 Feb 16  2020 mariadb

http://tinycorelinux.net/12.x/aarch64/tcz/src
drwxrwxr-x 2 root staff 4096 Jan  3  2021 nfs-utils
drwxrwxr-x 2 root staff 4096 Sep  7  2020 samba4

http://tinycorelinux.net/12.x/armv7/tcz/src
drwxrwxr-x 2 root staff 4096 Sep  7  2020 samba4

http://tinycorelinux.net/12.x/x86_64/tcz/src
drwxrwxr-x 2 root staff 4096 Jan 31  2021 mariadb
drwxrwxr-x 2 root staff 4096 Aug 21 10:24 unionfs-fuse

http://tinycorelinux.net/12.x/x86/tcz/src
drwxrwxr-x 2 root staff 4096 Mar 20  2021 nfs-utils
drwxrwxr-x 2 root staff 4096 Aug 27 13:36 unionfs-fuse

http://tinycorelinux.net/13.x/aarch64/tcz/src
drwxrwxr-x 2 root staff 4096 Sep  7  2020 samba4

http://tinycorelinux.net/13.x/armv7l/tcz/src
drwxrwxr-x 2 root staff 4096 Sep  7  2020 samba4

http://tinycorelinux.net/13.x/armv7/tcz/src
drwxrwxr-x 2 root staff 4096 Sep  7  2020 samba4

http://tinycorelinux.net/4.x/x86/tcz/src
drwxrwsr-x 2 root staff    4096 Dec 17  2012 apache2
drwxrwsr-x 2 root staff    4096 Dec 11  2015 apache2-mod-php5
drwxrwsr-x 2 root staff    4096 Dec 17  2012 apache-tomcat
drwxrwsr-x 2 root staff    4096 Dec 17  2012 libnfs
drwxrwsr-x 2 root staff    4096 Dec 17  2012 libnfsidmap
drwxrwsr-x 2 root staff    4096 Dec 17  2012 mysql
drwxrwsr-x 2 root staff    4096 Dec 17  2012 mysql-client
drwxrwsr-x 2 root staff    4096 Dec 17  2012 samba3
drwxrwsr-x 2 root staff    4096 Dec 17  2012 vblade

http://tinycorelinux.net/5.x/armv6/tcz/src
drwxrwxr-x 2 root staff      4096 Mar 21  2014 apache2.4
drwxrwxr-x 2 root staff      4096 Jan 30  2014 libnfsidmap
drwxrwxr-x 2 root staff      4096 Jul  2  2014 mariadb
-rw-rw-r-- 1 root staff   2747577 Nov  2  2013 nfs-utils-1.2.8.tar.bz2
-rw-rw-r-- 1 root staff       126 Nov  2  2013 nfs-utils.config
drwxrwxr-x 2 root staff      4096 Feb 23  2014 samba

http://tinycorelinux.net/5.x/x86_64/tcz/src
drwxrwsr-x 2 root staff    4096 Dec 18  2013 nfs-utils
drwxrwsr-x 2 root staff    4096 Dec 31  2013 samba

http://tinycorelinux.net/5.x/x86/tcz/src
drwxrwxr-x 2 root staff     4096 Mar 20  2014 apache2.4
drwxrwxr-x 2 root staff     4096 Jun 30  2014 mariadb
-rw-rw-r-- 1 root staff 45226492 Oct 17  2013 mariadb-5.5.33a.tar.gz
drwxrwsr-x 2 root staff     4096 Jan 18  2014 nfs-utils

http://tinycorelinux.net/6.x/armv6/tcz/src
drwxrwxr-x 2 root staff     4096 May 29  2015 apache2.4
drwxrwxr-x 2 root staff     4096 May 30  2015 libnfsidmap
drwxrwxr-x 2 root staff     4096 May 26  2015 mariadb
drwxrwxr-x 2 root staff     4096 Apr 17  2015 pymysql
drwxrwxr-x 2 root staff     4096 Jan 13  2015 samba4

http://tinycorelinux.net/6.x/armv7/tcz/src
drwxrwxr-x 2 root staff     4096 May 29  2015 apache2.4
drwxrwxr-x 2 root staff     4096 May 30  2015 libnfsidmap
drwxrwxr-x 2 root staff     4096 May 26  2015 mariadb
drwxrwxr-x 2 root staff     4096 Apr 17  2015 pymysql

http://tinycorelinux.net/6.x/x86/tcz/src
drwxrwxr-x 2 root staff    4096 Oct  7  2015 aoetools
drwxrwxr-x 2 root staff    4096 Oct  5  2015 apache2.4
drwxrwxr-x 2 root staff    4096 Apr  6  2015 mariadb
drwxrwxr-x 2 root staff    4096 Oct  7  2015 vblade

http://tinycorelinux.net/7.x/armv6/tcz/src
drwxrwxr-x   2 root staff  4096 Oct  7  2015 aoetools
drwxrwxr-x   2 root staff  4096 Jan  4  2016 apache2.4
drwxrwxr-x   2 root staff  4096 Jan 12  2016 mariadb-client
drwxrwxr-x   2 root staff  4096 Oct  7  2015 vblade

http://tinycorelinux.net/7.x/armv7/tcz/src
drwxrwxr-x 2 root staff     4096 Oct  7  2015 aoetools
drwxrwxr-x 2 root staff     4096 Jan  4  2016 apache2.4
drwxrwxr-x 2 root staff     4096 May 30  2015 libnfsidmap
drwxrwxr-x 2 root staff     4096 May 26  2015 mariadb
drwxrwxr-x 2 root staff     4096 Jan 12  2016 mariadb-client
drwxrwxr-x 2 root staff     4096 Apr 17  2015 pymysql
drwxrwxr-x 2 root staff     4096 Oct  7  2015 vblade

http://tinycorelinux.net/7.x/x86_64/tcz/src
drwxrwsr-x 2 root staff    4096 Jan 29  2017 apache
drwxrwsr-x 2 root staff    4096 Dec 29  2015 libnfs
drwxrwsr-x 2 root staff    4096 May  3  2016 mariadb
drwxrwsr-x 2 root staff    4096 Apr 11  2016 nfs-utils

http://tinycorelinux.net/7.x/x86/tcz/src
drwxrwxr-x 2 root staff 4096 Jul 24  2016 apache2.4
drwxrwxr-x 2 root staff 4096 Jul 24  2016 apache2.4-mod-php7.0
drwxrwsr-x 2 root staff 4096 Jul 27  2016 samba3

http://tinycorelinux.net/8.x/armv6/tcz/src
drwxrwxr-x 2 root staff     4096 Jul  8  2016 apache2.4
drwxrwxr-x 2 root staff     4096 Jun 29  2016 apache2.4-mod-php5
drwxrwxr-x 2 root staff     4096 Jun 29  2016 apache2.4-mod-php7
drwxrwxr-x 2 root staff     4096 Jun  2  2016 mariadb
drwxrwxr-x 2 root staff     4096 May 22  2016 mariadb-client
drwxrwxr-x 2 root staff     4096 Dec 25  2016 mysql-connector-python
drwxrwxr-x 2 root staff     4096 Dec 26  2016 pymysql
drwxrwxr-x 2 root staff     4096 Jun 12  2016 samba4

http://tinycorelinux.net/8.x/armv7/tcz/src
drwxrwxr-x 2 root staff     4096 Jul  8  2016 apache2.4
drwxrwxr-x 2 root staff     4096 Jun 29  2016 apache2.4-mod-php5
drwxrwxr-x 2 root staff     4096 Jun 29  2016 apache2.4-mod-php7
drwxrwxr-x 2 root staff     4096 Jun  2  2016 mariadb
drwxrwxr-x 2 root staff     4096 May 22  2016 mariadb-client
drwxrwxr-x 2 root staff     4096 Dec 25  2016 mysql-connector-python
drwxrwxr-x 2 root staff     4096 Dec 26  2016 pymysql
drwxrwxr-x 2 root staff     4096 Jun 12  2016 samba4

http://tinycorelinux.net/8.x/x86_64/tcz/src
drwxrwxr-x 2 root staff 4096 Jul  3  2017 mariadb
drwxrwxr-x 2 root staff 4096 May 22  2017 samba

http://tinycorelinux.net/9.x/armv6/tcz/src
drwxrwxr-x 2 root staff 4096 Jun 24  2017 apache2.4

http://tinycorelinux.net/9.x/armv7/tcz/src
drwxrwxr-x 2 root staff 4096 Jun 24  2017 apache2.4
drwxrwxr-x 2 root staff 4096 Feb 15  2018 nfs-utils

Offline CentralWare

  • Administrator
  • Hero Member
  • *****
  • Posts: 1652
Re: Servers for piCore?
« Reply #2 on: February 15, 2022, 08:55:27 PM »
@Rich: Thanks, Rich.  I built a repo browser for version/arch/tcz directories but didn't think to add support for sources! :)

Take care, stay safe & stay healthy!
Over 90% of all computer problems can be traced back to the interface between the keyboard and the chair

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 10957
Re: Servers for piCore?
« Reply #3 on: February 16, 2022, 12:32:54 AM »
bftpd is my preferred ftp server. Small and easy to configure.

Using rpis as servers usually runs into trouble with the unreliable microsd (and on older versions, unreliable USB IIRC).
The only barriers that can stop you are the ones you create yourself.

Offline CentralWare

  • Administrator
  • Hero Member
  • *****
  • Posts: 1652
Re: Servers for piCore?
« Reply #4 on: February 16, 2022, 01:56:23 AM »
@Curaga: The plan on our end is 1) Boot from uSD (init=rootfs.gz,modules.gz and servers.gz) 2) servers.gz would have a custom bootlocal.sh which basically tends to launching the services and 3) in theory, mmcblk0p1 is then unloaded once complete as it's no longer necessary since technically, no extensions were loaded from the tcedir.
Speaking of which, is it safe to assume the above scenario is "safe" for our bootlocal.sh to overwrite the one that already exists in rootfs.gz?  (The tc-config hasn't even launched at this point, so I can't foresee a problem, but you guys know the system a bit better than I do! :) )
Over 90% of all computer problems can be traced back to the interface between the keyboard and the chair

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11178
Re: Servers for piCore?
« Reply #5 on: February 16, 2022, 06:58:46 AM »
Hi centralware
... is it safe to assume the above scenario is "safe" for our bootlocal.sh to overwrite the one that already exists in rootfs.gz? ...
Both  bootsync.sh  and  bootlocal.sh  are there for you to customize. When restored, they overwrite the  rootfs.gz
version anyway.

The sequence goes like this:
sbin/init->etc/inittab->etc/init.d/rcS->etc/init.d/tc-config(load extensions, restore)->opt/bootsync.sh->opt/bootlocal.sh

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 10957
Re: Servers for piCore?
« Reply #6 on: February 16, 2022, 08:02:29 AM »
Overwriting files in an initrd is totally fine. Unmounting the SD card during runtime does seem the most reliable option.
The only barriers that can stop you are the ones you create yourself.

Offline Greg Erskine

  • Sr. Member
  • ****
  • Posts: 402
Re: Servers for piCore?
« Reply #7 on: February 16, 2022, 11:26:31 AM »
I have been using Raspberry Pi's with a USB drive for music servers since the original RPi1B (256mb) without issues.

I only buy SanDisk Sd cards, so corruption has not been a problem for me.

Most Raspberry Pi's can boot from a USB device, so SD cards are not really necessary.

Proper power supplies for the RPi and USB drives is the main thing to look out for.

YMMV



Offline CentralWare

  • Administrator
  • Hero Member
  • *****
  • Posts: 1652
Re: Servers for piCore?
« Reply #8 on: February 16, 2022, 08:58:47 PM »
Hello everyone!

Power supply: 5v 70A driving (up to/relay controlled) two dozen units, mostly Pi2 and Pi3 with a few 4s in the mix; with a spare 70A ready to go should the first ever take a dump.  They're currently header-fed which doesn't do the job with USB devices being driven from the Pi, but the headers are temporary as a proof-of-concept run for the project.

bootsync/bootlocal would require updating core.gz (or rootfs) which would work out fine for some, but the goal here was to use pristine kernel/init files and just append our own GZ to the pxe/extlinux configuration to make it plug-n-play without having to keep an inventory of all of the custom cores, or the need for mydata, which in PXE isn't a preference (nor an option in most cases.)  The trial running pxeboot.gz (our init which overwrites bootlocal, adds nfs-utils, numerous drivers and a few other necessities) seems to be working flawlessly, but I still felt it best to inquire.

Scenario: A fresh, untouched machine boots via PXE, isn't in the system yet with its own storage, so this appended core boots with the updated bootlocal/sync files allowing it to create its NFS/AoE storage remotely, have the default profile (setup, desktop, server, development, etc.) created which in turn, adds the PXE configuration file onto the TFTP server, calls into DNS/DHCP where a hostname and IP are assigned so the machine can be looked up without having to know much about it and a few minutes later, the box is ready to be SSH'ed into for profiling and use.  We still have to perfect VNC/RPC for the different platforms, drivers, etc. so we're not done just yet.

SD Cards: Over the years, I've bought mostly MicroCenter SD cards for in-house and Samsung for client use for different applications ranging from CCTV cameras to the Pi, and found one of the biggest flaws isn't just manufacturing/dependability, but it runs the chance the pads on the SD (or the pins of the SD socket) have a potential of not making a great connection over time, especially in tablets (where the card isn't sealed well), cameras, my truck's stereo! and of course, the Pi.  We've overcome this with a two stage fix which has done wonders: First, clean the contacts with Iso/rubbing alcohol to remove oxidation/debris, especially for those cards that get frequently man-handled...  then a thin coating of EKG/Electrode conductive gel - insert the cards in/out a few times in the socket, wipe off any excess on the card, and it seems to make a clean, conductive seal between the pins and the card's pads.  For devices which have already begun to oxidize on the socket pins, an electronics contact cleaner (burnishing tool) tends to revive them well.

Thanks to everyone for their input!  Take care and stay safe!

Over 90% of all computer problems can be traced back to the interface between the keyboard and the chair

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11178
Re: Servers for piCore?
« Reply #9 on: February 16, 2022, 09:30:59 PM »
Hi centralware
... allowing it to create its NFS/AoE storage remotely, ...
I don't know what they do or if they'll be of any use to you, but if you look through  tc-config , it contains
nfsmount  and  aoe  boot codes.

Offline CNK

  • Full Member
  • ***
  • Posts: 234
Re: Servers for piCore?
« Reply #10 on: February 16, 2022, 11:49:43 PM »
For an SMTP/POP3/IMAP x86/x86_64 example, as described in the thread where I asked about it at the time, the only extension that ever seems to have offered any of these was GNU mailutils. That extension was built for TC5 on x86 and kept until TC10. build notes here.

I created my own mailutils extension, but have been lazy and never submitted it (also I never tested it much because I haven't got around to setting up that aspect of my home server). Also I built version 3.13, and 3.14 is available now.

The dependencies had changed a little since the TC 5 extension, I ended up with:
Code: [Select]
readline.tcz
libltdl.tcz
gnutls35.tcz
tcp_wrappers.tcz
libunistring.tcz

Build dependencies:
Code: [Select]
readline-dev libunistring-dev gnutls35-dev tcp_wrappers-dev libltdl libtool-dev compiletc squashfs-tools

With: "./configure --prefix=/usr/local --disable-static", these programs are built:
Code: [Select]
Servers: pop3d imap4d comsat mda lmtpd
Clients: putmail frm mail sieve messages readmsg dotlock movemail mimeview decodemail mh

For FTP I'm currently running GNU inetutils ftpd in TC13 x86_64 (inetutils-servers.tcz) and it works well for my usage. It doesn't seem to be available on ARM, but here are juanito's compiling notes for TC9 Pure64. The only dependency is readline7.tcz. tftpd is also in it, and telnetd etc., as well as the inetd super server which runs them.

@centralware: Thanks for the Micro SD contacts tip by the way.
« Last Edit: February 16, 2022, 11:55:23 PM by CNK »

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11178
Re: Servers for piCore?
« Reply #11 on: February 17, 2022, 06:34:23 AM »
Hi centralware
The most recent  inetutils  build scripts for ARMv7 can be found here:
http://tinycorelinux.net/8.x/armv7/tcz/src/inetutils/
http://tinycorelinux.net/8.x/armv7/tcz/src/inetutils-servers/


Offline CentralWare

  • Administrator
  • Hero Member
  • *****
  • Posts: 1652
Re: Servers for piCore?
« Reply #12 on: February 18, 2022, 08:18:23 PM »
I don't know what they do or if they'll be of any use to you, but if you look through  tc-config , it contains nfsmount  and  aoe  boot codes.
@Rich: Probably not, as the boot codes are intended as client-side; but yes, I'm pretty familiar with the inner-workings of tc-config.  I appreciate the bump in that direction, though!
Quote
The most recent  inetutils  build scripts for ARMv7 can be found here:
@Rich: I think you just made my point from another post :)  The most recent version of said extension can be found somewhere around 5 releases ago. I'll take a peek; thanks!

@Admins: Suggesting a repo update.  When submitting a project into the repository, recommending a new file extension_name.tcz.build.xsh be added to assist in the compiling/recompiling process.  I'll submit a form example when time permits, but basically it's a 3-Step script; first, download all source files from at-that-moment valid URLs, ensure build and app dependencies are loaded, secondly, do the actual config/compilation and finally, packing up the extension/tcz.  At very-very least, documentation or notes regarding STEPS #1-2.  I'd imagine I'm not the only one here who hates re-inventing the wheel and/or saving precious hours!

@Everyone: I appreciate the feedback; I'll post scripting and extension submissions for those which are successful and let everyone here decide if they're desirable in the repo.  (Including mailutils.  If memory serves, busybox even has smtp (sendmail) and pop support; but I don't recall IMAP being among those services.  Thanks for the info!)
Over 90% of all computer problems can be traced back to the interface between the keyboard and the chair

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11178
Re: Servers for piCore?
« Reply #13 on: February 18, 2022, 09:23:32 PM »
Hi centralware
The  inetutils  build scripts I linked to do everything you mentioned except fetch the source package.
I created my own version of build scripts. You can find some examples of how I do it here:
http://tinycorelinux.net/12.x/x86/tcz/src/Resizer/
http://tinycorelinux.net/10.x/x86/tcz/src/AutoCursor/
http://tinycorelinux.net/10.x/x86/tcz/src/PicFormat/
http://tinycorelinux.net/11.x/x86/tcz/src/lshw/
http://tinycorelinux.net/10.x/x86/tcz/src/exfat/
http://tinycorelinux.net/10.x/x86/tcz/src/gpicview/

The first 3 are programs I wrote. The tarballs contain source files and build scripts.
The last 3 are packages I built. Those links contain separate build scripts and source tarballs.
I've pretty much managed to automate everything except updating the change log in the  .info  file.

Offline CentralWare

  • Administrator
  • Hero Member
  • *****
  • Posts: 1652
Re: Servers for piCore?
« Reply #14 on: February 19, 2022, 05:16:03 AM »
@Rich: Thank you for the links; I'll look into them once we're able to power back up (Winter storm is causing havoc on the electricity and the UPS devices are being tripped at least once per minute, thus since I'm here I might as well power everything down - it's going to happen and when it does... there's going be be a ripped power line that takes at least ten minutes longer than the UPS batteries are good for!! :)
Over 90% of all computer problems can be traced back to the interface between the keyboard and the chair