Tiny Core Linux
Tiny Core Extensions => TCE Talk => Topic started by: CentralWare on December 13, 2024, 01:03:57 AM
-
I had planned to put together DistCC as an extension for x86/64 and ARM6/7/8 but I'm running into a few obstacles.
The first thing she complained about is missing the Libery Library (libiberty) which I'm "guessing" was deprecated a while back as I can see it in GCC source, but doesn't mean it was compiled in, so I'm building --without-libiberty to get passed the issue for the moment and config/make goes through without issue.
make check, however, fails with a python self-test (see source note below)
A fresh set of eyes would be truly appreciated!
#!/bin/sh
CUR=$(pwd)
# Common ANSI Colors
RED="$(echo -e '\033[1;31m')"; GREEN="$(echo -e '\033[1;32m')"
YELLOW="$(echo -e '\033[1;33m')"; BLUE="$(echo -e '\033[1;34m')"
MAGENTA="$(echo -e '\033[1;35m')"; CYAN="$(echo -e '\033[1;36m')"
WHITE="$(echo -e '\033[1;37m')"; NORMAL="$(echo -e '\033[0;39m')"
GetProcessorType()
{
BITS=""; [ ! "$(uname -r | grep 64)" == "" ] && BITS="64"
CPU=`uname -m`
case "$CPU" in
i686)
CFLAGS="$FLTO -fuse-linker-plugin -march=i486 -mtune=i686 $OPTIMIZE $SYMBOLS $DEFINES -pipe -Wall -Wextra -fno-plt"
CXXFLAGS="$FLTO -fuse-linker-plugin -march=i486 -mtune=i686 $OPTIMIZE $SYMBOLS -pipe -Wall -Wextra -fno-exceptions -fno-rtti"
LDFLAGS="-Wl,-T/usr/local/lib/ldscripts/elf_i386.xbn"
;;
x86_64)
CFLAGS="$FLTO -fuse-linker-plugin -mtune=generic $OPTIMIZE $SYMBOLS $DEFINES -pipe -Wall -Wextra -fno-plt"
CXXFLAGS="$FLTO -fuse-linker-plugin -mtune=generic $OPTIMIZE $SYMBOLS -pipe -Wall -Wextra -fno-exceptions -fno-rtti"
LDFLAGS="-Wl,-T/usr/local/lib/ldscripts/elf_x86_64.xbn"
;;
armv*)
CFLAGS="-march=armv6zk -mtune=arm1176jzf-s -mfpu=vfp $OPTIMIZE $SYMBOLS $DEFINES -pipe -Wall -Wextra"
CXXFLAGS="-march=armv6zk -mtune=arm1176jzf-s -mfpu=vfp $OPTIMIZE $SYMBOLS -pipe -Wall -Wextra -fno-exceptions -fno-rtti"
LDFLAGS="-Wl,-O1"
;;
aarch64)
CFLAGS="-march=armv8-a+crc -mtune=cortex-a72 $OPTIMIZE $SYMBOLS $DEFINES -pipe -Wall -Wextra"
CXXFLAGS="-march=armv8-a+crc -mtune=cortex-a72 $OPTIMIZE $SYMBOLS -pipe -Wall -Wextra -fno-exceptions -fno-rtti"
LDFLAGS="-Wl,-O1"
;;
*)
echo "$PROCESSOR_TYPE: Unknown processor type. Please add an entry for it in this script."
exit
;;
esac
export CPU BITS CFLAGS CXXFLAGS LDFLAGS
}
nproc()
{
LARGE=0
ITEMS=$(cat /proc/cpuinfo | grep "processor" | grep ":" | awk -F: '{print $2}')
for ITEM in $ITEMS
do
[ $ITEM -gt $LARGE ] && LARGE=$ITEM
done
LARGE=$(expr $LARGE + 1) 2>/dev/null
echo $LARGE
}
###################################################################
### This is just to create a clean slate for running extensions ###
###################################################################
# echo "Dismounting extensions"; cd /tmp/tcloop; sudo umount * >/dev/null 2>&1; sudo rmdir * >/dev/null 2>&1
# echo "Clearing Installations"; cd /usr/local/tce.installed; sudo rm *
cd $CUR
echo -n "Loading extensions "
EXTS="wget compiletc cpp bc ncursesw-dev mpc-dev coreutils glibc_apps gettext gdb python3.9-dev \
git autoconf automake udev-lib-dev texinfo coreutils elfutils-dev zsync squashfs-tools \
libtirpc-dev libtool-dev pkg-config gtk4-dev libgnomecanvas-dev libglade-dev pango-dev"
CNT=0
for EXT in $EXTS
do
EXT=${EXT/.tcz/}
echo -n " ${YELLOW}${EXT}${NORMAL}"
[ ! -f /etc/sysconfig/tcedir/optional/${EXT}.tcz ] && tce-load -w ${EXT} >/dev/null 2>&1
[ ! -d /tmp/tcloop/${EXT} ] && tce-load -i ${EXT} >/dev/null 2>&1
CNT=$(expr $CNT + 1)
if [ $CNT -gt 6 ]; then
echo
echo -n " "
CNT=0
fi
done
echo
####################################################################
### This is just to get a list of tags/versions available on GIT ###
####################################################################
echo "Gathering source tags"
[ ! -f tags.html ] && wget -q -O tags.html https://github.com/distcc/distcc/tags
cat tags.html | grep "/tag/" | awk -F "href=" '{print $2}' | awk -F "\"" '{print $2}' > tags.lst
mkdir -p /tmp/taglist && rm /tmp/taglist -fR
mkdir -p /tmp/taglist
for TAG in $(cat tags.lst)
do
TAG=$(basename $TAG)
touch /tmp/taglist/$TAG
done
cd /tmp/taglist
ls -1 | grep "v" | sort -r > $CUR/tags.lst
########################################
### The version we're going to build ###
########################################
version=v3.4
### Go GIT our intended source
cd $CUR
[ ! -d distcc ] && git clone https://github.com/distcc/distcc.git
### Switch over to $version ###
cd distcc; git checkout ${version}
# Real Binary Python3 - probably not "NECESSARY" but just in case #
sudo rm /usr/local/bin/python3
sudo cp -a /usr/local/bin/python3.9 /usr/local/bin/python3
GetProcessorType
if [ ! -f ../configure ]; then
echo "Launching autogen"
./autogen.sh || exit 1
fi
mkdir -p build; cd build
#########################################
### LibIberty no longer in compiler?? ###
### MAY be necessary for @file calls ###
#########################################
if [ ! -f ../Makefile ]; then
echo "Launching configure"
../configure --without-libiberty || exit 1
fi
echo "${CYAN}Running MAKE${NORMAL}"
make -j$(nproc) || exit 1
### ERROR: include_analyzer_test.py
# ======================================================================
# FAIL: test_AdvancedComputedIncludes (__main__.IncludeAnalyzerTest)
# ----------------------------------------------------------------------
echo "${CYAN}Running MAKE : CHECK${NORMAL}"
make check || exit 1
echo "${CYAN}Running MAKE : INSTALL${NORMAL}"
sudo make DESTDIR=/tmp/distcc install
#echo "Running MAKE : INSTALL : CHECK"
#sudo make DESTDIR=/tmp/distcc installcheck
-
Hi, CentralWare.
I'm testing distcc on armhf right now.
The extensions is ready, I haven't mailed it yet.
Distcc works great on Raspberry Pis.
Here's my build instructions in case you're interested.
I'm planning to port it on all ports, just need some more time.
Note that you won't find some extensions on the repo, cause I haven't mail them yet. ::)
Compiled on RPI4B-8G with 6.6.47-piCore-v8 on portable SSD
Compile Flags:
CC='clang --gcc-install-dir=/usr/local/lib/gcc/armv7l-unknown-linux-gnueabihf/13.2.0'
CXX='clang++ --gcc-install-dir=/usr/local/lib/gcc/armv7l-unknown-linux-gnueabihf/13.2.0'
CFLAGS='-march=armv6zk -mtune=arm1176jzf-s -mfpu=vfp -Os -pipe'
CXXFLAGS='-march=armv6zk -mtune=arm1176jzf-s -mfpu=vfp -Os -pipe'
EXTRA_C=' -L/usr/local/lib -flto=full -fuse-ld=lld -fno-exceptions -Wno-fortify-source'
EXTRA_CXX=' -L/usr/local/lib -flto=full -fuse-ld=lld -fno-exceptions -fno-rtti -Wno-fortify-source'
CFLAGS="${CFLAGS}${EXTRA_C}"
CXXFLAGS="${CXXFLAGS}${EXTRA_CXX}"
LDFLAGS='-Wl,-L/usr/local/lib -Wl,-rpath,/usr/local/lib -Wl,-O1'
export CC CXX CFLAGS CXXFLAGS LDFLAGS
Dependencies:
automake
clang
coreutils
gettext-dev
git
gtk3-dev
libtool-dev
lld
procps-ng
python3.11-setuptools
python3.11-dev
Commands:
# System adjustment, stop python from adding unwanted flags (-g, -O3)
# Make these files are modified
py1=/usr/local/lib/python3.11/config-3.11-arm-linux-gnueabihf/Makefile
py2=/usr/local/lib/python3.11/_sysconfigdata__linux_arm-linux-gnueabihf.py
sudo busybox cp $(realpath $py1) $py1
sudo busybox cp $(realpath $py2) $py2
sudo sed -i 's/-g -fwrapv -O3/-fwrapv/g' $py1 $py2
unset py1 py2
git clone --depth 1 https://github.com/distcc/distcc.git
cd distcc/
############################################
$ git log -n 1
commit 5394870bed00c20fc9914e303356103ecd3394a6 (grafted, HEAD -> master, origin/master, origin/HEAD)
Author: Martin Pool <mbp@sourcefrog.net>
Date: Tue Oct 15 08:16:13 2024 -0700
Merge pull request #532 from distcc/ci-macos
fix: disable macOS CI for now
############################################
# -Wno-strict-prototypes
patch -Np1 < ../distcc.void.patch
# adjust hardcoded file path for TC (/usr/lib --> /usr/local/lib)
patch -Np1 < ../distcc.path.patch
./autogen.sh
./configure --prefix=/usr/local --without-libiberty
time make -j4
real 0m 27.70s
user 1m 8.05s
sys 0m 4.58s
make install DESTDIR=/mnt/sda1/unstripped-copies/distcc
# packing
cd $(mktemp -d)
cp -r /mnt/sda1/unstripped-copies/distcc .
rm -fr ./distcc/usr/local/sbin ./distcc/usr/local/lib/python3.11/site-packages/include_server/__pycache__/ ./distcc/usr/local/share/man ./distcc/usr/local/share/doc/distcc/example ./distcc/usr/local/share/doc/distcc/*txt
# MASQUERADING
mkdir ./distcc/usr/local/lib/distcc
cd ./distcc/usr/local/lib/distcc
COMPILER='gfortran
armv7l-unknown-linux-gnueabihf-c++
armv7l-unknown-linux-gnueabihf-g++
armv7l-unknown-linux-gnueabihf-gcc
c++
cc
g++
gcc'
for COM in $COMPILER; do
ln -s /usr/local/bin/distcc $COM
done
# clang needs special care on piCore 32-bit
vi clang clang++
chmod +x clang clang++
# /usr/local/tce.installed/distcc
cd ../..
mkdir -m 775 tce.installed
cp /mnt/sda1/notes/tce-installed/distcc tce.installed/
# strip
find -type f -exec head -c4 {} ';' -print | awk '/^\x7fELF/{print(substr($0,5))}' | xargs llvm-strip --strip-unneeded
# example file
cp /mnt/sda1/notes/extra-files/distcc/tc.sh ./share/doc/distcc/
# commands.allow.sh
cat /mnt/sda1/notes/extra-files/distcc/commands.allow.sh > ./etc/distcc/commands.allow.sh
# extra deps
echo procps-ng.tcz >> distcc.tcz.dep
-
On second thought, it's working fine.
Just mailed it.
The "How To Use" instructions are described in distcc.tcz.info
Please check. :)
-
On second thought, it's working fine.
Just mailed it.
@polikuo: It sounds like you may have saved me a step or three! :)
I have to create a build script for each of the five main platforms; under x86_64 right now I ran into a snafu with __main__.IncludeAnalyzerTest -- but it doesn't describe WHAT the issue was very well, so I have to work backward... we'll see if your build details shed some light!
Goal: To create a native compiling cluster/farm for ARM6/7/8 and PC86/64
Challenge: distcc server(s) need to be headless and preferably without GUIs/X and the rest of the graphical overhead if possible.
Could you post your patches (or links where you obtained them from?)
Thanks!
-
Goal: To create a native compiling cluster/farm for ARM6/7/8 and PC86/64
Challenge: distcc server(s) need to be headless and preferably without GUIs/X and the rest of the graphical overhead if possible.
Could you post your patches (or links where you obtained them from?)
Thanks!
Hi, CentralWare.
The extension has been posted, you can check out the files here:
http://tinycorelinux.net/15.x/armhf/tcz/src/distcc/ (http://tinycorelinux.net/15.x/armhf/tcz/src/distcc/)
The setup instructions are explained in distcc.tcz.info
I just finished compiling FLTK 1.4 with full support on my cluster.
I'm attaching my note here, so you can try out distcc on your pies.
Note that you won't find libdecor.tcz on the repo just yet.
There will be some minor adjustment before the submission of fltk-full.tcz.
-
Notes and specs received; thank you!
Now for the fun part... see my next post
Your flags pushed me into asking :)