location of .build scripts? building python packages and gfortran

[seandepagnier]

Sorry if I just didnt find it, but looking for the location to rebuild the packages listed:
http://tinycorelinux.net/16.x/armhf/tcz/

I am again updating my version of tinycore linux (tinypilot to run pypilot used as an autopilot) from version 9, and this time I wanted to improve the scripts to make them more reusable.

I am mostly building a lot of python packages, so was hoping to develop a generic build script that uses pip and makes a tcz for any python package, and wanted to check if this was already in place.

Also I again need scipy.  Before I had to recompile gcc with fortran supportto get gfortran to build blas, but didnt have to make it into a package since it was a temporary dependency used to build blas.

The gfortran package doesnt seem to provide the needed compiler, not sure what it is intended for so again build scripts are helpful

I would love to contribute the build scripts back to the picore project if I can, so any suggestions, and sorry if i missed some obvious pointers.   Already many thousands of people steered their boats using tinycore and it has been a reliable stable platform for almost 10 years.

[Rich]

Hi seandepagnier
The repos have a tcz/src/ directory for source packages and build scripts:
http://tinycorelinux.net/16.x/armhf/tcz/src/

Sometimes packages get copied over without recompiling. When that
happens, you would need to look for source packages in previous releases.

[Rich]

Hi seandepagnier
Don't know if it helps, but I ran a search for a few of the
names you mentioned:

Code: [Select]

rich@tcbox:~$ ./FindSrcDirs.sh pilot
http://tinycorelinux.net/11.x/x86/tcz/src/
drwxrwxr-x 2 root staff 4096 Sep  7  2020 jpilot/
-rw-rw-r-- 1 root staff    1384 Sep  7  2020 compile_jpilot
-rw-rw-r-- 1 root staff 1740735 Apr 26  2020 jpilot-1.8.2.tar.gz

drwxrwxr-x 3 root staff 4096 Sep  7  2020 pilot-link/
-rw-rw-r-- 1 root staff    1727 Sep  7  2020 compile_pilot-link
drwxrwxr-x 2 root staff    4096 Sep  7  2020 patches/
-rw-rw-r-- 1 root staff 1259831 Jun 14  2014 pilot-link_0.12.5-dfsg.orig.tar.gz


rich@tcbox:~$ ./FindSrcDirs.sh scipy
http://tinycorelinux.net/13.x/x86_64/tcz/src/
drwxrwxr-x 2 root staff 4096 Apr 23  2022 py3.9-scipy/
-rw-rw-r-- 1 root staff     1431 Apr 23  2022 compile-scipy.txt
-rw-rw-r-- 1 root staff 28745160 Apr 23  2022 scipy-1.8.0.tar.xz


rich@tcbox:~$ ./FindSrcDirs.sh blas
http://tinycorelinux.net/13.x/x86_64/tcz/src/
drwxrwxr-x 2 root staff 4096 Apr 23  2022 openblas/
-rw-rw-r-- 1 root staff      323 Apr 23  2022 compile-openblas.txt
-rw-rw-r-- 1 root staff 12742441 Apr 23  2022 OpenBLAS-0.3.20.tar.gz

drwxrwxr-x 2 root staff 4096 Apr 23  2022 py3.9-numpy-with-openblas/


rich@tcbox:~$ ./FindSrcDirs.sh gfortran
http://tinycorelinux.net/10.x/x86/tcz/src/
drwxrwxr-x 2 root staff 4096 Oct 23  2019 gfortran/
-rw-rw-r-- 1 root staff 598 Oct 23  2019 compile_gfortran


http://tinycorelinux.net/12.x/aarch64/tcz/src/
drwxrwxr-x 2 root staff 4096 Jan  3  2021 gfortran/
-rw-rw-r-- 1 root staff     1216 Jan  3  2021 compile_gfortran
-rw-rw-r-- 1 root staff 74591240 Jul 24  2020 gcc-10.1.0.tar.xz


http://tinycorelinux.net/12.x/armv7/tcz/src/
drwxrwxr-x 2 root staff 4096 Dec 30  2020 gfortran/
-rw-rw-r-- 1 root staff     1138 Dec 30  2020 compile_gfortran
-rw-rw-r-- 1 root staff 74591240 Jul 22  2020 gcc-10.1.0.tar.xz


http://tinycorelinux.net/12.x/x86_64/tcz/src/
drwxrwxr-x 2 root staff 4096 Apr 18  2021 gfortran/
-rw-rw-r-- 1 root staff 3090 Apr 18  2021 compile-gfortran.txt


http://tinycorelinux.net/13.x/aarch64/tcz/src/
drwxrwxr-x 2 root staff  4096 Feb  2  2022 gfortran/
-rw-rw-r-- 1 root staff 6765 Feb  2  2022 gcc1020.build
-rw-rw-r-- 1 root staff 2472 Feb  2  2022 packing-gfortran.txt


http://tinycorelinux.net/13.x/x86_64/tcz/src/
drwxrwxr-x 2 root staff 4096 Mar 14  2022 gfortran/
-rw-rw-r-- 1 root staff 3854 Mar 14  2022 compile-gfortran.txt


http://tinycorelinux.net/7.x/x86/tcz/src/
drwxrwsr-x 2 root staff 4096 Aug 13  2016 gfortran/
-rw-rw-r-- 1 root staff      636 Aug 13  2016 compile_gfortran
-rw-rw-r-- 1 root staff 95221552 Nov  5  2015 gcc-5.2.0.tar.bz2

[seandepagnier]

Thanks!


I had a look at the compile-scipy.txt for example that you linked and this doesnt appear to be the build script that creates the tcz, there is no mksquashfs commad, so not sure how would this generate the needed tcz file?

Also one that is closer to what I am trying to do:
http://tinycorelinux.net/16.x/armhf/tcz/src/python3.11-markupsafe/compile_markupsafe

The last line is:
sudo python3.11 setup.py install --root=/tmp/pkg

so where does it make the tcz?

[Paul_123]

Because the file is just compile notes.   After you install the program to the temp directory, you need to analyze whet got installed there, clean it up, then make a tcz of it.

Making python extensions is not always trivial.

[seandepagnier]

Yes I understand.  I was doing something similar but wanted to compare the process.

Are you saying none of the python packages checked in have this step documented of exactly how the tcz was produced so it can be reproduced?

I suppose there is also debate to include pyc files or not, as they save some time compiling to byte code but take up more space.

[Paul_123]

Yes you need to remove pyc.

[Juanito]

Basically, you need this:

Code: [Select]

...
sudo python3.11 setup.py install --root=/tmp/pkg
sudo rm -r /tmp/pkg/usr/local/lib/python3.11/site-packages/markupsafe/__pycache__
sudo strip --strip-unneeded /tmp/pkg/usr/local/lib/python3.11/site-packages/markupsafe/_speedups.cpython-311-aarch64-linux-gnu.so
cd /tmp
sudo mksquashfs pkg/ python3.11-markupsafe.tcz
sudo chown tc:staff *tcz
for file in *.tcz; do md5sum "$file" > "$file".md5.txt ; done
for file in *.tcz; do unsquashfs -lc "$file" > "$file".list ; done && sed -i 's/squashfs-root//g' *tcz.list
In other python packages you may need to descend several levels of subdirectories deleting __pycache__ and to search out *.so* dynamic libs to strip.

[seandepagnier]

Oh i had been piping find through grep so$ then xargs strip...

I will proceed then, I was hoping we could have a generic script to create a tcz from any pip package but maybe it is asking too much?