WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: pip3 directory persistence  (Read 1635 times)

Offline gatorback

  • Newbie
  • *
  • Posts: 17
pip3 directory persistence
« on: March 01, 2020, 01:08:08 PM »
pip3 was successfully installed:

Code: [Select]
mkdir /home/tc/pip3Persistent
cd /home/tc/pip3Persistent
wget https://bootstrap.pypa.io/get-pip.py
sudo -H python3.6 get-pip.py

pip3 is was successfully tested:

Code: [Select]
sudo -H pip3 install pip-install-test
python3.6 -c 'import pip_install_test'
Four new directories (tagged with current date \ time):

Code: [Select]
tc@box:~/pip3Persistent$ ls -lt /usr/local/lib/python3.6/site-packages/
total 0
drwxr-xr-x    4 root     root           280 Mar  1 21:16 wheel/
drwxr-xr-x    2 root     root           180 Mar  1 21:16 wheel-0.34.2.dist-info/
drwxr-xr-x    2 root     root           180 Mar  1 21:16 pip-20.0.2.dist-info/
drwxr-xr-x    5 root     root           140 Mar  1 21:16 pip/
lrwxrwxrwx    1 root     root            66 Jan  1  1970 README -> /tmp/tcloop/python3.6/usr/local/lib/python3.6/site-packages/README
lrwxrwxrwx    1 root     root            70 Jan  1  1970 README.txt -> /tmp/tcloop/python3.6/usr/local/lib/python3.6/site-packages/README.txt
drwxr-xr-x    2 root     root            40 Jan  1  1970 __pycache__/
lrwxrwxrwx    1 root     root            75 Jan  1  1970 easy_install.py -> /tmp/tcloop/python3.6/usr/local/lib/python3.6/site-packages/easy_install.py
drwxr-xr-x    5 root     root           120 Jan  1  1970 pkg_resources/
drwxr-xr-x    5 root     root           680 Jan  1  1970 setuptools/
drwxr-xr-x    2 root     root           240 Jan  1  1970 setuptools-28.8.0.dist-info/

The four directories:
/usr/local/lib/python3.6/site-packages/pip/
/usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info/
/usr/local/lib/python3.6/site-packages/wheel/
/usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info/


are added to filetool.lst so that the directories are backed up,via filetool.sh -b, to ensure persistence between reboots:

Code: [Select]
vi /opt/.filetool.lst
filetool.sh -b
Code: [Select]
tc@box:~/pip3Persistent$ tail /opt/.filetool.lst
usr/local/etc/ssh/ssh_host_ed25519_key.pub
usr/local/etc/ssh/ssh_host_rsa_key
usr/local/etc/ssh/ssh_host_rsa_key.pub
/usr/local/lib/python3.6/site-packages/pip/ 
/usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info/
/usr/local/lib/python3.6/site-packages/wheel/ 
/usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info/

/usr/local/lib/python3.6/site-packages/pip/
/usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info/
/usr/local/lib/python3.6/site-packages/wheel/
/usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info/



The bold directories 2-4 do not survive reboot and pip3 fails after reboot.  What needs to be done to maintain reboot persistence?
 


« Last Edit: March 01, 2020, 01:33:23 PM by gatorback »
Raspberry Pi Zero W

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11178
Re: pip3 directory persistence
« Reply #1 on: March 01, 2020, 01:28:52 PM »
Hi gatorback
... Four new directories (tagged with current date \ time):

/usr/local/lib/python3.6/site-packages/pip/
/usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info/
/usr/local/lib/python3.6/site-packages/wheel/
/usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info/


are added to the file and backed up to ensure persistence between reboots: ...
You might need to lose those trailing slashes if your looking to back up subdirectories:
Code: [Select]
usr/local/lib/python3.6/site-packages/pip
usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info
usr/local/lib/python3.6/site-packages/wheel
usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info

You also don't want the leading slashes.

If those subdirectories are not subject to changes, you might want to create an extension for yourself:
Code: [Select]
tce-load -w squashfs-tools
tce-load -i squashfs-tools
mkdir -p /tmp/MyPIP/usr/local/lib/python3.6/site-packages
cp -a /usr/local/lib/python3.6/site-packages/pip /tmp/MyPIP/usr/local/lib/python3.6/site-packages/
cp -a /usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info /tmp/MyPIP/usr/local/lib/python3.6/site-packages/
cp -a /usr/local/lib/python3.6/site-packages/wheel /tmp/MyPIP/usr/local/lib/python3.6/site-packages/
cp -a /usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info /tmp/MyPIP/usr/local/lib/python3.6/site-packages
cd tmp
mksquashfs MyPIP pip3.tcz

Offline andyj

  • Hero Member
  • *****
  • Posts: 1020
Re: pip3 directory persistence
« Reply #2 on: March 01, 2020, 04:02:44 PM »
Need to lose the leading slashes too.

Offline gatorback

  • Newbie
  • *
  • Posts: 17
Re: pip3 directory persistence
« Reply #3 on: March 01, 2020, 07:27:13 PM »
ATTEMPT TO REMEDY DIRECTORY PERSISTANCE
=============================
The directory slashes were removed at the head and tail:
Code: [Select]
tc@box:~$ tail /opt/.filetool.lst
usr/local/etc/ssh/ssh_host_ed25519_key.pub
usr/local/etc/ssh/ssh_host_rsa_key
usr/local/etc/ssh/ssh_host_rsa_key.pub
usr/local/lib/python3.6/site-packages/pip 
usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info
usr/local/lib/python3.6/site-packages/wheel 
usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info
usr/local/lib/python2.7/site-packages/megaio-1.3.1-py2.7.egg/megaio
usr/local/lib/python2.7/site-packages/megaio-1.3.1-py2.7.egg

then ensured that pip3 was installed:

Code: [Select]
cd /home/tc/pip3Persistent
#wget https://bootstrap.pypa.io/get-pip.py
sudo -H python3.6 get-pip.py

performed a backup:
Code: [Select]
filetool.sh -b
rebooted piCore: directories did not survive.  What am I missing?

ATTEMPT TO CREATE PIP3 EXTENSION
=======================

Reinstalled pip3 with:

Code: [Select]
#mkdir cd /home/tc/pip3Persistent
cd /home/tc/pip3Persistent
#wget https://bootstrap.pypa.io/get-pip.py
sudo -H python3.6 get-pip.py

.tcz extension created and copied it to /mnt/mmcblk0p2/tce/onboot.lst
Code: [Select]
sudo cp -a /usr/local/lib/python3.6/site-packages/pip /tmp/MyPIP/usr/local/lib/python3.6/site-packages/
sudo cp -a /usr/local/lib/python3.6/site-packages/pip-20.0.2.dist-info /tmp/MyPIP/usr/local/lib/python3.6/site-packages/
sudo cp -a /usr/local/lib/python3.6/site-packages/wheel /tmp/MyPIP/usr/local/lib/python3.6/site-packages/
sudo cp -a /usr/local/lib/python3.6/site-packages/wheel-0.34.2.dist-info /tmp/MyPIP/usr/local/lib/python3.6/site-packages
cd /tmp
mksquashfs MyPIP pip3.tcz

cp pip3.tcz  /mnt/mmcblk0p2/tce/onboot.lst

Rebooted piCore:  directories of interest seem to be OK (persist):

Code: [Select]
tc@box:~$ ls -lt /usr/local/lib/python3.6/site-packages/
total 0
drwxr-xr-x   12 root     root           480 Jan  1  1970 pip/
drwxr-xr-x    2 root     root           180 Jan  1  1970 pip-20.0.2.dist-info/
drwxr-xr-x    4 root     root           280 Jan  1  1970 wheel/
drwxr-xr-x    2 root     root           180 Jan  1  1970 wheel-0.34.2.dist-info/
lrwxrwxrwx    1 root     root            66 Jan  1  1970 README -> /tmp/tcloop/python3.6/usr/local/lib/python3.6/site-packages/README
lrwxrwxrwx    1 root     root            70 Jan  1  1970 README.txt -> /tmp/tcloop/python3.6/usr/local/lib/python3.6/site-packages/README.txt
drwxr-xr-x    2 root     root            40 Jan  1  1970 __pycache__/
lrwxrwxrwx    1 root     root            75 Jan  1  1970 easy_install.py -> /tmp/tcloop/python3.6/usr/local/lib/python3.6/site-packages/easy_install.py
drwxr-xr-x    2 root     root           200 Jan  1  1970 pip-9.0.1.dist-info/
drwxr-xr-x    5 root     root           120 Jan  1  1970 pkg_resources/
drwxr-xr-x    5 root     root           680 Jan  1  1970 setuptools/
drwxr-xr-x    2 root     root           240 Jan  1  1970 setuptools-28.8.0.dist-info/
drwxr-xr-x    3 root     root           100 Jan  1  1970 smbus2/
drwxr-xr-x    2 root     root           140 Jan  1  1970 smbus2-0.3.0-py3.6.egg-info/

 ./pip-9.0.1.dist-info/ was removed when pip3 was installed, but it somehow is installed: maybe part of python3.tcz?

pip3 test failed.

Code: [Select]
tc@box:~$ pip3
-sh: pip3: not found
tc@box:~$ pip36
-sh: pip36: not found
Raspberry Pi Zero W

Offline vladosam

  • Newbie
  • *
  • Posts: 11
Re: pip3 directory persistence
« Reply #4 on: March 03, 2020, 12:06:33 PM »
Did you tried to load your pip.tcz with tce-load -ic pip.tcz?