Tiny Core Linux

Tiny Core Extensions => TCE Q&A Forum => Topic started by: drendar on September 04, 2015, 05:58:35 AM

Title: Python SSL issues, no easy_install or PIP
Post by: drendar on September 04, 2015, 05:58:35 AM
Hi All,

I'm having a hell of a time trying to get some python modules loaded up - so I've been trying to install PIP.

In order to install pip, I obviously need easy_install (Part of python-setuptools) however this package is missing from the 6.x repo

I followed the instructions at http://forum.tinycorelinux.net/index.php/topic,18781.msg115128.html#msg115128 to download the setuptools package for 5.x and got this installed OK.

The problem I now have is that it seems the python version rolled up into python.tcz doesn't have SSL enabled...

Code: [Select]
tc@box:~$ sudo easy_install pip
Searching for pip
Reading https://pypi.python.org/simple/pip/
Download error on https://pypi.python.org/simple/pip/: unknown url type: https -- Some packages may not be found!
Couldn't find index page for 'pip' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: unknown url type: https -- Some packages may not be found!
No local packages or download links found for pip
error: Could not find suitable distribution for Requirement.parse('pip')

Searching on this error it's recommended in several places that I need to install openssl-dev and then reinstall python.

So, I've now got installed:
Code: [Select]
openssl
openssl-dev
openssl-1.0.1
openssl-1.0.1-dev

Still no change.

I tried removing and reinstalling the python package, but again that made no difference.

So now I'm stuck, and I have no idea how to progress from here...    Do I need to use an older version of TC, which has the required packages?

Many Thanks in advance.

Dash
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 04, 2015, 06:10:51 AM
The openssl-1.0.1 extension was recently updated and renamed to openssl.

If you loaded the updated extension, openssl, you will need to use the apps gui to update your dep files - this will rename openssl-1.0.1 -> openssl in any local dep files containing openssl-1.0.1. After a reboot, you should no longer require openssl-1.0.1 to be loaded.

I sort of doubt that having openssl and openssl-1.0.1 both loaded is causing your problem, but it is worth checking.
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 04, 2015, 06:29:37 AM
I get the same error as you, so the problem is not having both openssl and openssl-1.0.1 loaded.

Since python-setuptools itself seems to work, it has been copied to the 6.x repo.

In the meantime you could download the pip source code and compile/install it?
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 04, 2015, 08:06:31 AM
If I compile pip, running it fails with "ImportError: cannot import name HTTPSHandler".

@BMarkus - has python has been compiled without openssl support?
Title: Re: Python SSL issues, no easy_install or PIP
Post by: bmarkus on September 04, 2015, 09:04:58 AM
@BMarkus - has python has been compiled without openssl support?

Built with SSL support. Try

Code: [Select]
import ssl
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 04, 2015, 09:28:04 AM
Hmm - looks like the problem is there:
Code: [Select]
$ python
Python 2.7.10 (default, May 25 2015, 09:55:35)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: /usr/local/lib/python2.7/lib-dynload/_ssl.so: undefined symbol: SSLv2_method

the latest openssl has sslv2 and sslv3 disabled due to one being broken and the other being a security risk...
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 04, 2015, 11:21:45 AM
python recompiled and posted, python_pip added.

Things seem to work now...
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 05, 2015, 04:44:59 AM
python reposted, better optimised for size
Title: Re: Python SSL issues, no easy_install or PIP
Post by: drendar on September 08, 2015, 12:30:09 PM
Awesome, thanks guys!!
Title: Re: Python SSL issues, no easy_install or PIP
Post by: drendar on September 10, 2015, 10:35:54 AM
Hey guys,

There looks to be a missing dep (libffi.tcz) on python-setuptools.  This causes both setup_tools and pip to fail.


Code: [Select]
tc@IDF-Console:/mnt/sda2/tce/optional$ tce-load -iw python_pip
python_pip.tcz.dep OK
python-setuptools.tcz.dep OK
Downloading: python-setuptools.tcz
Connecting to repo.tinycorelinux.net (89.22.99.37:80)
python-setuptools.tc 100% |********************************************************************|   508k  0:00:00 ETA
python-setuptools.tcz: OK
Downloading: python_pip.tcz
Connecting to repo.tinycorelinux.net (89.22.99.37:80)
python_pip.tcz       100% |********************************************************************|  1212k  0:00:00 ETA
python_pip.tcz: OK


tc@IDF-Console:/mnt/sda2/tce/optional$ sudo pip install pyserial
Collecting pyserial
  Downloading pyserial-2.7.tar.gz (122kB)
    100% |################################| 122kB 1.1MB/s
setuptools must be installed to install from a source distribution



Removed pip, installed only setuptools, tried to install PIP via setuptools:
Code: [Select]
ImportError: libffi.so.6: cannot open shared object file: No such file or directory

So I loaded libffi along with python_pip again and it's all working!
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 10, 2015, 11:36:20 AM
As python is compiled to use the system libffi, it's probably more correct to put libffi as a dep of python - dep file adjusted, thanks.
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Misalf on September 12, 2015, 06:56:10 AM
Hi
It seems to me that there is still something that needs fixing.
After updating python and openssl today (plus adjusting .dep files and reboot), I'm not able to use youtube-dl anymore.

Code: [Select]
$ youtube-dl
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/tc/.local/bin/youtube-dl/__main__.py", line 16, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/__init__.py", line 16, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/options.py", line 8, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/downloader/__init__.py", line 3, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/downloader/common.py", line 9, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/utils.py", line 25, in <module>
  File "/usr/local/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: /usr/local/lib/python2.7/lib-dynload/_ssl.so: undefined symbol: SSL_CTX_set_alpn_protos

Code: [Select]
$ python
Python 2.7.10 (default, Sep  4 2015, 16:16:17)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: /usr/local/lib/python2.7/lib-dynload/_ssl.so: undefined symbol: SSL_CTX_set_alpn_protos
>>>

Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 12, 2015, 08:27:50 AM
I just double-checked that I have the latest versions of openssl and python - I do - and everything works fine:
Code: [Select]
$ python
Python 2.7.10 (default, Sep  4 2015, 16:16:17)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>>

$ /tmp/youtube-dl/youtube-dl 'https://www.youtube.com/watch?t=4&v=BaW_jenozKc'
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading video info webpage
[youtube] BaW_jenozKc: Extracting video information
[youtube] BaW_jenozKc: Downloading DASH manifest
[youtube] BaW_jenozKc: Downloading DASH manifest
[download] Destination: youtube-dl test video ''_-BaW_jenozKc.mp4
[download] 100% of 1.53MiB in 00:19

Maybe you used a mirror that hasn't caught up yet?
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Misalf on September 12, 2015, 08:47:06 AM
I'm using  http://repo.tinycorelinux.net/ .
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 12, 2015, 08:54:44 AM
I'm surprised that the apps gui "check for updates" function doesn't show that either openssl or python needs updating.

Perhaps you still have openssl-1.0.1 loaded via the dep file of another extension?

If not, then maybe you could try deleting your local copies of openssl and python and downloading them again with the apps gui?
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Misalf on September 12, 2015, 09:52:20 AM
openssl-1.0.1 was not loaded.
I have re-downloaded openssl.tcz and python.tcz, then rebooted. Error message persists.
Same if I boot with "base norestore".

262203f41e4b60b0ac6fffafe503fb1d  python.tcz
cd5d7fb1b0bacecd0ac120de33ac706d  openssl.tcz
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Juanito on September 12, 2015, 10:06:14 AM
Code: [Select]
$ cat /mnt/sdb1/tce/optional/openssl.tcz.md5.txt
cd5d7fb1b0bacecd0ac120de33ac706d  openssl.tcz
$ cat /mnt/sdb1/tce/optional/python.tcz.md5.txt
262203f41e4b60b0ac6fffafe503fb1d  python.tcz
$ python
Python 2.7.10 (default, Sep  4 2015, 16:16:17)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>>

very odd  :o
Title: Re: Python SSL issues, no easy_install or PIP
Post by: gordon64 on September 12, 2015, 10:33:44 AM
Hi

On x86 youtube works for me with similar md5s as Juanito. Maybe Misalf could ensure python is not in his "boot" list, reboot  and show his manual loading to see if that is normal? However I still had two deps loading in my boot list...due to mc and lxterminal. My python dep contents
Quote
sqlite3.tcz
bzip2-lib.tcz
openssl.tcz
gdbm.tcz
ncurses.tcz
readline.tcz

eg mine is
Code: [Select]
tce-load -i python
readline.tcz: OK
gdbm.tcz: OK
openssl.tcz: OK
sqlite3.tcz: OK
python.tcz: OK

possible redudant dep of ncurses as its a dep of readline?

good luck
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Misalf on September 12, 2015, 11:03:31 AM
gordon64, shouldn't you have  libffi.tcz  in your python.tcz.dep?

Without having python in onboot.lst:
Code: [Select]
16:57:53 tc;box:~/$> youtube-dl
env: can't execute 'python': No such file or directory
16:57:56!tc;box:~/$> tcz python
gdbm.tcz: OK
python.tcz: OK
16:58:02 tc;box:~/$> youtube-dl
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/home/tc/.local/bin/youtube-dl/__main__.py", line 16, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/__init__.py", line 15, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/options.py", line 7, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/downloader/__init__.py", line 3, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/downloader/common.py", line 9, in <module>
  File "/home/tc/.local/bin/youtube-dl/youtube_dl/utils.py", line 25, in <module>
  File "/usr/local/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: /usr/local/lib/python2.7/lib-dynload/_ssl.so: undefined symbol: SSL_CTX_set_alpn_protos
16:58:07!tc;box:~/$>

PS: The command  tcz  is just a wrapper script for  tce-load -i .
Title: Re: Python SSL issues, no easy_install or PIP
Post by: Misalf on September 12, 2015, 12:02:52 PM
Sorry guys! Entirely my fault.
I have some custom initrds and my boot stanza for a clean boot was not correct.

Everything works as expected.
Title: Re: Python SSL issues, no easy_install or PIP
Post by: gordon64 on September 12, 2015, 08:00:38 PM
Misalf

Quote
shouldn't you have  libffi.tcz  in your python.tcz.dep?

thanks fixed, but youtube worked without it.