Openssl and Python versions for TC 4.x repo.

[Jason W]

With the approaching of TC 4.x, we need to decide if we want to update openssl and we need to decide on what python version we will use.

There needs to be only one version of python that we stick with for the duration ov TC 4.x.  I recommend staying with 2.7.x, as many things still require python 2.x and I think we are losing almost nothing by sticking with a 2.x version.

Openssl is of course at version 1.0.0 upstream.  Building that extension would be easy, but it would require rebuilding a huge portion of the repo, losing so many existing extensions that could otherwise just be copied over.  I lean towards staying with 0.9.8 as it is still supported.  The cost of moving to 1.0.0 will be very steep, but benefits would be few if any aside from version number.

I would like to hear what the extension makers think on this.

[bmarkus]

Python: Agree, use the latest stable 2.x version.

openssl: I'm not really familiar with it so can't say nothing about real benefits having 1.0.0 but for sure, rebuild extensions just to have 1.0.0 would take extra energy from all extension maintainers. I vote for 0.9.8

(Checked RedHat EL6, there is 1.0.0. But it is a commercial product)

[Jason W]

Yeah, most distros are on openssl 1.0.0 now, but their package building and updating model is different than ours and allows more frequent updates.  Debian stable is still on 0.9.8, though.

THere are 996 tcz extensions that have openssl as a dependent in their dep tree.  And there are no known vulnerabilities that have not been fixed by the "r" version of 0.9.8 that we are now using.  Also, I think I remember reading that most of openssl exploits that are discovered only affect openssl when used as a server, and not when used as a shared lib.  We use it here as a shared lib almost exclusively.

If someone wants to use a newer version of openssl to use it as a server, it is a simple "configure && make && make install" to keep up with the latest version.  And if one wants to use the newest openssl to build against, they can either package the newer libs in a self contained extension or link to a newer openssl statically.

I want to hear what the other main extension makers think, but at this point I think we probably are all on the same page.

[robc]

I think python 2.7.x would be better then 3.x at this point.

I don't have any problem with updating my extensions to openssl 1.0.0.

On a note about the extensions.  Since the repo is growing and will keep growing, there should be a more standardized way to rebuild the extensions.  This will help out the TC team and the extension makers when extensions like openssl and python are updated.  Perhaps an extension could be made that can pull the build script and source for an extension and build it automatically.

[Arslan S.]

as long as my build does not complain about ssl version it is ok for me
also i see no harm to have a seperate python3 extension, there are some projects that requires python3

[Jason W]

If we were also updating python and perhaps one other major low level toolkit or lib that would require a mass rebuild, then I would think it is time to rebuild.  But the consensus seems to be to stay with our current openssl.

As for a build system, I prefer to use an automated "one command" script that creates the extension and takes care of the details by just running the script.  Much easier that way during updates though it takes more initial time investment.  But at the same time, each person has his own level of automation and requiring a certain style or level of scripting would raise the barrier for contribution.    I am one who likes to say "never say never", but source distros have enough issue maintaining and troubleshooting an automated build process across all their packages when that is their sole focus.

[danielibarnes]

Will there be any further updates for the 0.9.8 series? If not, then any vulnerabilities discovered will never be addressed and TC 4.0 will be out of date for its entire existence.

I support choosing 1.0.0 simply because we will be able to update it and stay current. I believe the pain of updating extensions cannot be avoided, only delayed. Eventually, I expect there will be a security issue with 0.9.8 that will render that series insecure. If the only option to remain secure is to use a more recent version, then we may be forced to upgrade quickly rather than at a time and pace of our choosing.

there should be a more standardized way to rebuild the extensions.

This, too, is very difficult, but it would be a big boost for Tiny Core. Standardization would also increase the quality of extensions. Note, however, that this doesn't mean Jason or anyone else on the Tiny Core Team has to do the work. They can spell out the requirements, but anyone is free to take up the banner.

[robc]

also i see no harm to have a seperate python3 extension, there are some projects that requires python3

I agree with this.  The main python.tcz extension should be 2.7 but a python3 extension would be nice and add flexibility.

there should be a more standardized way to rebuild the extensions.

This, too, is very difficult, but it would be a big boost for Tiny Core. Standardization would also increase the quality of extensions. Note, however, that this doesn't mean Jason or anyone else on the Tiny Core Team has to do the work. They can spell out the requirements, but anyone is free to take up the banner.

I wasn't proposing a complete building standard but rather the requirement of a build script and auto-installation of required deps for building the package in the script.  And possibly a standard name for the build script.  This would make large update processes easier to handle.

[Jason W]

As for an openssl update being delayed instead of avoided, that is true.  But 0.9.8 is used in Debian Stable, which was released this last February.  And they will be supporting that version of openssl for the life of stable as well as when it moves to oldstable.  And that would be surely later than the end of our 4.x lifecycle.  Their security patches can be used assuming there are any vulnerabilities that are not fixed by openssl itself.  The "r" version of openssl was released in Feb, the same time the current 1.0.0d was released, so they are still supporting it upstream.

I see the logic on both sides.  The openssl changelog that has the CVS changes already mentions version 1.1.0.  Instead of rebuilding everything for TC 4.x, just to feel the need to update to version 1.1.0 for TC 5.x, lets wait until 5.x and certainly plan on an openssl update that will surely be 1.1.0 by then.

I won't fight there being a python 3.x extension, but if someone decides to start updating all their stuff to use python 3.x while other packages dependent on them require 2.x, then we can have a mess on our hands.  The existence of a 3.x python would have to be wielded with care.

[Jason W]

On the openssl end, I have uploaded an openssl-1.0.0-static extension that is installed to /usr/local/openssl-1.0.0 if one wants to build apps with the latest openssl linked statically.  This eliminates the need for an openssl extension dependency in the resulting app.

I have done this for wget, links, and curl.  Curl how has static openssl and libssh2 functionality without any dependencies.  I have tested several apps that depend on a combination of curl and openssl and all seems to be working well.  I will continue to test and please report any issues found that may be related.  Since some apps need openssl yet had their openssl dependency satisfied by their curl dependency, I had to add openssl-0.9.8.tcz to the dep file of these apps.  Please remember to include openssl-0.9.8.tcz in the dep files when updating the apps mentioned below, if you are making any changes to the existing dep file upon extension update.

CurlFtpFS.tcz.dep
ario.tcz.dep
bitstormlite.tcz.dep
centerim.tcz.dep
drizzle-server.tcz.dep
ecore.tcz.dep
fbreader.tcz.dep
gammu-2.6.tcz.dep
gdal.tcz.dep
gnash.tcz.dep
gpredict.tcz.dep
libtunepimp.tcz.dep
logjam.tcz.dep
mpd-0.15.12.tcz.dep
mpd-minimal.tcz.dep
ncmpcpp.tcz.dep
netsurf.tcz.dep
openlierox.tcz.dep
php5.tcz.dep
raptor.tcz.dep
rtorrent.tcz.dep
sword.tcz.dep
tangogps.tcz.dep
transmission-gtk.tcz.dep
transmission.tcz.dep
virtualbox-ose.tcz.dep
vorbis-tools.tcz.dep
xine-ui.tcz.dep
xine-xvesa.tcz.dep

[bmarkus]

gpredict doesn't depend directly on openssl, only via curl, therefore no need to add openssl-0.9.8.tcz to dep file. Please revert back to the previous dep file. Tested, it works fine in current environment without openssl-0.9.8.tcz installed.

[Jason W]

Ok, done.

I was using an ldd output of the ELF binaries and libraries of the stuff in the repo, and the last time I ran it the gpredict binary depended in libssl-0.9.8, but no longer with the current extension.  I will regenerate the ldd maps and look over them again to make sure others have not changed also. 

[Jason W]

python.tcz is now built statically against openssl-1.0.0d.  I carefully tested the dep files versus the ldd output of the repo and according to my findings anything that depends on python does not in and of itself depend on openssl but for the previous python dependency.  It is basically a build time only dependency that is no longer a required shared library during use. 

But please inform me if there are any issues as always. 

[Jason W]

I miscalculated, the extensions below depend on openssl that was satisfied by the python dep but now needs it in their own dep file.  I added the openssl dep to these.

Code: [Select]

pycurl.tcz
openoffice3.tcz
ntop.tcz
gnome-python-desktop.tcz
libreoffice.tcz
deskbar-applet.tcz
python-libxml2.tcz



EDIT:  A static openssl in curl and python brings down the number of extenions that have openssl somewhere in their dep tree down from 996 to a current 529 in the 4.x repo.  Hopefully will be helpful in the future.