openssl version

[maro]

In light of Jason's current efforts to clean-up the repository I wonder what should happen with the openssl extensions:

We have currently two versions in our repository openssl-0.9.8h.tczl and openssl-0.9.8k.tcz (plus the respective -dev extensions). Taking a cue from the .dep files the vast majority of extensions depends on the older version: 103 (+3 devs) vs. 6.

Now judging by the timestamp of the source tar-balls (http://www.openssl.org/source) those releases are either 18 or 8 month old. The latest stable release (openssl-0.9.8l) is ca. 3 weeks old. As I understand this latest release was made to "... Disable renegotiation completely - this fixes a severe security problem (CVE-2009-3555) ..."

I'm sure there was a good reason to keep an older library version around, but in light of the ever increasing focus on security aspects I wonder what will be required to update all extensions to use the latest version.

I understand that breaking many extensions all at once by withdrawing an old library won't be popular. But I'd like to also understand what were the type of problems that required the older version.

[Jason W]

Here is also where this issue and it's options were being discussed:

http://forum.tinycorelinux.net/index.php?topic=2889.0

In the beginning, I didn't think of placing openssl in it's own self contained directory so it could be updated seamlessly and not break apps.  Openssl is one of the few extensions that must break the current naming conventions as any update to it requires a recompile of it's dependent apps.  So knowing the version number of the extension by it's extension name is a must to avoid trouble.
 
There are two ways to use openssl, as a shared lib and as openssl.  Most everyone uses it as a shared lib as opposed to using it as an application.  From what I understand, most vulnerabilities come from using openssl as openssl and not as a shared lib.  Thus it is perhaps not as vital to keep up with the current for shared use.  And obviously, updating the openssl-0.9.8h that so many extensions are built against is not going to happen.  Any updates will be to a self contained directory, which is what I have read openssl is normally installed to anyway.  Therefore the extension builder has a choice to use the more up to date version to build against. 

For those who use openssl as openssl, I could build an openssl-current extension that is to never be built against but there for use as an app.  That is one way to have a current openssl.

What I have planned is to update openssl about once every year, leaving the old ones in the repo for continuity until no longer needed.  While I am one who generally likes to keep current, we cannot stay totally current with openssl in a sane manner.  The latest version was just released this month, and before that the openssl-0.9.8k extension was current. 

I would encourage folks to use the most current openssl extension to build against, but this is one area where it is left to the extension builder which one to use. 

[Jason W]

This curl dependency situation brings up the issue of openssl updates.  openssl-0.9.8m is released, and I will make an extension for it contained in /usr/local/openssl-0.9.8m.  I am not sure if anyone else has used openssl-0.9.8k to build against, but I will rebuild my apps to use openssl-0.9.8m and remove openssl-0.9.8k since it has not been yet widely used. 

There is no easy way to manage openssl updates.  And I don't want to have more than two versions in the repo at any given time: the old one, and the one we are migrating to.   Ideally, the newest openssl would be built against until all dependency on the old version is gone, and then the old version removed.  The rate at which that would happen would determine the rate openssl could be updated.  It may become less confusing when we no longer have a version installed into the /usr/local path. 

The alternative would be to keep openssl installed into /usr/local and update it perhaps every or every other major TC upgrade.  But that would be a lot of recompiling, and I personally would rather no forced recompiling and keep it seamless.  I personally prefer the first method mentioned, but I am open to ideas.

[danielibarnes]

I am not sure if anyone else has used openssl-0.9.8k to build against

I was just minutes away from submitting a new lighttpd built against 'k'. I will wait for your 'm' extension.

[Jason W]

I was looking at the repo to see which extensions are build against the "k" version, wireshark and tshark are others.  Sorry you were just ready to submit a build against "k".  I will build "m" tonight. 

With the newly released "m", it should carry us at least for the life of TC 3.x alongside "h" for supporting existing apps. 

[maro]

Jason: After I had posted the original question in November I had a go at building for the purpose of testing a openssl-0.9.8l version in the "standard" way (i.e. in /usr/local/lib). I never followed it up with sufficient testing to post my findings here. Nevertheless the occurrence of the "m" version has now triggered me to post a quick note:

I for one am not sure the a "non-standard" openssl build (as done for "k") is required. In my testing a few month back I observed that all 0.9.8 versions (IIRC I used "h", "k", and "l") could be used as "drop-in replacements" for each other. When I tested against a 0.9.7 version a re-compile of the application under test was required. So I strongly believe that the change of the patch level (e.g. 0.9.8h to 0.9.8m) does not require a rebuild of all dependent applications. I've noted other distributions (e.g. Slitaz) just do "drop-in replacements" of the openssl shared libraries (for the 0.9.8 series) without any changes to applications.

I'd like to learn more about the problems you have mentioned, so could you please point me to articles about those issues that convinced you to maintain two separate 0.9.8 builds for TC 2.x.

I'm very short of time right now and will have to leave the house in a few minutes. I've just done a quick & dirty build of 0.9.8m (with the attached build script). I then tested this build against the "old" version of curl (build against 0.9.8h) where I replaced "m" for "h" in the .dep file. This is by now means sufficient testing, but the only thing I could do in the time available. Way more applications will have to show that they continue to work with such a simplistic replacement. That is what I had planned to do and never got around.

I therefore would like to ask you to hold back with the release of your 0.9.8m extension. Maybe you could repeat some tests with a "standard" build of "m" instead of creating another "non-standard" one.

[Jason W]

You may well have a point.  The article that I read was this one, and looking again at it, it was saying that changes in version levels is what breaks apps and not patchsets.:

http://rt.openssl.org/Ticket/Display.html?user=guest&pass=guest&id=479

I also saw a Solaris article where it states that binary compatibility among patchsets of the same version number is guaranteed by openssl:

http://blogs.sun.com/janp/entry/on_openssl_versions_in_solaris

So we may well be able just to stick openssl-0.9.8m in /usr/local and go with it.  Of course, I would want to do some testing first, but it does sound like it should be possible.

Lunar linux runs "lunar fix" on all openssl dependent packages, but lunar fix is mainly for making sure linked libraries are found, and that would not be affected by a patchset upgrade of openssl. 

If this is the case, then we can keep openssl in /usr/local and keep the latest patchlevel.   Version numbers that would require a rebuild of all dependent apps could happen on a major TC release, but it of course won't be 3.0 and probably won't be 4.0 either.

I know it is a pain in the neck to build against an openssl in it's own directory, and as it appears we can drop in an update it would save a lot of headache in the long run. 

[Jason W]

I have been testing many apps that were built with openssl-0.9.8h by replacing it with 0.9.8m and all seems to be fine.   Since other distros don't rebuild all openssl apps unless there is a version number change, and this is also documented as the planned behavior, I will update openssl in place to 0.9.8m in /usr/local/.  I simply misunderstood the deal of rebuilding openssl apps upon a patchlevel chenge.

Thanks for the build script, a lot of time obviously went into it.  Of course, many extensions that are submitted include very well planned build scripts that make it easy to rebuild the extension.

I will upload an updated openssl extension and name it openssl-0.9.8 since the patchlevel can be updated seamlessly but not the version number.  Will do so in the next day.  Then openssl-0.9.8k can be moved away from and retired.  I will adjust the affected dep files when I upload the new openssl-0.9.8.tcz.

[maro]

After I came home I did a round of testing (mostly browsers) and had written a lengthy post describing my findings (i.e. "all good, no problems"). I had also written a proposal of how to migrate.

I was going to suggest exactly what you have just posted. As I was about to hit the "Post" button I realized that you had meanwhile written essentially the same. So no need to bore the world with my thoughts.

With regards to the build script: Most of it I had used for my 0.9.8l build. But when I read this morning about your plan the build 0.9.8m I quickly made some small changes. So this version of the script has actually not been tested well at all. That was quite a rushed job. I usually would not publish anything which I had only given a quick glance. But you kind of forced me into it since I wanted to raise my hand before you could have gone further down that path of the "non-standard" build. In essence: I hope you reviewed the script, or will review it in case you plan to use it for your 0.9.8 build.

In the end I'm just glad that an easier way forward could be found.

[Jason W]

Openssl 0.9.8 was released in 2005, so that is an at least 5 year cycle between having to rebuild openssl dependent apps and stay current if that is the rate.  That is quite livable.  But the day that openssl 1.0 is released, I will be resistant to the push to upgrade that I know will  follow, as the community as a whole will be ok with 0.9.8 for a while just as 0.9.8h is still the version of choice for most folks.  Perhaps after python 3.0, gtk 3.0,  or another major rebuild situation we can kill at least two birds with one stone during a major TC release by combining that upgrade with the next openssl upgrade.

[Jason W]

I will upgrade openssl tonight when I get home.  I am also glad to know now that openssl can be patchlevel upgraded in place as that will make our lives much easier.

[curaga]

IIRC OpenSSH was an exception, in that it should be built against the exact openssl to run with. I'm not against the update, just saying this one app might need a rebuild for it.

[Jason W]

Good point.  We will rebuild openssh after the update, not a bad idea even if it is not absolutely necessary as it is a critical app closely related to openssl..

[Jason W]

I have uploaded the new openssl to this directory, keeping the patchlevel in the name since it is important to know for things like openssh:

http://distro.ibiblio.org/pub/linux/distributions/tinycorelinux/2.x/tce/src/openssl-0.9.8m/


When openssh is rebuild against this new version I will upload.

Then it is just a matter of changing dep files and weeding out 0.9.8k.

EDIT: I see the strip-unneeded was used in the script for shared libraries, I will change it to strip -g and reupload.

11am: uploaded

[maro]

With regards to OpenSSH: I had done a quick test of the openssh.tcz extension using my 0.9.8m build yesterday and both server (i.e. sshd) and client (i.e. ssh) seemed to behave well.

I'm not saying that a "quick once over" is a proper and full test, and I don't see any harm in rebuilding OpenSSH, I just wonder why this application should be a different case to others (e.g. curl, and all the browsers that are using openssl).

I also like to point out that in my build script I included the "m" in the extension name. I actually wanted to just call it 'openssl-0.9.8.tcz' but changed it in the last moment in an attempt not to "scare the horses". If I understood Jason correctly he also planned to go down that path (of removing the patch level information from the extension name). I just wonder what my happen in the end as the files in the '.../src/openssl-0.9.8m' directory are not yet following this naming convention.