TL;DR

Software based privacy tools rely on the cryptography. Popular crypto software components, such as OpenSSL, are used to build these tools. New versions of crypto software components are often security updates. In safety and privacy critical software it might be safest to expect these components to be up to date. Component versions used in the latest tools mentioned on privacytools.io page were analyzed. Component versions were used to calculate “best before dates” for the tool downloads. I found out that for my taste some of the tools may have gone stale and their safety may have expired.

Survey itself

chose target tools systematically from the https://www.privacytools.io
collected all the tools with simple downloads from there
recorded tool’s name, platform, download format and download URL
recorded the reason for skipping some of the tools

skipped Android, IOS, web application and plugin based tools
skipped not-yet-released and build-it-yourself tools
skipped downloads of over 100MB due to an analysis tool size limit

downloaded the tools over the weekend of 2015-04-11 - 2015-04-12
uploaded all the downloaded tools to the http://bomtotal.com
recorded the resulting Bill of Materials URLs
recorded the automatic tool version identification
recorded the identified OpenSSL, OpenVPN, NSS, GnuTLS and PolarSSL versions
decided to focus on the OpenSSL versions due to lack of time
collected the OpenSSL version history from the OpenSSL release notes
used the OpenSSL version history to determine best before dates for the old versions
compared identified OpenSSL versions for the tools against the OpenSSL history
calculated a best before date for each tool with an identified OpenSSL component
published the resulting survey spreadsheet
wrote and published this study document

Results

The survey spreadsheet is available with all the details from:

https://docs.google.com/spreadsheets/d/12YWSVuItaXtsgY2kH134D1_vt1Nkszs_QrhkPISHrgQ/pubhtml# (html)

https://docs.google.com/spreadsheets/d/12YWSVuItaXtsgY2kH134D1_vt1Nkszs_QrhkPISHrgQ/edit?usp=sharing (javascript)

Roughly 1/3rd of tools were identified to contain OpenSSL, roughly 1/3rd didn’t contain OpenSSL and for roughly 1/3rd of the tools the analysis was not applicable. Sometimes the BOMtotal failed to identify any components in the Bill of Materials.

Different tools contained traces of OpenSSL versions 0.9.6c, 0.9.8e, 0.9.8y, 0.9.8za, 1.0.0e, 1.0.0f, 1.0.0k, 1.0.1c, 1.0.1e, 1.0.1g, 1.0.1h, 1.0.1j, 1.0.1k, 1.0.1l, 1.0.1m and 1.0.2a. That is 16 different versions, some of them being seriously out of date or outright dangerous. Versions affected by the Heartbleed vulnerability and a version over 10 years old were found. In some cases the OpenSSL was perfectly up to date.

Some tools contained multiple versions of the OpenSSL, worst one containing three different versions. Same happened with other components as well, e.g. multiple versions of the zlib were found from a single download. One download had three competing SSL/TLS implementations (OpenSSL, GnuTLS and PolarSSL) in its list of ingredients.

The best before date was calculated based on the oldest version of the OpenSSL used in the tool binary in question. In the case of multiple versions of the OpenSSL this calculation could have been done based on the latest version included but better to be safe than sorry?

Even if the software contains traces of vulnerable OpenSSL, it may not be exploitable due to how it is used, but again better to be safe than sorry?

Browsers stood out as a category of tools with most up to date components.

Credits & Thanks

To the authors of privacy tools covered in this survey for the tools themselves
To the authors of privacytools.io for a valuable compilation
To the EFF for promoting privacy and electronic rights
To Ossi for the idea of going through the privacytools.io systematically
To my school for not giving me homework for this weekend
Least to my dad for help in translating this into somewhat English :)

Conclusions

Authors of the privacy tools should keep at least the crypto components up to date
Authors of the browsers are doing a good job, other tool categories could improve
Authors of the critical components should provide a machine parseable version history
The best before dates should be tracked both for the component and tool versions
Someone could survey the other critical components (BouncyCastle, zlib, ...)
Someone could survey the tools skipped in this study (at least Android and IOS)
Someone could determine the best before dates based on other components as well
I learned a lot about the available privacy tools thanks to the privacytools.io
Caveat Emptor, some tools may have gone stale and their safety may have expired