Report: Open source software quality is better than proprietary software

Coverity's latest Coverity Scan Open Source Report has found that the quality of open source code is equal to, or even better than that of proprietary software. This is the third open source report from Coverity since it took over producing the report in 2009. The project was originally founded by the US Department of Homeland Security in conjunction with Coverity in 2006. Coverity's conclusion is that the assumption that open source software is of lower quality should not be considered a valid reason for avoiding open source.

The study investigates the integrity and quality of open source software and proprietary software using Coverity's own static analysis platform, which looks for defects such as pointer dereferencing, uninitialised variables, memory corruption or control flow issues. It does not look at flaws found through testing or while the software is in use. Coverity also only counts "high" or "medium" severity defects for the purposes of the report.

The results are based on an analysis of more than 37 million lines of open source code and over 300 million lines of proprietary software code. This year is the first year that proprietary code from Coverity users, albeit anonymised, has been included in the survey. The proprietary sample is taken from a variety of industries and attempts to match project ages with the length of adoption times of the various open source projects.

The researchers examined the code of 45 major open source projects. On average, the examined projects consisted of about 820,000 lines of code. The open source projects were found to have an average defect density of 0.45, based on the number of defects per 1,000 lines of code.

Coverity found that there were 0.64 defects in every 1,000 lines of proprietary software code. For its study, the company examined 41 of its customers' software developments, which on average included 7.55 million lines of code. This defect density is, however, still below Coverity's 1.0 benchmark for high quality software.

In their announcement of the latest edition of their open source report, the Coverity researchers highlighted Linux 2.6, PHP 5.3 and PostgreSQL as projects of excellent code quality, calling them "model citizens". These projects were found to have defect densities of 0.62, 0.20 and 0.21, said Coverity. Linux was commended because its approximately 7 million lines of code exhibit a defect density that is almost identical to that of competing proprietary software products.

Open source developers can make use of Coverity's Scan through a programme for appropriately qualified software and organisations. Coverity hopes to extend the service to additional projects in 2012.

(djwm)