LexisNexis joins Linux Foundation, open sources HPCC platform

LexisNexis Risk Solutions, the business intelligence provider, has announced that it has joined the Linux Foundation. The announcement comes in the wake of the company open sourcing its HPCC (High Performance Computing Cluster) – its own alternative to Hadoop for "Big Data" processing – with commercial and community editions. HPCC Systems is the name of the company set up by LexisNexis to market the platform, which makes use of high performance clustered systems, a market where Linux is very popular. The company has joined the Foundation to engage with the community and expand its work in that area.

LexisNexis developed HPCC over the last ten years, as it created systems for its own use to process billions of records and provide the results to its customers. "We've been doing this quietly for years for our customers with great success" said James Peck, CEO of LexisNexis Risk Solutions who hopes the community will spur greater adoption of, and innovation in, the company's dual licensed platform.

How HPCC works
Source: HPCC Systems

HPCC is built around ECL (Enterprise Control Language), an extensible, datacentric language, which comes with an IDE. ECL code is then run on Thor or Roxie nodes; Thor is HPCC's "Data Refinery" which processes data, while Roxie is a "Rapid Data Delivery Engine" designed to search for single records or sets of records. The community edition of HPCC is licensed under the GPLv3, but despite the announcement, binary and source downloads are marked as coming soon. An FAQ and other resources on the HPCC Systems site provide more information.

Potential community contributors should note some unique elements to HPCC Systems' community. The terms and conditions attached to registering for a forum login include, in section 4.6, an exclusive option for HPCC Systems to purchase all rights to any "patentable subject matter that you submit" to the web site for $1000. The terms and conditions are dated 2009 and as such are designed for the community around the proprietary edition of HPCC. There may be a revised version of the terms and conditions in future as the company completes its open sourcing. There is also a contribution agreement which is only disclosed upon the contribution of code.

(djwm)