HPCC

Owned by David Harms
Last updated: Oct 04, 2013 by RobertF
2 min read

HPCC stands for High Performance Computing Cluster, an open source software project created by LexisNexis.

The LexisNexis usage is in data processing that cannot be accommodated by a single machine. The answer is to split up the data and the processing so you can have from several to hundreds (or perhaps thousands) of machines (or nodes) all working away at pieces of a larger problem. HPCC is divided into two primary computing clusters; THOR, which is the Data Factory (or Refinery), responsible for cleaning and linking large amounts of data in many formats, and ROXIE, which is the Rapid Data Delivery Engine responsible for providing query results at sub-second response times. Behind both of these clusters is the Enterprise Control Language, which is ideally suited for the manipulation of big data. It is scalable, extensible, non-procedural and declarative.

There's a significant Clarion connection here: many of the developers who created Clarion for Windows now work for LexisNexis, and specifically they're on the team that created the High Performance Computing Cluster (HPCC) product. The HPCC team includes a number of former Clarion folk such as Bob Foreman, Richard Taylor, Jim DeFabia, Richard Chapman, David Bayliss, Gavin Halliday, Gordon Smith, Jake Smith, Jo Prichard, Trent Buck, Greg Whitaker, Mike Gould, and the late Nigel Hicks.

Another Clarion connection is in the Clarion.Net product itself. Clarion developers can use the Clarion.NET development tool to create wrappers around the ROXIE queries used by their customers. Examples of protocols supported include HTML, SOAP, JSON, and REST. (UPDATE: Check out Dries Dressen's presentation at the CIDC 2013 conference, you can now use Win32 Clarion to do the same thing.)

In October 2011 Bob Foreman gave a presentation on HPCC at the Clarion DevCon. Read the report in Clarion Magazine.

In Feb 2012 HPCC announced a one-click solution for deploying an HPCC cluster in the cloud.

And in 2013, the HPCC website has added much more, including language extensions to ECL. In addition to C++ which has always been supported, there is now capabilities to add Python, Javascript, and R inline to any ECL workunit. We also introduced FREE online training to help you familiarize yourself with HPCC and ECL at your own pace.

Mike Hanson points out that HPCC is on Twitter @hpccsystems. Also, Bob Foreman mentioned that there is a Facebook page: HPCCSystems

For more information, including documentation, tutorials, and downloads, visit the HPCC Systems web site at http://hpccsystems.com