Terra and Lua offer new high performance computing strategy

Zach DeVito and a team at Stanford and Purdue University have published details of Terra, a new approach to generating code for high performance computing. Using Lua as a linguistic host for a new low-level language, the team have come up with a system which allows a developer to write high-level code in Lua and high-performance code in Terra, iterating code from high-level experimentation to high-performance optimised code over time. This is, though, just one of the use cases for Terra and Lua.

The developers say it can also work as an embeddable JIT compiler, thanks to Lua's ability to be easily embedded into C applications, and generating Lua programs at runtime and then compiling them into Terra code. Terra is also a stand alone low level language which can be run independently of Lua; Terra code can be saved by the Lua code which contains it as a .o or executable file.

This approach is, they believe, superior to the way domain specific languages are created for HPC with optimisers doing domain specific transformations for execution planners that feed code to complilers that build for a runtime. The complexity makes it difficult, for example, to feed back runtime statistics to do problem specific optimisations.

Terra answers that call with by allowing developers to write their high-level code, which can interact with Terra code in the same files, in the same lexical space so that variables are hygienically available in both Lua and Terra. Terra's low level code generation then runs the Terra code in its own environment, independently of Lua's runtime to ensure that no Lua features creep into the Terra execution. Terra can also access features of hardware such as vector instructions.

To test their proposition, the developers of Terra have reimplemented various multi-language systems entirely in Terra, with the language's auto-tuner getting within 20% of ATLAS code and the team's DSL for stencil computations running over twice as fast as hand-written C. Details of how Terra can be used for generative programming and language compilation and how embedding and interoperability works are on the Terra site.

There's also a Getting Started guide, which covers what you need to run Terra (Mac OS X or Linux, LLVM 3.2, Clang 3.2 and LuaJIT 2.0), an API reference and a research paper which will be presented at June's Programming Language and Implemention 2013 conference in Seattle, Washington. Terra's code is available on on GitHub under the MIT licence.

(djwm)