Linaro's DMZ for the ARMies

by Dj Walker-Morgan

Linaro is leading ARM Linux unification efforts, according to its CEO, and the organisation is looking to lead the ARMies of chip makers in other areas of Linux development as well by acting as a safe "demilitarised zone" for the many ARM vendors that make up its membership.

Linaro , a non-profit organisation, was formed in June 2010 by ARM, Freescale, IBM, Samsung, ST-Ericsson and Texas Instruments, in part to take on the problem of an explosion of ARM-based System-on-a-Chip (SoC) implementations and Linux variants to support them. Compared to the desktop processors that we are more familiar with, ARM-based chips allow for a lot of variation, especially in SoC implementations, as the only common component tends to be the ARM-designed cores. With embedded applications though, the software is being written specifically for the chip, so the desktop concerns of portable code are pushed to one side as each chip maker creates numerous specialised ARM chips, each with their own operating system. More and more, that operating system is Linux and with the success of ARM there was also an massive expansion in the number of variants of Linux being sent upstream to the mainstream kernel.

Within nine months of Linaro forming, that expansion became a real issue as Linus Torvalds lost patience. Linaro uses the principles of open source collaboration and collaborating around open source to take on that challenge. Now, with 120 full-time engineers distributed around the world – half of them assigned to the organisation by member companies and 90 of them working on shared common problems – Linaro has grown substantially. The H talked to CEO George Grey and CTO David Rusling to catch up on how they were taking on this challenge.

The H: I hear from some developers wanting to get into ARM development that they are somewhat unsure where to start and where the various paths of ARM are going.

George Grey: I often say that the huge benefit of ARM is its diversity and the ability for these SoC vendors to go and produce different SoC's for different markets and sectors. And the disadvantage of ARM is its diversity, because the result, historically, has been everybody going and creating their own software and upstreaming it, and there's been this explosion. And as ARM has got more successful, the challenges of this have got bigger and bigger and last year was the latest of many problems that have occurred in this space.

I think that one of the big things that Linaro is doing is getting recognition by the bigger players that structurally they could not force their changes upstream on everybody else, not because they didn't have good people or kernel maintainers or whatever, but simply because the upstream maintainers wouldn't take a patch set from one company that was going to affect other companies without their agreement.

Linaro – in its work on the device tree, the ARM SoC tree and now memory management with UMM – has really proved to be a place where competitors can come together and deal with the core architecture that everybody has got to deal with. And a recognition that its better to share investment and ultimately get a single implementation upstream for core ARM architecture and then free up more of their own resources in engineering to focus on their differentiation. It makes a lot of sense, but obviously it's easier to say than do, but I think we're getting there.

The H: So in some ways Linaro is defining the common core of ARM.

David Rusling: Yes, that's a good description of it in terms of code structure and where code should go. The other thing is that if you think of Linux more generally, Linux's heritage is actually via big iron. It has been very influenced by server, less so by desktop. And actually the mobile space has some different attributes about it. It's going strongly and differently with things like BIG.little and different capability cores. It's obsessed with power/energy and battery life, and there's a whole bunch of kernel subsystems that aren't well tuned for that.

We've moved from fixing broken stuff and moving code around and getting the code in the same tree, to wanting to extend the APIs, invent some new ones, and pull it all together. That requires a great deal of collaboration, across many different communities and different ARM hardware, and that's what we spend a lot of time at Linaro Connect events discussing. We've moved from following to leading in many areas.

The H: You mentioned the kernel tree; how is the challenge of sorting out the kernel tree mess coming along?

David Rusling: The whole thing where Linus pushed early last year and said "look guys get your act together" – we figured that's why Linaro was founded so we got our act together. We created and support what is basically a sub-architecture tree for ARM Linux for all maintainers called ARM SoC. We put folks in place, guys like Nico Pitre, Arnd Bergmann, Paul McKenney and Deepak Saxena who runs the kernel working group. All of the members, maintainers and handlers got together and started using that tree. By the end of the year, certainly by November, Linus was saying, yes, we could see an improvement, we can see order rising out of the chaos.

Actually Jon Corbet of LWN made a nice prediction in his end/start of year wrap-up and predictions where he is confidently predicting that we'll have sorted out the ARM kernel mess completely by the end of this year. There's a lot of code shovelling gone on but the benefits are there. People are finding the right places to put code and code is going upstream nicely, and proof of the pudding is we've not had any complaints from Linus for a while, so we're obviously not irritating him too much.

Linus knows exactly what buttons to press when he wants something to happen and actually it was the right thing. In the history of Linux on ARM it is growing up really, from the community and from the ARM maintainers becoming overall more influential and interesting to the rest of the Linux community.

The H: Mark Shuttleworth recently said Canonical's teams lead the ARM tree unification effort. Is this accurate?

George Grey: Linaro itself is leading members' efforts on ARM unification and optimisation for Linux. Linaro's development team of over 120 engineers currently comprises around half member-assignee engineers and half Linaro engineers.

The Linaro engineers are a mix of direct Linaro employees and contractors, including contractors from Canonical and other open source partners. In the early days of Linaro, Canonical's engineers helped Linaro get off to a very fast start, and since then there has remained a strong partnership between Linaro and Canonical. Canonical themselves use Linaro's work for enablement of Ubuntu on members' ARM SoCs.

Today the whole Linaro team works on consolidating and optimising Linux for ARM SoCs, and works on improving the ARM code in kernel.org. We are distribution agnostic, and we provide builds of key distributions to our members' requirements including Linaro builds of both Android and Ubuntu.

But, it's worth noting that Linaro doesn't yet cover all ARM vendors. When asked which companies he'd like on board, Qualcomm and NVIDIA were top of Grey's list; NVIDIA is interesting as they are working with quad cores and a supporting core, a similar yet different approach to the problems of big.LITTLE ARM developments. But he does note that, because of the commitment in terms of engineering resources that Linaro looks for, joining is a relatively expensive proposition and companies looking to join will want a solid demonstration of value.

Although Linaro hasn't proven its value to all the ARM SoC makers, Grey does think that they've proven themselves good open source citizens who care about the code, the kernel and the future. With silicon getting more and more complex, the old ARM SoC Linux approach wasn't working and it takes collaboration, or "co-opetition", like Linaro, not one single company, to solve the problems. The real measure of success for Linaro is if it ensures that companies can start using ARM faster and in a better supported position.