The Pentagon’s Mad Science Is Going Open SourceBy Klint Finley 02.05.14 6:30 AM
Illustration: Ross Patton/WIREDNational security is often synonymous with secrecy. But when it comes to software development, the U.S. defense and intelligence establishment can be surprisingly open.
This week, the Defense Advanced Research Projects Agency — or DARPA, the research arm of the U.S. Defense department — published a list of all the open source computer science projects it has funded, including links to source code and academic papers that detail the code’s underlying concepts.
Anyone is free to not only peruse the source code and add to it, but actually use it to build their own software — and that includes foreign governments. The belief is that because anyone can contribute to these projects, the quality of the code will only improve, making the software more useful to everyone. It’s an approach that has paid off in spades among web companies from Google and Facebook to Twitter and Square, and the government has now realized that it too can benefit from the open source ethos.
The Softer Side of DARPA
DARPA is known for some pretty whacked out projects. Mind controlled exoskeletons. Space colonization. Turning pets into intelligence assets. That sort of thing. But it does have a more sober side. The agency funded the creation of the network that eventually became the internet, for example. And, more recently, it funded work on Mesos, the open source platform used by Twitter to scale applications across thousands of servers. It’s more of the latter that shows up on DARPA’s new site.
The site is focused on computer science research, so projects that fall outside of that discipline — such as the OpenBCI brain scanner and the open source amphibious tank — won’t be found on the list. But there’s still quite a few important projects, including Mesos, the in-memory data processing system Apache Spark, and the Julia programming language for mathematicians and scientists.
Most of these DARPA-backed projects are on GitHub, the popular code hosting and collaboration service that has come to symbolize the type of non-hierarchical collaboration celebrated by open source enthusiasts and tech culture in general. The site makes it easy for anyone to examine source code, suggest changes, and discuss decisions. Mirroring the way it treats software, the company itself operates with no job titles, no middle management, and only a thin layer of top-level management, preferring instead flat or “holacratic” structure.
When the Military Invented Open Source
That sort of non-hierarchical thinking may seem at odds with military culture, but in reality, many of these ideas were pioneered by military researchers. Today, we often trace the origins of open source software to work done by industrial research labs like Bell Labs and Xerox PARC. But in his book From Counterculture to Cyberculture, Fred Turner argues that open source’s roots stretch back even further to the World War II era defense research laboratories that created technologies such as radar, the atomic bomb, submarines, aircraft, and, yes, digital computers. “The laboratories within which the research and development took place witnessed a flourishing of nonhierarchical, interdisciplinary collaboration,” Turner writes.
He points to the MIT Radiation Laboratory — which was formed by the National Defense Research Committee, a predecessor of sorts to DARPA — as a model example. “It brought together scientists and mathematicians from MIT and elsewhere, engineers and designers from industry, and many different military and government planners,” Turner says. “Formerly specialized scientists were urged to become generalists in their research, able not only to theorize but also to design and build new technologies.”
Today, we’re more familiar with the NSA’s cloak and dagger approach to research, but the collaborative approach of the WWII era military-industrial-academic complex has never really gone away. The Army recently partnered with Local Motors to crowdsource new military vehicle designs. The CIA created In-Q-Tel, a venture capital firm that funds tech startups, including open source big data companies like Cloudant and MongoDB. Even the NSA is part of the action, open sourcing its big data storage system Accumulo.
In other words, the defense industry sees what Facebook and Twitter and so many other web companies see: that innovation often comes from openness.
http://www.wired.com/wiredenterprise/2014/02/darpa-open-source/?mbid=synd_yfinance
6 new categories and 72 new items added to the shop!
