Integrated intelligence database

NPR tells us about a new intelligence agency that was founded after the creation of the office of the Director of National Intelligence. The new agency is called the National Counter-Terrorism Center (NCTC), and its job is to coordinate counter-terrorism operations. To do that, it needs to combine pertinent intelligence information from the many other agencies and sub-agencies that have it — CIA, NSA, NRO, FBI, DIA, and so on.

The thing is, though, that combining this information isn't easy, because collecting it isn't easy, even though all these agencies are part of the same government:

Here's the problem: to fulfill NCTC's key mission of sharing intelligence across all US spy agencies Brock and his staff need access to 28 different computer networks. [... But] none of the 28 networks talk to each other. So if Brock wants to search for the latest threat information on, say, Osama bin Laden, he has to search each network separately.

We're accustomed, of course, to communication difficulties within the government. And we're accustomed to difficulties in aggregating information across computer networks in any case. But this has been going on for a while. NPR says this about it:

Here's some perspective on how stubborn a problem this is: when we visited NCTC's predecessor, the Terrorist Threat Integration Center, more than two years ago, then-director John Brennan was grappling with the exact same headache. Brennan told us he hoped to have an integrated database within a few months. That was in April 2004. Today, Kevin Brock says he believes a solution is still a year or two away.

More than two years ago, indeed. In fact, there were efforts to provide smooth integration of this information more than twenty years ago, after the 1983 US embassy bombings in Beirut. So this isn't new, and didn't start with the creation of the DNI or the NCTC.

To be sure, it's not a trivial problem: this is sensitive information that exists on isolated networks, and any connections increase the exposure of the networks to hacking. Further, government classified information is distributed on a need-to-know basis that's restricted by mandatory access controls. It's certainly possible that software with appropriate controls and guarantees isn't available to handle this properly.

Still, after more than twenty years, at least, one would think the US government could see to it that such software was available, and that the networks and databases were connected as necessary to provide the information that agencies such as the NCTC need. It appears that in 2001 we had information that might have given us details and warnings of the attacks then, but the information wasn't connected in any useful way, and didn't appear important as discrete data.

Intelligence will help us more than bombs. "Intelligence" in the "thinking and reasoning" sense, of course, but, here, "intelligence" in the "information" sense. We need to put the funding and work into creating the integrated database that the NCTC needs, or else Mr Brock's prediction of "still a year or two away" will fall into another twenty years.