Rethinking OSINT: Information vs. intelligence

What’s the difference between data and intelligence? After all, our world is saturated with data.

July 13, 2022 3:16 pm

4 min read

This content is sponsored by Recorded Future.

What’s the difference between data and intelligence? After all, our world is saturated with data. Anything you want to know, it’s most likely out there, as long as you’re willing to look hard enough. But what separates public records, a social media feed or a news story from open source intelligence (OSINT)?

The word “intelligence” implies you’re using a process, a tried and true process, said Geoff Brown, vice president of global intelligence platforms at Recorded Future and former chief information security officer of New York City.

“At a very high level, you’re talking about collection, analysis, deployment and the feedback loop,” Brown said. “What I think is really important with that word ‘intelligence’ is that what you’re really driving for, in some sense, is deriving an insight.”

But what about the term “open source?” That doesn’t mean all the information out there, he said.

“You can get information from news sites,” Brown said. “You can get information by going to whatever your favorite internet search tool is. But you don’t really know in today’s digital landscape where that information that you collide with is coming from.”

Understanding the why behind information found in the open

A significant portion of information on the internet is subject to business needs, which means that at some level much of it exists simply to sell ads. But covert influence and overt disinformation also run rampant, Brown said. The intelligence process has to be able to confidently and accurately sort out the motive behind open source information and data to determine the level of trust that can be applied, he explained.

On the other hand, it’s precisely the proliferation of data availability and the need to examine more closely whether it can be trusted that is helping to drive a rise in credibility for OSINT, Brown said. The need to closely examine and ensure the trustworthiness of data has led to an appreciation for OSINT processes and the utility of this type of intelligence, he said.

When it comes to how to use and apply OSINT, solid tradecraft and training are so important to the intelligence community. There has to be a common baseline with respect to the processes applied that allows decision-makers to accurately evaluate the insights gleaned from the data. The same is true of OSINT, Brown said.

The intelligence community has been expanding its tradecraft and training during the past few decades to encompass technology and how to use it to facilitate information and data analysis.

Developing rigorous processes and focusing on mission

But the tools themselves aren’t the important part. Common OSINT frameworks are more critical than common tools, he said.

“When I think about a common platform, I’m not saying everybody has to have this thing,” Brown said. “I’m saying that everybody needs to recognize that if you’re going to make judgments based on the digital exhaust we’re all leaving behind us on the internet each and every day, then you’ve got to have a process around it, a capability to collect, organize and make sense of it at a rigorous enough level.”

That’s why it’s so important to abandon the notion that if you have all the available data, you can know anything. What’s far more important, he said, is ensuring you have the right data.

“As we build our open source intelligence capabilities, what we need to do is, as we collect, make sure that we’re collecting based on those priority intelligence requirements to have a continuous feedback loop” Brown said. “What is the right data for us to have in order to meet our mission requirements?”

The mission should always be the line, he said, between having the right data and having all the data. For example, a traditional cybersecurity program doesn’t need access to all the data in the system; it just needs security telemetry.

Accounting for privacy

Privacy also needs to be taken into account, Brown said. Data privacy is becoming a bigger concern for much of the population, and agencies should respect and reflect the values of their constituencies. They need to consider when and why it’s necessary to collect personal data, he said.

That’s increasingly important as more people and organizations come to the realization that anything with a physical imprint leaves a digital signature. And anything with a digital signature can have machines and algorithms applied against it to recognize patterns and insights. That’s convergence: Anything revolving around basic human behavior can be mined for insights to answer intelligence questions.

“Open source intelligence, and the capabilities that can be applied to it, allow for the rapid generation of insights across all data sets, which then corresponds to all threat types,” Brown said.

Increasingly, organizations can use those capabilities to rapidly apply sophisticated and powerful analytics across big sets of data, buttressed by analytic rigor and processes, to increase confidence in the OSINT they produce.

“Then, you likely can start to create unbelievably predictive insights and solve big problems,” Brown said. “That’s a big deal.”