Back in 2005, the head of the National Security Agency broke out his red marker and circled a section of a white paper written by cybersecurity experts and gave them a two-month deadline to bring this idea to bear.
The concept the experts detailed to Gen. Keith Alexander would let NSA use technology to identify adversary tradecraft in flight, outside the wire so to speak, and treat it as a network problem.
Alexander thought the technology would be a game changer — maybe not a silver bullet — but something that would give the Defense Department a head start against ever-increasing threat before they made their way into the network.
Insight by GDIT: Learn how the customer must drive the entire digital modernization process by downloading this exclusive e-book.
Now, 16 years later, experts say this type of technology would’ve gone far to prevent, or at least limit, the damage for most of the major cyber breaches federal agencies suffered since 2005.
“It was a big deal because we brought intelligence and defensive folks under one roof. The results were profound. We created a rich contextual threat intelligence about what adversaries were doing to DoD and we used it to warn incident responders and others,” said Steve Ryan, a former deputy director of NSA’s Threat Operations Center who coauthored the aforementioned white paper.
“We set out to do something big and bold. We created classified capabilities that were largely tuned to interfere with cyber outside the network,” said Ryan, who is now the CEO and co-founder of Trinity Cyber. “We were leveraging our knowledge of the adversaries.”
Ryan said his team fielded a pilot by the end of calendar year 2005 and presented it to Alexander. By December 2008, the capability was protecting all of DoD.
Generally speaking, the capability is focused on deep packet inspection and the ability to reroute traffic that is potentially a threat to the network. The tool would stop remote code execution and find malicious software to make it more difficult for hackers to get inside an organization’s network. Some experts said capabilities like this, especially now 13 years later, would have limited the impact of the SolarWinds attack.
As DoD rolled out the capability, NSA started talking to the Department of Homeland Security about adding the technology to the EINSTEIN program.
John Felker, a former Coast Guard and DHS cyber official, said NSA was set to implement a version of these capabilities in EINSTEIN.
“They got all the way to be ready to pull the trigger and the deputy secretary at the time decided to stop it, and decided that DHS could do something similar on their own,” said Felker, who now is president of Morse Alpha Associates. “That was unfortunate. There was an idea that DHS could do it themselves and folks were selling a program that would have a positive impact, but they oversold it. That may be a reason so many people don’t understand what EINSTEIN is or was supposed to do.”
That lack of understanding of what the EINSTEIN tools are and are not came to a head over the last few months as lawmakers and misinformed “experts” questioned why the more than $1 billion investment over the last 16 years didn’t stop the SolarWinds attack.
The fact is the EINSTEIN program was never designed to stop SolarWinds or the Microsoft Exchange hack or even the Office of Personnel Management hack.
Current and former DHS and White House officials said the investment in the EINSTEIN tools made agencies safer and met the initial goals laid out in 2004: To stop known attack vectors or signatures through intrusion detection and prevention tools.
“There seems to be the misconception that EINSTEIN should block every sophisticated cyber threat. Unfortunately, that is a false narrative,” Matt Hartman, the deputy executive assistant director for cyber at the Cybersecurity and Infrastructure Security Agency, said in an interview with Federal News Network. He called EINSTEIN just one component of a layered defense.
“It’s a key piece and its success relies on information provided by commercial and intelligence community partners,” he said. “But it’s not going to pick up a novel supply chain attack that was designed for many months and executed in a matter of hours. For that reason, it must be complemented with other tools like those through the continuous diagnostics and mitigation (CDM) program and through cybersecurity shared services.”
Understanding those layered defense concepts became more critical with the new cybersecurity executive order that President Joe Biden signed May 13. With dozens of new and expanded initiatives, lawmakers and agency leaders should heed the lessons learned from EINSTEIN and other governmentwide cyber programs: The need for flexible, iterative tools and capabilities. The White House needs to break down the DoD, intelligence community and civilian agency silos by not adhering to the old, “not invented here” mindset.
Karen Evans, the former administrator of e-government and IT at the Office of Management and Budget and former DHS chief information officer, said the combination of EINSTEIN, CDM, Automated Indicator Sharing (AIS) and other capabilities were laid out in the Comprehensive National Cybersecurity Initiative (CNCI) in 2008 during the waning days of the Bush administration.
“Our goal was to connect the security operations center initiatives across government,” Evans said. “DoD, NSA, DHS and others were supposed to bring them altogether.”
Evans is referencing paragraph 39 of the CNCI that called for a whole-of-government incident response capability.
Of course that never happened, so tools like EINSTEIN were left to fend for themselves.
A former DHS cybersecurity official, who requested anonymity because they didn’t get permission to speak to the press from their current private sector job, said a common choke point for EINSTEIN was the inability of DHS to get consent from agencies to monitor their internet traffic.
“Even once consent was reached, then it took time to schedule the cut overs onto the services. EINSTEIN versions 1 and 2 were easy. E3A was complex because it was inline and blocking. However, once those legal, privacy and technical hurdles were crossed, the later onboarding of agencies could move rapidly,” the former official said. “The big gap during my time was the lack of internal monitoring data – security alerts from applications, servers, desktops and other endpoints. EINSTEIN can only see what data it is receiving. Cyber is about the whole picture. I don’t think the question is about EINSTEIN as much as it is about whether the National Cybersecurity and Communications Integration Center (NCCIC) and U.S. Computer Emergency Readiness Team (CERT) are authorized and able to receive a much more comprehensive set of security event data.”
Felker, the former DHS and Coast Guard cyber official, said creating a signature for a malicious code, testing that signature and putting it into EINSTEIN is not as easy as some would like to think.
“We’ve had this before where signatures ended up blocking mission critical activities even with testing. That made signatures challenging,” he said. “Add to that the fact that agency networks are not homogenous, it becomes a balancing act and a risk management act.”
Once again, another executive order will try to break down those systemic barriers that prevented the NSA adding its capability to EINSTEIN. Congress and other administrations have attempted to do this through policy and laws, but few have succeeded in making real progress.
CISA’s Hartman said it’s clear no entity can know everything. This was never more true than with SolarWinds, an instance in which FireEye alerted the intelligence community, CISA and others about the attack.
“EINSTEIN is only as good as information it is receiving for both the intelligence community as well as from commercial partners to enable partners to build in load signatures into the system that can detect or prevent similar attack techniques,” he said. “There is a capability that is a part of EINSTEIN 3A, that is known as logical response aperture. This capability was developed as an initial attempt to utilize artificial intelligence and machine learning techniques with network-based data in an attempt to identify suspicious malicious data without prior intelligence that could be deployed in a signature-based detection and prevention capability. This is now deployed at two internet service provider locations within the National Cybersecurity Protection System EINSTEIN 3 architecture. It’s been a valuable analytics platform, and, quite frankly, it is limited in its ability to detect verifiable malicious, network-based activity.”
Hartman said CISA’s plan is to focus this capability as a component of its analytical environment, providing one more toolset to review and determine potential and real threats.
Experts says what Hartman is describing is more about how CISA is changing than any one tool or capability.
Evans said CISA is a service provider with authorities from Congress and OMB to manage the results of the technology versus managing the tools themselves.
“This is a culture change. What Congress is asking CISA is ‘what do you need?’ and holding the DHS secretary accountable for delivering results,” she said. “The question that Congress and OMB have to answer is how far they want CISA to go to enforce and manage federal networks. That is the question.”
Tom Bossert, the former Homeland Security advisor to President Donald Trump and now president of Trinity Cyber, said there are new capabilities, similar to the one NSA implemented so many years ago, that could provide greater benefit to agencies.
“[Expanding NSA’s tools] wasn’t necessarily a missed opportunity by government or the private sector, but a reflection of where we stand today. We have mismatched capabilities and defenders have not made the necessary changes as offenders are far more nimble. There are major developments in how we access networks, the diversification of edge and cloud services and a significant amount of innovative technology that could be applied in a different way to prevent cyber breaches,” Bossert said. “We must find happy medium ground within our collective cyber industry. There is a resistance to innovation and there is a strong risk aversion to change because we are worried about unintended consequences.”
Bossert added the latest cyber attacks have shown there is better interagency coordination and clarity of purpose, and that must continue as the threats evolve.
CISA’s Hartman said he believes that EINSTEIN has met its original intent and much more. The capability routinely finds instances of anomalous activity that are confirmed and stopped.
“We are constantly modernizing our portfolio of capabilities,” he said. “We are thinking about EINSTEIN, CDM, the cyber quality service management office (QSMO) and how to evolve all of those capabilities. The evolution is underway, and it will accelerate in the coming months as [a] result of new authorities under [the] 2021 defense authorization act and the funding from the American Rescue Plan Act.”