Stay up-to-date with the latest federal news. Sign up for our breaking news email alerts.

The most heralded federal agency is in serious risk of a major cyber attack and no one seems to care.

Not NASA executives. Not the contractor hired to protect its end-user devices. And especially not the everyday employees who send rockets into space.

Internal documents obtained by Federal News Radio indicate NASA has anywhere from hundreds of thousands to millions of out-of-date patches at every center across the country.

Security Scorecard, a cybersecurity company, found as many as 10,000 pings coming directly from NASA’s network to known malware hosts, some lasting weeks, if not months.

Multiple sources say Hewlett Packard Enterprise (HPE), the contractor hired to protect NASA’s desktops and end-user devices under a $2.5 billion contract called the Agency Consolidated End-user Services (ACES), is uncooperative at best and negligent at worst, and a major reason the agency’s data and systems are at risk.

One NASA source said the breach suffered by the Office of Personnel Management and the Office of Management and Budget’s cyber sprint didn’t serve as wake-up calls for the space agency, and cybersecurity remains a serious issue.

“HPE admits that it doesn’t have the resources to keep up with the patching and that this has been going on since the contract was awarded several years ago.” — Senior NASA IT official

“At the heart of it all are three or four major problems. Two of those problems are most serious. First, this is an IT operations issue. Security of NASA’s data and systems are seriously weakened and prone to compromise because IT operations appears to be failing at keeping up with a basic operational function and that function is patching of applications and operating systems,” said a senior NASA IT engineer familiar with IT operations. “Whoever is responsible for maintaining the daily operational health of systems and applications should be held accountable for allowing the IT environment to get to this state. This is a very clear example of malfeasance. Second, from what I understand in conversation with the individuals at NASA headquarters who manage the ACES contract, it is a requirement that HPE patch certain applications. The conversation taking place is that HPE admits that it doesn’t have the resources to keep up with the patching and that this has been going on since the contract was awarded several years ago. No one within NASA’s leadership managing the ACES contract or at HPE seems to be very concerned. This is very disturbing and disconcerting.”

It’s not just HPE, however. Two other sources said in separate interviews the NASA culture focuses on mission first and foremost, and cybersecurity a distant second.

“I would say NASA is worse than average when it comes to cybersecurity across the government,” said a former NASA official. “The number one problem is poor IT governance. There is no centralized authority that is empowered to do anything about security issues at the agency, including the chief information officer. The centers and other organizations are doing their own things, and cybersecurity is not viewed as a mission problem. It’s viewed as a CIO problem. But it really is a mission problem and until the agency understands that, their cyber problems will remain because it’s not something the CIO can solve. It needs strong leadership from the administrator.”

The former official, who requested anonymity, added leadership doesn’t prioritize cybersecurity from a budget standpoint and their decisions don’t reflect the importance or urgency of cybersecurity to the staff.

The former official pointed out that some of the patching challenges are related to NASA putting a freeze on all IT systems and software prior to a mission launch. The former official said because management doesn’t want anything to go wrong with a launch of a space craft, systems and networks are kept constant to ensure mission success.

But once NASA lifts the freeze, the former official said the agency and especially HPE haven’t kept up with protecting the systems and devices.

“If you can’t show them a direct mission impact, empirical evidence, you get discounted, especially around cybersecurity,” the NASA IT engineer said. “They are just doing compliance activities. When you talk about security programs at NASA, we seem to wait for DHS to tell us what to do. NASA lacks a focus on cyber and there is no real strategy for dealing with internal weaknesses.”

A government official with knowledge of NASA echoed similar concerns. The official said over the years, IT executives have brought up the poor patching by HPE with leaders and received little to no response.

“Really, security should be a mission element, but that is the piece they haven’t grasped yet.” — Government Official

“The whole notion of balancing mission versus security is something every agency is challenged with, but NASA’s leadership is not as in tune in striking the right balance of mission versus security,” the source said. “Really, security should be a mission element, but that is the piece they haven’t grasped yet.”

A NASA spokeswoman refuted many of the claims the space agency isn’t focused on securing its networks and data.

“NASA takes cybersecurity very seriously and is committed to devoting the necessary resources to ensure the safety and security of the agency’s information and information technology systems,” the spokeswoman said in email answers to questions from Federal News Radio. “The agency combines internal resources, such as its dedicated cybersecurity team, with the resources made available through its active participation in federal cybersecurity programs and initiatives to ensure its entire infrastructure is constantly protected and operational.”

The spokeswoman said NASA validates the number of devices monthly and uses enterprise-level continuous monitoring tools that gather software inventory and patch status through the ACES contract.

“Since the 2015 Cybersecurity Sprint, NASA has made substantial progress in tracking and managing vulnerabilities,” the spokeswoman said. “This agency effort is reflected in [Feb. 15’s] Department of Homeland Security Cyber Hygiene report on NASA, which shows zero critical vulnerabilities older than 30 days since September 2015.”

But sources pointed out that DHS is looking at external-facing systems only, and it’s the soft underbelly of any organization that puts it at more risk.

Internal documents from last summer say in the most severe instances the missing patches could open the door for a hacker to take over privileged administrative rights, and could let a hacker execute malware through a commonly used software title, meaning they are behind the firewall and other external cyber defenses with little effort.

Follow-up emails to NASA asking for clarification whether the DHS hygiene report focused on external or internal systems were not answered.

The data breach OPM suffered last year exemplifies this type of problem that many agencies face. Hackers obtained the credentials of a contractor through a phishing attack to breach OPM’s external network defenses. Once they were inside, hopping around from system to system and collecting data wasn’t difficult.

Vulnerable today as OPM was last summer

Sources and experts say NASA is as vulnerable today as OPM was before it was attacked.

A recent scan of the Internet found 10,000 pings emanating from NASA’s network back to known malware hosts.

Sam Kassoumeh, the chief operating officer and co-founder of Security Scorecard, which regularly conducts scans of the public Internet, said the malware activity coming from NASA is astonishing.

“Every company in the world probably has a malware infection. Everybody accidentally clicks a link or opens an attachment they shouldn’t have. So it’s not necessarily surprising to see that a company has a malware infection, it’s very common,” Kassoumeh said. “What you generally want to see in an organization that has a very healthy security posture, you want to see an infection, so you see a spike, and then maybe in the same day, the malware is remediated and that spike goes away. But when you see malware spike and then persist over days or weeks or months, that tells us there are little to no internal security controls inside the organization to detect and respond to incidents.”

Security Scorecard collects threat intelligence from across the Internet. It doesn’t focus on any one organization, but vacuums up all data. He said then when the analysis shows a lot of activity, Security Scorecard attributes where the beaconing is coming from, in this case NASA.

“We’ve seen just loads of various malware families, these different types of malware, actively beaconing from inside of NASA’s technology infrastructure, and some of the malware duration goes on for not just a day, but sometimes even for weeks or for months. We see the same malware family signals,” Kassoumeh said. “We see not just the beacon, but the source IP address of the machine that’s beaconing.”

When malware attacks persist for weeks or months, it means hackers already are in the network, Kassoumeh said.

“We are seeing over 40 unique malware families over the past year. That doesn’t mean that there are 40 unique cases of malware currently infecting NASA. I’m claiming that over the past year from February 2015 to this year, we have seen 40 different families of malware emanating signals from [NASA’s] digital footprint,” Kassoumeh said. “Some of these malware families are some of the nastiest known viruses in existence. They are not the run of the mill, click this link and a bunch of popups or spam flood your computer or your personal email accounts starts sending spam email to all of your contacts. That may be the minority of the behavior, but some of the malware families that we’ve seen over the course of the year are some of the nastiest known really in existence.”

The NASA spokeswoman said the Security Scorecard analysis is incorrect.

“The agency’s continuous monitoring tools and scans … and various independent third-party audits … do not support this claim of a broad malware infection in NASA’s IT infrastructure.” — NASA spokeswoman

“NASA regularly receives information, observations and findings from various channels,” she said. “The agency’s continuous monitoring tools and scans, a set of monitoring and scans performed by Department of Homeland Security, and various independent third-party audits of NASA’s computing environment do not support this claim of a broad malware infection in NASA’s IT infrastructure.”

Yet the spokeswoman couldn’t answer how many patches NASA is in need of because “the number of needed patches is not static due to the constant stream of vendor-released patches.”

But internal NASA documents from last summer — after the White House’s cyber sprint ended — tell a different story about the health of the agency’s internal networks. Headquarters and every center are struggling with securing systems with between 10,000 and 138,000 missing critical patches.

Sources asked Federal News Radio not to share specifics about each center’s missing patches for fear of making those organizations more of a target by hackers.

NASA paid $35M to Hewlett-Packard Enterprise Services

The union representing NASA Ames employees raised these concerns to executives over the summer after the OPM data breach became public.

In an undated letter sent by union officials at AMES, and obtained by Federal News Radio, employees brought their concerns about patches and the ACES contract up to leadership. The contents and timing of the letter match up with the internal documents Federal News Radio got a hold of.

“In a 2014 settlement, the ACES vendor [HPE] was paid some $35 million above and beyond the terms, conditions and fees of the original contract, but yet NASA continues to pay this vendor millions of dollars each year, when they have not fulfilled their obligation and continue to place NASA sensitive data and information systems at risk by not applying close to 700,000 critical patches to NASA’s systems and applications,” the letter stated. “In addition to this, NASA does not have a reasonable inventory of its assets. Under the settlement agreement, the ACES vendor was to provide NASA with a system to manage IT assets. To date, and nearly a year after the settlement, ACES has no system in place to manage IT assets. Without a robust asset inventory, there is no way that a protection solution can be implemented; one can’t protect what they don’t know needs protection. Similarly, NASA does not have full visibility into its antivirus posture. The ACES vendor has yet to provide full visibility into virus activity within the NASA infrastructure, yet NASA has paid millions of dollars for this information but to date, has limited or no visibility, leaving bargaining unit members and NASA sensitive information at risk.”

NASA confirmed the $35 million payout to HPE and the fact it picked up the first three-year option of the contract, extending ACES to October 2018.

“Hewlett Packard Enterprise takes security very seriously and remains committed to our close partnership with NASA,” said a Hewlett Packard Enterprise (HPE) spokesperson in an email to Federal News Radio.

Sources say HPE threatened to sue NASA in 2014 over a disagreement in the contract terms over how many email seats the contractor was supposed to support. That led to a less than harmonious relationship between HPE and NASA, which many say is contributing to the patching problems.

“HPE is going to be very reluctant to do anything more than they have to and the only way is by withholding money, but NASA stopped doing that and gave away strength they had,” said the former NASA official. “When HPE threatened the lawsuit, NASA should’ve ended the contract. NASA had an opportunity to get out of the contract and they didn’t.”

The government official with knowledge of NASA said HPE is so far behind in patching, they don’t know where to start first, which compounds the resource issue some say HPE is struggling with.

And sources say center CIOs and chief information security officers have no say or control over their own networks so they can’t force HPE to patch, and they can’t fix their networks with NASA employees.

“What largely concerns many people is HPE is still playing a role because I don’t think they have the understanding that it’s that important to deal with the cyber issues,” said the government official with knowledge of NASA. “If you step back and look at the company’s actions versus words, the inaction is what I guess frustrates so many people today. We are having a contractual discussion about something that should be inherent in providing IT services across the board. It’s a money discussion, instead of being a mission discussion. That’s way out of balance.”

NASA says HPE met 98 percent of security metrics

The senior NASA IT engineer said the number of out-of-date or missing patches may reach a few million as the agency and HPE continue to struggle to know exactly how many systems are in need of updates.

“We’ve seen an escalation over the last few years as more and more agency executives are talking about cybersecurity. A lot of questions are coming up, not just about patching, but security in general and the answers aren’t good,” said the senior IT engineer. “Patching is the biggest thing because it hasn’t gotten better and you have HPE saying we know it’s in the contract, but we don’t have the resources, and they are getting other priorities from headquarters. That is the real issue that many have, HPE isn’t fulfilling requirements of contract and there is no pressure on them to do so.”
The NASA spokeswoman said HPE’s performance has been solid since the start of ACES.

“HPES has met 98 percent of the seven monthly ACES security-related metrics,” she said. “NASA can retain a percentage of cost incurred for contract performance measures not met.”

Lee Stone, the co-chairman of NASA Labor-Management Forum and Western Federal Area vice president of the International Federation of Professional and Technical Engineers (IFPTE), said in an email to Federal News Radio that he’s seen progress in securing systems and data over the last few months.

“Since the arrival of a new CIO [Renee Wynn] last summer, this backlog has been significantly reduced and NASA is continuing to work this issue. As far as the unresolved critical vulnerabilities, the new CIO has been able reduce resolution time down below that required by DHS,” Stone said. “Labor is committed to continuing to urge NASA leadership to enhance NASA’s IT security posture and to ask Congress to increase funding for this important priority. We are pleased with the recent progress, but there is more that can and should be done at NASA and elsewhere. Additionally, we remain concerned that the centralized and outsourced structure of NASA’s IT enterprise continues to pose a challenge, as it is not as nimble or adaptive as it should be in this era of evolving threats.”

Stone said the union continues to push NASA Ames to bring key IT oversight and security functions back in-house and tighten up control over contractors in the future.

Sources said NASA needs a wake-up call, and China’s suspected hack of the Landsat-7 satellite in 2008 wasn’t enough. NASA confirmed in 2011 that it experienced two suspicious events, but said no data was changed or captured, and no commands were successfully sent to the satellite.

“I would’ve thought that would have made them go ‘holy crap,’ and be more aggressive about doing stuff around cyber,” the NASA IT engineer said.

Sen. Thune asked cyber questions

The NASA spokeswoman said Wynn, the agency’s new CIO, is restructuring and streamlining existing IT boards to ensure all IT investments meet federal cyber policies.

“Working with NASA’s chief financial officer, the OCIO will conduct a formal annual capital investment review as part of the program planning budgeting and evaluation process that will include all IT investments,” the spokeswoman said. “The OCIO will work jointly with the agency’s assistant administrator for procurement to formalize guidance on strategic IT sourcing and strengthen and expand the NASA CIO’s role in monitoring agency IT program performance. Finally, OCIO will conduct functional reviews of all NASA centers on a three-year rotating basis.”
NASA cybersecurity challenges haven’t gone unnoticed on Capitol Hill.

Sen. John Thune (R-S.D.), chairman of the Commerce, Science and Transportation Committee, wrote to NASA on Feb. 10 asking seven questions about the state of the agency’s cybersecurity efforts and an alleged intrusion into its networks.

NASA responded on Feb. 18, saying it found no credible evidence of a compromise of its systems or exfiltration of sensitive data. The agency’s IT staff also met with committee staff at least twice over the last month to further discuss cybersecurity challenges.

But until NASA re-takes control of the ACES contract and convinces the mission areas that cybersecurity is as important as launching space ships and probes, sources say the agency’s networks and data will continue to be at risk and likely already are under the control of hackers, nation states and/or others looking for intellectual property or looking to do harm.

“When NASA gets close to understanding it and has cyber professionals in systems engineering then they will get it,” said the former NASA official. “They need cybersecurity awareness deep inside the mission.”

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.