A knotty, tangled mess of IT systems

The House Oversight and Government Reform Committee garnered a lot of national media attention from its “discovery” of computer systems that are four or five decades old and still running critical functions in the Defense Department, National Oceanic and Atmospheric Administration and several other agencies.

Rep. Jason Chaffetz (R-Utah), chairman of the committee, held up an 8½ inch floppy disk, similar to the one DoD uses to with its 53-year-old Strategic Automated Command and Control System, as exhibit A of the problem nearly every agency is facing.

A 81/2 inch floppy disk like this one is being used by a DoD computer system that is more than 50 years old.
A 81/2 inch floppy disk like this one is being used by a DoD computer system that is more than 50 years old.

Chaffetz also said according to best estimates, the government as a whole uses 930 million lines of code using more than 70 legacy programming languages, including more than 155 million lines of COBOL and more than 135 million lines of Fortran.

While the hearing produced good theater and a lot of shocked comments from “Joe and Jane citizen,” the discussion actually did two other more important things. First, Chaffetz and the other committee members furthered the case for the $3.1 billion IT Modernization Fund the White House is pushing.

But maybe more importantly, the hearing showed just how much of a tangled, knotty mess agency systems are in and why the advances in using agile or iterative development is the only way to untie these stubborn knots.

If you haven’t read the Government Accountability Office’s testimony about the status of legacy IT systems in government, it’s well worthwhile.

It’s a fascinating look into the history of federal IT.

But it may not be an accurate look at the state of federal IT.

“The idea of jumping on COBOL and saying that’s our issue isn’t necessarily true. I guarantee there are things that are functional and we would be a fool to redo them,” said Bob Woods, a principal with Topside Consulting and a retired federal IT executive who helped set up many of these older systems in the 1970s and 1980s. “Just take the tax code system and how much it’s changed in the last 53 years. It probably has very little of the original logic. But that’s because the IRS gets changes from Congress and it has to take hundreds of millions of tax records and compile it while making the change. That IRS code is nowhere near what is was when they started it. It has to evolve over time.”

The IRS runs two systems in use that are 56-years-old — the individual master file and the business master file. Both hold continuously updated records of citizens and businesses around tax payment and history.

Woods said all of these systems that are long in the tooth are a mix of old and new technologies, which makes it hard just to modernize them the way current systems can be updated with new and faster servers, or new software code.

The 53-year-old system DoD uses to run the nuclear forces is a good example of what Woods is talking about. While maybe some of the logic or the basics of the system have been around for 53 years, the Strategic Automated Command and Control System runs on an IBM Series 1 computer — a 16-bit minicomputer, introduced in 1976. IBM stopped marketing the Series 1 in 1988 so presumably the system is pushing 30-years-old. So, yes, old, but not five decades old.

Advertisement

Terry Halvorsen, the DoD chief information officer, pushed back against questioning about the system from Rep. Ted Lieu (D-Calif.), saying the command and control system provides five 9s of reliability and is a closed system, meaning it’s secure.

Halvorsen said he expects DoD will modernize the system in the next three years. But while he didn’t dispute the system needs updating, the fact is not too many other modern systems can give that level of reliability and security as this legacy system is providing to DoD.

Bob Suda, president and CEO of Suda and Associates and a retired federal IT executive, said many times these systems are not fully modernized is because agencies don’t have the documentation.

“You need to make sure you’re not eliminating functionality that has been around for 30-40 years,” he said. “You have to know the history of what’s in the system. The bigger challenge is not to miss something that is critical to the functionality.”

Suda also downplayed many of the concerns about how much COBOL, Fortran and Assembly language are currently in use across the government.

“There are a lot of transactional systems in the commercial sector that run COBOL, and run it well,” he said. “You have to look at the unintended consequences of modernizing one of these legacy systems. You have to take it in small baby steps and do the analysis from the hardware, software, security and resource perspectives.”

Woods echoed Suda’s comments about how to approach the modernization of these long-time systems.

First off, no matter what Congress believes unplugging or launching a full replacement in the cloud, for example, isn’t rationale.

“When you get down into it, you have to layer it and do a triage and take the ones that are easiest to convert and replace, and that have the highest return on investment for making change and do those in priority order,” Woods said. “Eventually, you may break up into modules, which can be maintained individually.”

Which takes us back to our agile or iterative discussion, and why agencies need to take advantage of the advancements in understanding of this concept. In fact, this is what Rob Klopp, the Social Security Administration CIO, is doing with his legacy systems.

Klopp said in March that his plan to upgrade systems that run more than 60 million lines of COBOL is to use the cloud to expand the development options for SSA’s software engineers.

Klopp is writing a blog on the CIO.gov website highlighting his effort to modernize. The most recent installment looks at the difference between today and yesterday.

“To state the bottom line up front — the modernization issue is not about COBOL. It is about building software that takes full advantage of the economics of a modern hardware infrastructure. It is about architecture, not about a programming language,” Klopp wrote. “Modern software is architected to run over a stack that can be distributed across a cluster of servers. It is designed so that each layer of the stack and so that every business application can be distributed and deployed and scaled across a cluster of servers.”

Klopp isn’t the only CIO taking on legacy systems in this approach. But it will take time and money.

Klopp, for instance, is hoping to use money saved from SSA’s data center consolidation effort to put toward these modernization efforts, but if the lawyers or lawmakers say they can’t, the effort is slowed.

For this very reason, Congress should get behind the administration’s IT Modernization Fund — at the least on a pilot basis for a year.