More than a dozen years after the Department of the Navy awarded a contract to build what it has touted as the military’s first true large-scale enterprise network, the Navy is discovering hundreds of networks and tens of thousands of applications that have been operating for years under the radar, outside the scope of the Navy-Marine Corps Intranet and beyond the oversight of the maritime services’ headquarters.

NMCI, which was first conceived as a way to eliminate information technology stovepipes and reduce the number of disparate, purpose-built applications in the naval services, hosts more than 700,000 users across the Navy and Marine Corps. The DoN is expected to award NMCI’s successor, the Next Generation Enterprise Network (NGEN), later this year.

But ever since the Navy decided it was time to cut its IT expenses by about 25 percent two years ago, its IT leadership has been making interesting discoveries about what the service is spending on technology as it scours the system to tally its costs and look for efficiencies. In fiscal 2011, the Navy estimated its overall IT budget was around $7 billion, but as the Navy chief information officer’s staff began poking into nooks and crannies and developing better “all-in” estimates, it figured last year the number was more like $9 billion.

This year, the estimate is $11 billion, said Janice Haith, the Department of the Navy’s deputy CIO who oversees IT for the Navy side of the department (as distinct from the Marine Corps).

“That’s not because the budget is getting bigger, it’s because we’re making people report, and we’re finding out where money is being spent. We had no idea before,” she told an AFCEA Northern Virginia luncheon in Vienna, Va., Friday. “Both [former Chief of Naval Operations] Adm. Gary Roughhead and [Chief of Naval Operations] Adm. Jonathan Greenert have directed us to get a handle on this and control it.”

Legacy networks getting shut down

In the effort to centralize control, the Navy has come across vast swaths of IT infrastructure the CIO’s office never knew about — servers, data centers, applications and software licenses. Haith said the Navy, until recently, had been operating under the mistaken presumption that it already had consolidated most of its IT spend under NMCI.

“Turns out we had 302 legacy and excepted networks still sitting out there. Nobody looked at them. They decided they didn’t want to be bothered with them, but now the CNO has said, ‘Get rid of ’em. You’ve got until March 2014. They’ve gotta go.’ We’ve started doing that, and boy, has this been pain,” she said. “I have never seen so many people tell me, ‘I can’t move to the enterprise environment because I’m special. My data doesn’t belong in the mass environment, and so you have to keep it.'”

Those arguments come before a new IT governance panel called the Navy Enterprise Information Technology Guidance Board. Each month, it convenes to review pleas for special dispensation from Navy IT managers who still don’t want their networks to be assimilated into NMCI.

“And guess what? We’re not approving any more networks,” Haith said. “We’re not even allowing waivers. We’re bringing them all in to the NMCI environment. And if we can’t get them all migrated by March of 2014, they will be migrating into NGEN.”

Haith said the problem of IT operations running off the Navy’s broader grid wasn’t limited to networks — data centers to support them have popped up around the world. As part of the federal data center consolidation initiative, the Navy has been trying to cut down its number of server facilities. First though, the service had to determine how many it actually had. Two years ago, it did not know.

The CIO’s office tasked the Navy’s Space and Naval Warfare Systems Command with counting them and determining which could be shut down.

“We did the boots-on-the-ground thing. We sent SPAWAR out in teams to go out to all these sites and tell us what we’ve got, then come back and report it,” she said. “Wow. We thought we only had 6,000 servers and maybe 100 data centers. Turns out we have 32,000 servers that we know of today, and we have 210 data centers. We were like, ‘Where did all this come from?’ We have a problem. We never recorded it. We never knew how much money we were spending on it.”

Just like Gmail

The Navy decided it would try to scale back its number of data centers to just 25 and Haith said, as with the network consolidation effort, there was a rebellion in the ranks.

“They said, ‘You can’t take my data center. My data has to be right here with me.’ We had to say, ‘Do you know what the cloud is? Do you have a Gmail account?’ That’s basically what we’re talking about,” she said. “You don’t have to have your data with you. Education for us has been enormous in this area, particularly with people in my generation and older, who think data has to be sitting right next to you in order to do your job.”

Haith said the perceived need to have data stored locally wound up leading to a massive underutilization of servers the Navy already owns.

“People don’t virtualize. They just don’t do it. They’re like, ‘We’ll just buy another one.’ We keep buying new servers over and over again,” she said. “Our hardware procurements, we think, are almost $500 million, just on servers, every year.”

Many of those data centers sprang up to support specific applications that Navy components determined they needed in order to meet their individual missions. Two years ago, the Navy said it had about 15,000 distinct software applications, and that consolidating and eliminating them would be a huge challenge.

But once again, that turned out to be only the tip of the iceberg.

“We started doing some checking, and found out we have 42,000 applications just in our (research, development, test and evaluation) community. Then it turns out we have another 27,000 that are not in programs of record that need to be migrated. And we still haven’t figured out how many we have on the warfighting side,” she said. “And a lot of those applications have problems. They’re not compliant with current regulations. They’re in some sort of sustainment mode, or if we get rid of one, it will break something else. It’s a big challenge for us.”

The next challenge, she said, is consolidating software licenses. Last year, the Navy signed its first big enterprise-wide licensing agreement. It was for Microsoft products, and all Navy components now are required to use it if they want to buy anything the Redmond, Wash., company sells. Next, the Navy expects to sign a deal with Oracle, and 17 major providers are next in line.

And as in other areas of the Navy’s IT spend, Haith said at the headquarters level, officials still don’t know what software licenses the Navy owns today, nor whether it’s using what it’s already paid for.

“Not only do we not know what we’ve got, we can’t even get anybody to tell us what we need for the future. That’s also because we don’t have an IT asset management tool, but we’re working toward that,” she said. “We’ve also been asked by many of the software vendors to do this, because they’re tired of dealing with 23 different Navy budget submitting offices, everybody with different requirements, and then the individual program offices below that. There’s big dollar savings for us there, and we expect it to happen soon.”

Investment needed upfront in some cases

Even if the centralization of authority over IT spending will bring the long-run savings the Navy’s hoping for, Haith said many of the IT modernization efforts the Navy wants to undertake will require some up-front investment. While that’s a tough sales pitch at a time when every corner of DoD is being told to cut back on operation and maintenance spending immediately, she said it’s a pitch the CIO’s office needs to continue to make.

“We have a challenge with our leadership in getting them to understand that in order to get where they want us to be, they have to give us some money to get there first,” she said. “All of the other services have the same problem. We cannot get them to understand. If it’s going to cost us close to $50 million to close data centers and the return on investment is $200 million, you’ve got to give it to us up front. We can’t cut everything right now, because we’ll never get there.”

But the Navy’s IT leadership bears some blame for failing to articulate those possible returns on investment, because it hasn’t been able to make concrete assessments of precisely where the money’s going, Haith said.

“For instance, on data center consolidation, we didn’t look at the full gamut. We only looked at hardware, software, and people,” she said. “But there was an energy cost, and unfortunately, we’re only starting to put smart meters on our buildings, so we weren’t sure what the [energy] cost was. We didn’t think about the security in the buildings, we didn’t think about tech refresh, we only looked at the here and now. We didn’t think about all the people to take care of the applications, because we didn’t even know how many we had.”

In the short term, amid the Pentagon’s uncertainty about the 2013 budget, the Navy has ordered another 25 percent reduction in IT spending in this year alone. In response to a question from Federal News Radio, Haith said the initial guidance from Jonathan Greenert, the chief of Naval Operations, appeared to be based on the assumption that the Navy was incurring exorbitant expenses on day-to-day office technology such as mobile devices and printers, and that the spending could be quickly scaled back.

“That’s not where we’re spending the money,” she said.

The spending, she said, is on servers and other IT infrastructure. In response to Greenert’s guidance, which is intended to help the Navy conserve as much cash as possible in anticipation of both a full-year continuing resolution and budget cuts under sequestration, Haith issued a memo last week, telling Navy components that many big IT purchases were to be put on hold.

“I was surprised my car was still out in the parking lot after that memo,” she said. “But I have a $32 million purchase request for servers sitting on my desk right now, and I’m like, ‘Why? Why do you need $32 million worth of servers for this capability? We’re putting a restriction on it. It’s got to be mission- essential in terms of loss of life to our sailors if we don’t do it, or we can’t deliver a critical milestone in a program of record. Otherwise, it’s going to sit in a hold pile until we get some resolution in this budget. We’re scrutinizing everything.”

RELATED STORIES:

Navy begins dismantling NMCI for NGEN

Navy must cut IT business systems spending by 25 percent

Navy targets December for NGEN RFP

Navy targets 15 large software vendors for enterprise deals

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.