Email, in many ways, is the original enterprise application because literally everyone in an organization uses it. IT departments in government and industry alike have always turned to email as the first application to deploy on new platforms. First on mainframe terminals, then on local area networks, then on enterprise networks.
Now email is, for many large organizations, the first application moving to a cloud-hosted platform. A case in point: the Veterans Affairs Department, one of the first federal agencies to move email and related employee productivity software off its servers and into a commercial cloud environment. With five years of experience using, in this case, Microsoft Office 365, VA’s IT staff has learned lessons they’re applying to the expansion of cloud activities on a significant scale.
Among the lessons, according to Dave Catanoso VA’s director of Enterprise Cloud Solutions, is simply this: “Don’t be afraid to do it. You really have to just be confident you can succeed and move forward.”
Speaking at the Federal News Network Civilian Cloud Exchange, Catanoso described one benefit of having moved user accounts, along with the associated Microsoft Exchange servers, to the cloud. Namely, the ability to adapt quickly when the pandemic hit.
“We actually pivoted from a legacy of Skype infrastructure capability to Microsoft Teams in the middle of the COVID pandemic,” Catanoso said on Federal News Network’s Cloud Exchange. “It was probably one of the largest migrations of anyone to Teams ever. We did it during the pandemic. And it’s worked great.”
With more than 375,000 employees, VA shares the basic scaling challenges that other large agencies and large companies do. But what about VA specifically as a health care organization – how is cloud helping the VA missions of health care, other statutorily mandated benefits, and burial services?
Here again, Catanoso said, VA’s scale calls for careful planning of any move to the cloud. He pointed out that VA operates some 140 hospitals and 1,200 clinics. The Veterans Benefits Administration processes thousands of claims a day from among its millions of eligible recipients.
As for applications, Catanoso said the specific goal for the Enterprise Cloud Solutions office is to move about a third of VA’s 1,000 major applications by the end of fiscal year 2024.
“We’ve got almost 130 applications in the enterprise cloud today,” he said, not including software-as-a-service subscriptions from SalesForce and Microsoft Dynamics.
As VA gets past the applications that are easy to “lift and shift” to the cloud, Catanoso said, migrations require more planning and technical work so that applications can take advantage of services offered natively by the commercial cloud provider. For its enterprise cloud efforts, VA has contracts with Amazon Web Services and Microsoft Azure.
To enable future migrations, Catanoso said, his office has created a service called VA Platform One. It “is designed to put in a standardized way for VA to containerize applications, and use microservices” from cloud providers, he said. “And really take advantage of all the cloud optimization capabilities out there. We have teams using the cloud at various levels, but we’re trying to make it more standardized.”
As it proceeds with moving hundreds of applications, Catanoso said, the VA Enterprise Cloud is taking an unorthodox approach.
“Normally what an organization will do, and any consulting team will tell you, when you migrate to cloud, you want to start with some small applications,” he said. “First, you want to pilot it, get some experience and then gradually get bigger.”
But not VA.
“Because of a compelling business need at the very beginning of our existence in 2018, we took the exact opposite approach. We decided to move some of our biggest, most mission critical systems into the cloud immediately,” he said. “Once we demonstrated they could move and be successful, then it was really a big confidence builder for everybody that wow, if these will run in the cloud, then there’s really not anything we couldn’t move if we need to.”
That early list of big applications included the Veterans Benefits Management System. It operates worldwide, with an average of 5,000 VBA employees logged at any given time. It was hosted in an expensive third party data center, and it suffered from performance problems, Catanoso said.
The migration started with a software “adapter” that helped users locate a given document from among 800 million during the time it took to migrate documents from the legacy host to the enterprise cloud. Once all the documents were in the cloud — in this case to Amazon using the company’s Snowball utility — the Enterprise Cloud office proceeded with the application logic.
“In May of 2019, we cut it over and was working great. From day one, we get great performance,” Catanoso said. For example, 100-megabyte documents in the old system took up to 120 seconds to download; now it’s less than 30 seconds.
Although VA is in the midst of a highly visible project with Cerner Corporation to develop a new electronic health care record system, Catanoso noted that the legacy VISTA system will remain a part of the VA computing complex for the foreseeable future. VISTA, too, is part of the cloud migration.
In one early piece of work, and as a demonstration project, the VA Enterprise Cloud office migrated the VISTA instance of a small VA facility in Texas. Catanoso said that when that proved successful, the office moved the VISTA instance of one of the largest medical centers, that of Omaha, Nebraska. A challenge was reliably routing the fixed IP addresses of medical devices to the dynamically changing IP addresses of cloud-hosted VISTA instances.
Catanoso added, “But for the most part, it turned out to be fairly straightforward to get the application itself to run”
The two have been operating as production business systems in VA’s enterprise cloud since 2019. Catanoso said. Now the VA Enterprise Cloud is hosting all of the disaster recovery backup systems for all of the VISTA instances across the country. It is also planning the migrations of all of the medical centers’ VISTA instances to the enterprise cloud.
Catanoso pointed out that the application itself is not what VA is modernizing, but rather “the hosting infrastructure to improve reliability, performance, scalability and security.”
My HealtheVet is another critical application Catanoso’s crew has migrated to the cloud. It lets veterans create accounts and manage their prescriptions, appointments and health records. Program managers were immediately worried about whether the system could run in the cloud.
“We spent about six to seven months really working together to figure out how to change the architecture from some of the technology that was required to run it in the legacy data center,” Catanoso said. That included the database technology required to maintain a high level of performance and reliability.
The months of planning and testing led to an overnight cutover from the legacy data center to the cloud.
“We had to do that overnight, on the weekend,” Catanoso said. “It was a very, very deliberate exercise.”
Now that My HealtheVet is running in the cloud, Catanoso said VA is planning to host it in multiple cloud instances geographically, with direct internet connections to improve performance. He said one enabler will be the final version of the Trusted Internet Connection 3.0 from the Department of Homeland Security.
Ultimately, Catanoso said, agencies expecting greater use of cloud computing will continue to deal with complex, hybrid environments. And with a combination of applications that migrate natively, those that require a new code base and architecture to take advantage of cloud services, and those that will remain on premises.
He envisions what he called a standard containerization platform, via the VA Platform One, that will let VA run applications in either of its major cloud providers or on prem. That will let developers “focus on developing solutions and not ordering servers and trying to figure out what software they have to install and configuring their pipelines and those kind of things.”