Seven years ago, the Veterans Affairs Department decided it wanted to use technology to revolutionize the ways in which it manages its medical supply chain and tracks equipment throughout its sprawling nationwide health care system.
But in the case of at least two contracts that have already cost the department more than a half billion dollars, the results have been questionable, at best. In one instance, the department has decided to walk away from the project entirely, despite having spent $117 million and several years to implement it.
Insight by Galvanize: During this webinar Marianne Roth, the chief risk officer of the Consumer Financial Protection Bureau, will provide a deep dive into enterprise risk management at CFPB. Additionally, Dan Zitting, the CEO of Galvanize, will discuss how making better use of data and technology can help federal agencies more rapidly allow decision makers address and mitigate risks.
The recent missteps have raised serious doubts among lawmakers about VA’s ability to manage large IT projects, including an upcoming replacement of its electronic health record system, which is expected to cost more than $10 billion.
One of the programs, called the Point Of Use Inventory System, was supposed to integrate with the department’s existing health IT infrastructure and give managers deep insight into VA’s supply chain for consumable medical supplies, including by keeping tabs on hospitals’ existing inventories and ensuring they ordered the right amounts to replenish what clinicians were using.
But after VA awarded the contract in 2013, the system — based on a commercial software package called Catamaran — was almost immediately hobbled by crippling technical problems across the 22 sites where it was first deployed.
The department opted against exercising any more of its contract options to install the system at other facilities, concluding that it would take at least a decade before the theoretical cost savings it could generate reached the break-even point, considering the money VA had already spent. Last year, it began reverting all 22 sites back to the legacy system with which VA started, the Generic Inventory Package.
“Automated [health care] supply chain systems do exist, and have existed in the private sector, but perhaps this wasn’t the best choice for us,” Tammy Czarnecki, the Veterans Health Administration’s assistant deputy undersecretary for administrative operations, said Tuesday during testimony before the House Veterans Affairs Committee. “We had made an incorrect assumption that since it was a commercial-off-the-shelf product, we would be able to implement it fairly easily. That did not happen.”
VA had been attempting to deploy the Catamaran product in tandem with another larger logistics project, called the Real-Time Location System (RTLS). That contract, awarded to HP in 2012, had an initial ceiling value of $543 million. But the department has decided to end that contract as well, despite having spent $360 million so far.
RTLS was intended to keep tabs on more durable medical equipment. Wi-Fi-connected location trackers were supposed to let hospital staff figure out exactly where items were located in a given facility; it also aimed to improve patient safety by making sure each and every piece of surgical equipment went through the proper steps in the sterilization process before they were returned to operating rooms.
But by early 2015, it was clear that the system wasn’t quite doing what VA’s clinicians and information technologists had in mind.
The department flagged 245 separate defects, and issued a “cure notice” telling HP to fix them. Among the problems: the system could only accurately track the equipment it was supposed to monitor 40 percent of the time, and it only identified an item’s general location in a medical facility – not the precise room in which the piece of equipment was actually located.
Dozens of the problems were still unresolved a year later, and in September of that year, the department wound up restructuring the contract as part of a confidential agreement in which it also absolved HP of any liability for failing to meet the terms of the contract.
Czarnecki said the department took that approach because it’s not clear that HP was totally at fault.
Rather, she chalked the system’s problems up to a mutual misunderstanding about what VA actually wanted the system to do.
“We have to remember that a system like RTLS was new in the medical space at that time,” she said. “We had written in the contract that we needed to have location accuracy. The contractor’s definition of location accuracy and our definition was different . We needed clinical accuracy. We needed to know that piece of equipment was in that particular room, so that we could move it to where we needed it. The contractor wasn’t necessarily wrong: there were definition differences in a number of the areas, and so those needed to be clarified.”
But VA’s inspector general, which has already conducted its own extensive oversight of RTLS, says the department’s failure to clearly define its requirements for IT systems is not a one-off problem.
“I do think VA has a capability of managing these kind of projects, but they chose a Big Bang approach, as opposed to an agile, incremental approach,” said Michael Bowman, the director of the OIG’s IT and security audits division. “If you break the project up into smaller pieces and get the end user involved early on, you discover early on whether or not your contract requirements are being met. This is a theme that has showed up in various IT development projects over the years.”
Indeed, in a December 2017 report, the IG found that VA had disregarded its own policies regarding incremental software development during its rollout of RTLS.
The Performance Management and Accountability System (PMAS), which the department implemented in 2009, mandated an agile approach to large IT rollouts. PMAS directed that big projects be broken down into smaller chunks that took no more than six months at a time to develop, with frequent and ongoing feedback from users.
But the VA project management office in charge of RTLS disregarded those policies, even though it knew the program included a large amount of risk, said Nicholas Dahl, the deputy VA inspector general for audits and evaluations.
He added that VA’s approach also violated federal cybersecurity policies, since it let the new logistics system connect to the department’s IT network without obtaining the necessary authority to operate (ATO).
“And despite the guidance from the technical acquisition center and VA policy, the RTLS PMO did not ensure the vendor could meet contracted functionality requirements, such as accurate asset tracking on the initial $7.5 milllion task order before ultimately committing a total of $431 million to the same vendor for further RTLS deployments,” Dahl said.
However, Czarnecki rejected the notion that VA’s investment in RTLS has been entirely wasted.
Although its contractual relationship with the prime vendor who built it is ending this summer, she contended the department has derived significant value from the project, including new capabilities to track the sterilization of surgical and dental instruments at the 60 facilities where the system has been implemented.
“VA can be assured the right instruments are being delivered to the right operating room for the right procedure,” she said. “VHA and OIT have continued to align and improve project management processes following the conclusion and the publication of the OIG report. The deployment efforts have been managed utilizing sound project management principles. VA will perform continual risk assessments to assure that risk associated with deploying RTLS on the VA network are minimized.”
The department’s missteps relating to logistics systems precede what is likely to be the largest health IT investment in its history, an anticipated multi-billion dollar award to Cerner, intended to let VA adopt the same electronic health record the Defense Department has already begun to deploy.
Rep. Jack Bergman (R-Mich.), the chairman of the House Veterans Affairs Subcommittee on Oversight and Investigations, said he agreed that RTLS delivered at least some usable capability for VA.
He added, however, that he hoped the department had learned some recent technology acquisition lessons before it moves onto its EHR acquisition, the value of which he pegged at $14 billion.
“The stakes are dramatically increasing,” he said. The electronic health records modernization program, which we have been awaiting since last year, is perhaps the largest medical information technology collaboration in the history of the federal government. Its scope and its scale is challenging enough, and the VA cannot afford this sort of organizational infighting. The committee is committed to exercising vigorous oversight of the Cerner implementation.”
Along those lines, the Veterans Affairs Committee greenlighted legislation earlier on Tuesday that would impose strict requirements for VA to deliver progress reports on its pending agreement with Cerner.
Among them, the department would have to send lawmakers copies of all orders, contracts or agreements for the new EHR within five days of their signing, plus notification to Congress within 10 days anytime the acquisition project encounters trouble, including data breaches, bid protests or other delays.