Few VA employees easily navigating new EHR after training, months of use, auditors say

The inspector general at the Department of Veterans Affairs delivered a double-edged blow to VA’s massive electronic health record modernization effort this week, detailing both employee training failures with the new system and unreliable IT cost estimates for the overall project.

A small percentage of VA employees at the first go-live site in Spokane, Washington said they could easily use the Cerner Millennium EHR after training and using the product for two-to-three months, the IG said Thursday in a new report.

And on Wednesday, the IG said VA’s cost estimates for the IT infrastructure upgrades needed to support the EHR were unreliable. VA previously told Congress it would need $4.3 billion for IT infrastructure upgrades.

But the department failed to identify an additional $2.5 billion in “critical program-related IT infrastructure upgrade costs,” auditors said.

The IG’s latest report, again, calls into question the total lifecycle costs for the EHR modernization effort. Auditors in May said VA had underestimated the costs of physical infrastructure upgrades also needed to support the new EHR. Those are potentially worth $3.1-to-3.7 billion, costs that have never been reported to Congress.

These figures are estimates, the IG said, because VA lacks a reliable method for reporting infrastructure costs. VA’s Office of Information and Technology and the Veterans Health Administration are responsible for paying for IT infrastructure upgrades, yet the department’s Office of Electronic Health Record Modernization is supposed to report those costs to Congress.

VHA’s role is ‘undefined’

How these three entities — VA OI&T, VHA and OEHRM — interact with each other is partly to blame for the failures with employee training at the department’s first go-live site in Spokane, auditors concluded.

The IG said VHA doesn’t have a defined role in the decision-making or oversight associated with the EHR project, even though its clinicians and employees are the principle users of the health record.

The Spokane facility director and VHA acting undersecretary, for example, told VA OEHRM they felt an operational readiness assessment was necessary before the initial go-live.

“In addition, also months prior to the initial go-live date, the OIG found that the facility director and acting undersecretary for health reported significant concerns to VA OEHRM leadership regarding disturbing feedback from staff on the insufficient training curriculum,” auditors wrote. “With both issues, VHA met resistance from VA OEHRM leaders or was not included in decision-making discussions with VA OEHRM leaders. The OIG is concerned that failure to include VHA leaders and end users of the new EHR in operational decisions can affect the success of implementation.”

That could be why the training often failed to teach employees how to use the EHR to perform specific tasks core to their jobs.

“Facility leaders reported that training did not provide opportunities to test proficiency in navigating a wide variety of clinical scenarios; instead, instruction focused on the multiple steps required to complete specific tasks,” the IG said. “Facility leaders coined the term ‘button-ology’ to describe applications training content, as its focus was on which button to press to get a desired system outcome. One facility staff training coordinator reported, ‘it was just people sitting down and learning to use the buttons and not having any context for what they were doing.'”

An ‘utter waste of time’

Auditors described a wide array of problems with EHR training — and the checks VA used to measure employee proficiency with the new system after instruction.

Employees didn’t have enough time for training exercises, and the platform they used for instruction didn’t closely mirror the actual, live EHR, auditors found. Cerner classroom trainers often couldn’t answer employee questions, the IG said.

During the winter months before the go-live, VHA “super users” described training as “an utter waste of time,” and the Spokane facility director said he was “blown away” by their negativity. The IG said the department as a whole lacked a process to address those concerns from leadership and employees in Spokane.

Auditors surveyed VA employees at the Spokane facility about their experiences with the new platform. Just 5% of survey responders reported in the affirmative to all four items, the IG said.

Leaders within VA’s OEHRM acknowledged the problems with employee training — and suggested deficiencies with Cerner’s own training work.

“VA OEHRM Change Management completed ongoing assessments of Cerner’s work on training and identified frequent, recurring deficits in meeting project deadlines, staffing, management and quality of products,” the IG said. “Despite the extensive report of deficiencies with Cerner’s work on training, VA OEHRM contracting officials scored Cerner’s performance largely at the satisfactory level, the minimum rating to meet contractual requirements.”

Notably, auditors also expressed concerns with the proficiency check data OEHRM officials provided to the IG — and at one point suggesting VA cooked the books detailing the number of employees who had passed tests after training.

“Disturbingly, leaders from VA OEHRM Change Management withheld some training evaluation data requested by the OIG and altered other data prior to sending to the OIG,” the report reads.

The department told auditors 89% of VA employees passed their proficiency checks with a score of 80% or higher in three attempts or less.

But the IG said it found an earlier version of VA proficiency check results, which were drafted but not sent to auditors. Those results showed a significantly different outcome, where 44% of employees passed their proficiency check with a score of 80% or higher, and “27% passed on 1st attempt, 12% passed on 2nd attempt, 5% passed on 3rd attempt.”

When auditors asked OEHRM about the differences, VA said it removed the outliers from the data it later provided to the IG.

“The integrity and thoroughness of information provided by VA is required by law and is critical to the OIG’s mission,” auditors wrote. “The OIG has notified VA senior leaders of this issue and is further pursuing the matter.”

EHR strategic review results coming next week

Auditors said they submitted findings and recommendations to Carolyn Clancy, VA’s acting deputy secretary, and Richard Stone, acting undersecretary for health, in May, expecting to receive responses from both of them. VA’s EHRM office responded instead.

Those responses from VA’s EHRM office did not reflect their views, Stone and Clancy said later in mid-June. They concurred with all of the IG’s findings and later provided action plans, which hint at findings from VA’s recently completed strategic review of the EHR rollout.

VA acknowledged, for example, it needs to create a training environment that’s as close as possible to the actual EHR that clinicians will use. The department will also re-charter and repopulate its EHR councils with individuals who have expertise in VHA operations. These councils will evaluate clinical workflows, VA said in its action plan.

The department first launched the review back in March after hearing about systemic failures and concerns with the EHR rollout from employees at the first go-live site in Spokane, many of which mirror the findings from the IG’s latest report.

Department officials will describe their findings from the EHR strategic review before the Senate VA Committee next Wednesday, the panel said.

“This report makes clear the electronic health record modernization program needs strong oversight by VA and Congress to ensure it delivers reliable support to department medical staff and the veterans they serve,” Sen. Jon Tester (D-Mont.), committee chairman, said Wednesday in a statement. “While Secretary McDonough has taken positive steps to get this program back on track, the Senate needs to confirm a VA deputy secretary to help manage this effort, protect taxpayer dollars and deliver for all veterans. This project is critical to the future of our veterans’ health care, and I look forward to hearing directly from VA next week on their plans to make sure it is implemented correctly.”

Members of Congress on both sides of the aisle have questioned the accuracy of VA’s costs estimates for some time. The House VA Committee will host its own hearing the following week, Democrats said Thursday. The committee has invited McDonough to testify.

“Based on these reports and my previous conversations with VA, I continue to have serious concerns about the management of the EHRM program, including the lack of clarity around life-cycle costs, failures in staff training and inadequate change management practices,” Rep. Frank Mrvan (D-Ind.), chairman of the technology modernization subcommittee, said Thursday.

Mrvan said the subcommittee hearing, scheduled for July 21, will focus on the current status of the EHR project, as well as VA’s planned changes and path forward.

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.