An email from Kevin Allen, the Navy’s program manager for Enterprise Systems and Services on June 11, which Federal News Network obtained, outlined just how problematic the electronic procurement system (ePS) program is after almost two years since the Navy made the contract award.
Allen wrote that problems ranged from vendor performance to implementing an unsatisfactory and inflexible product to significant cost and schedule increases.
For instance, the Navy said “100% of [limited deployment] requirements NOT met — 295 open defects (user acceptance testing (UAT) and IT) as of 6/3 (over 100 critical/high).”
Another issue was user feedback. According to the email:
Users graded the system’s usability as an “F,” and said it was not an improvement to the system it’s intended to replace.
Even after users were trained, they still needed “significant support” from CGI
ePS can’t produce “legally sufficient” contract documents and doesn’t validate data when it’s entered
19 of the defects deemed “critical” or “high” would have still been in place when the system went live even after more than 18 months of development, a state of affairs Allen characterized as “unacceptable risk.”
Around cost and schedule, Allen wrote:
CGI estimates delays the program has already incurred would cause cost increases of $1 million per month; and
The Navy’s program management office estimates total costs would rise by 350%, and the timeline to deliver the first and second software release would be extended by 84%.
“The ePS program experienced several technical and schedule challenges that led the Navy to pause the current work,” Les Hubbard, the program executive officer for the Program Executive Office for Manpower, Logistics and Business Solutions (PEO MLB), wrote in an email to Federal News Network. “This pause will allow the Department of the Navy to review approaches, to include agile-based development constructs, to best meet the DON contract writing system requirements. Software capabilities consistently evolve and the Navy wants to ensure ePS remains a world-class contract writing system into the future. The pause will allow us to evaluate new developments, sources and capabilities that may improve future performance.”
CGI was unable to secure client approval to comment on the story at the time of publication.
The Army, which hired CGI under a 10-year, $133.9 million contract in June 2017 to implement the Army Contract Writing System (ACWS), based on the same commercial software, also is facing problems.
Army facing schedule slips, rebaselining
The Army, which issued CGI a “cure notice” in 2019 to address schedule concerns and challenges with complex interface requirements with other systems, has also seen cost and schedule slippage.
“ACWS product development costs have been realigned to more accurately reflect the types of work being performed in each area. Product development costs decrease in fiscal 2022 completes initial operating capability (IOC) and transitions to focus on development of full operating capability,” the Army wrote in 2022 budget justification documents. “Cybersecurity support and testing activities have been shifted to those sections driving the decrease. Development priorities have been delayed due to schedule slips due to requirement complexity being underestimated by the vendor. The program recently rebaselined its schedule to more accurately reflect the complexity of requirements to support the Army’s contracting community.”
An industry source familiar with both programs said the Army’s problems are much worse than the service makes it out.
“The Army did something similar to an analysis of alternatives and wanted to have some answers about where the ACWS was going before new leadership came in,” said the industry source, who requested anonymity in order to talk candidly about the acquisition. “They wanted to say, ‘here is [the] problem and here is what we are doing about it.’ So the Army engaged in market research by reaching out to vendors and others to see what is possible.”
Col. Donald Burton, the project manager for the Army’s PEO EIS’s Defense Integrated Business Systems portfolio, which manages ACWS, said in an email to Federal News Network that they have not issued a stop work order to CGI. But he confirmed the service had looked at other potential options.
“The Army conducted limited market research to assess other potential capabilities and approaches. This market research included the Air Force CON-IT system,” Burton said. “The purpose was to confirm what new developments, sources and capabilities can be validated that may enhance/improve future performance as the system comes on line in 2022.”
The industry source said the Navy and Army are seriously looking at the Air Force’s system, which is considered a government-off-the-shelf (GOTS) type of system, which is built using software from Appian.
Moving to the Air Force CON-IT system seems to be appealing given the challenges with the CGI effort and the fact that both the Air Force and the 4th Estate agencies are using it.
Former DoD Undersecretary of Defense for Acquisition and Sustainment Ellen Lord issued a memo in August 2020 requiring the 4th Estate to use the government-developed acquisition writing system among other steps.
“The popular answer is to use the home grown system from another service because they tried a commercial option and that didn’t work,” the industry source said. “But both the Navy and the Army are trying to modernize a GOTS system in the first place. The question should be how they can best deliver on mission now. I think both are doing a lot of soul searching about whether it was the commercial COTS approach that was flawed or did they simply choose the wrong vendor and integrator, and what role did the government play in the struggles?”
Navy’s Hubbard wouldn’t comment on a potential move to the Air Force CON-IT system, but said, “the DON regularly conducts a review of technologies available in the market as part of our due diligence in being good stewards of taxpayer dollars. An Air Force system is one of several solutions the DON is considering as part of our commitment to deliver a contract writing system capability to the Navy and Marine Corps.”
The Army’s Burton said his office “will continue to monitor and coordinate with the Air Force as they move forward on the CON-IT program,” but the service is committed to the CGI effort.
“Additionally, and on an ongoing basis the Army — to the maximum extent practicable — will take advantage of available market research methods in accordance with the Federal Acquisition Regulation to identify emerging technologies, government or commercially sourced, that are available to meet the requirements of the agency,” he said. “Based on the information available today about the current Air Force system’s functionality, CON-IT currently meets a subset of Army requirements, and it would require investment for Army requirements, as well as potentially require investment in development of interface integrations with other Army Enterprise Resource Planning (ERP) systems.”
Tens of millions of dollars already spent
The investment piece Burton talked about is critical. Both services already have spent quite a bit of money with CGI and have little to show for it.
The Navy spent more than $15.7 million in 2020 and $10 million in previous fiscal years for product development. It expected to spend another $21.5 million in 2021, but the pause likely will reduce that total spend.
For 2022, the Navy asked for $7.3 million, a $4.4 million increase over 2021 to support limited deployment of ePS and additional license maintenance costs, according to Navy budget documents.
The Army, meanwhile, is asking for an increase of $4.6 million for a total of $22.9 million in 2022 for its research, development, test and evaluation (RDT&E) budget, according to budget documents.
The service plans to spend $22 million in 2021 and spent $16.3 million the year before.
It also is asking for another $17 million for procurement and another $13 million for operations and maintenance next year, according to budget documents.
That would mean the Army wants to spend another $52.9 million on ACWS in 2022 despite limited progress.
And the limited progress isn’t new. DoD has been trying to move to a standard enterprisewide contract writing system since 2011. It’s also why Congress eventually got involved in the fiscal 2018 National Defense Authorization Act, supporting DoD’s portfolio approach and the need to consolidate the overall numbers of systems.
“Based on the progress in this area, the conferees encourage the department to initiate or accelerate such efforts in other areas, to include personnel and pay, financial management, and enterprise resource programs,” lawmakers wrote in 2018.
But over the last few years, lawmakers have been less excited about DoD’s progress, threatening to cut funding in 2020. In the end, lawmakers kept the full allocation, but their frustration with the program was obvious.
Lawmakers wrote in the fiscal 2020 NDAA that the Senate Armed Services Committee “remains concerned about the inability of disparate acquisition strategies to leverage common requirements, commercial processes and solutions for writing contracts.”
The 2021 NDAA didn’t include any new language about the contract writing system. They also authorized another $22 million for the Army and another $8 million for the Navy contract writing system initiatives, respectively.
Navy data migrations on hold
Both services seem to recognize the contract writing systems are struggling.
Navy’s Hubbard said schedule and technical challenges initially pushed back the first roll out of capabilities in 2020, but further delays and challenges hampered that effort.
“The DON has not started data migration yet. However, the DON has identified potential challenges with the migration of data from legacy systems. We performed data cleansing and conducted mock migrations in preparation of data migration which has not started,” he said.
The Army’s Burton said initial operating capability for the ACWS remains the goal in 2022 at the Carlisle Barracks Mission Installation Contracting Command (MICC) and the Army Corps of Engineers in Savannah. He said the Army also plans to upgrade of the current deployed minimum viable solution software at Humphreys Engineer Center Support Activity (HESCA) at Fort Belvoir.
“While there have been schedule slips, the Army has incentivized CGI and held CGI contractually accountable to meet IOC schedule and quality parameters while protecting against unplanned financial commitments,” he said. “The Army is in the early stages of testing data migration and has not encountered challenges to date. Data migration functionality was not included as part of the pilot deployment to HECSA.”
Another challenge the Army will face going forward is the loss of a key senior leader. Army Deputy Assistant Secretary of the Army for Procurement Rebecca Weirick is retiring on July 9 and was considered by many to be a “forceful agent of change.” Her retirement comes about a year after Stuart Hazlett, the previous deputy assistant secretary of the Army for procurement, left in May 2020.
Both the Navy and Army have tried to work with CGI to fix these ongoing problems.
Hubbard said the Navy has taken several steps to correct deficiencies in the project.
“The DON uses risk and schedule management practices as a standard for monitoring development programs. We held system engineering technical reviews with entry and exit criteria and quarterly program reviews with CGI, senior executives and stakeholders, which we used to identify and issue resolution guidance similar with all Navy acquisition projects,” he said. “During the product development life cycle, the Navy and CGI have collaborated towards joint resolution of challenges as they arose until the challenges and risks led to a decision to pause.”
The industry source said both the Army and the Navy are trying to deploy a commercial contract writing system that has never been deployed in DoD, which is a big part of the problem.
“They chose a product that was not as mature as CGI represented at the time of award and now they are dealing with that,” the source said. “The question is why didn’t they see that when they bought it? I think that’s part of the soul searching they are having now. The other question they have to answer is if they go with another COTS product, are they going to have a better experience? That’s the question many in industry are wondering.”