The Department of the Navy’s third attempt to modernize its contract writing system and overall electronic procurement system is on shaky ground. The latest bid, which followed at least two previous failures, is facing similar troubles as earlier projects: incomplete planning, a technology platform with questionable maturity and, maybe most striking, the hubris of the leadership that their current plan will be successful no matter what evidence emerges that tells a different story.

While organizations can more directly address shortcomings in planning and technology, experts say, time and again, it’s clear that overconfidence and inflexibility tend to sink large scale technology programs.

That hubris reared its ugly head most recently in June when the Navy retracted a technical assessment that found shortcomings in the Navy’s electronic procurement system (ePS) program. In its internal objective assessment of the technical approach for ePS, the Naval Information Warfare Systems Command (NAVWAR) detailed a host of questions and potential problems about the project.

But within three months, and following a series of questions from Federal News Network, and possible pressure from Appian, whose platform the Navy is using to build ePS, NAVWAR said the report is erroneous and has no plans to redo it.

“The Electronic Procurement System (ePS) Technical Assessment was developed using partial and outdated information. After receiving additional and updated information, it was determined that the report was not accurate,” a NAVWAR spokesperson told Federal News Network in an email. “No new or revised assessment is required because the additional information we received eliminates the need for one.”

The decision to fully retract the report and not issue a new one opens the door to even more questions about the future of ePS and whether the Navy finally will successfully modernize its 30-year-old electronic procurement system.

And experts said merely withdrawing the technical assessment, which the Navy’s Program Executive Office for Manpower, Logistics and Business Solutions (PEO-MLB) asked for as part of its risk management strategy, doesn’t offset the critical findings.

It also raises even more questions about how the assessment went through the entire development process, multiple layers of review and finally approval by Rob Wolborsky, who is NAVWAR’s chief engineer, only to be so poorly done and missing key information that the organization decided to take it back a few months later.

“The technical assessment team received additional information, including additional architecture documents, through technical exchange meetings and discussions with the PEO MLB ePS project team and functional requirements owners,” the NAVWAR spokesperson said, without offering any further details about how these new documents addressed the findings of the technical assessment team.

Additionally, NAVWAR also said it mislabeled the document as unclassified, again raising even more questions among observers about how a well-worn process fell apart in so many different places.

The spokesperson said NAVWAR is reviewing the document sign-off and release processes.

Despite these clear missteps, PEO-MLB requested the technical assessment for a real reason. Christine Rodriguez, program executive officer for PEO-MLB, said in an email to Federal News Network that the technical assessment is “to ensure a solid engineering foundation and processes are established and followed.”

In the now retracted technical assessment, which Federal News Network obtained, NAVWAR found several glaring holes in PEO-MLB’s approach to implementing ePS, which is based on the Air Force’s contracting IT (CON-IT) platform:

• Communication challenges across the effort
• Software and licensing costs
• Missing capabilities requirement document
• Overly complex solution (over-architected)
• High-risk development workflow
• High-risk deployment workflow
• Missing documentation
• Lack of dedicated engineering resources
• Missing sustainment plan
• Appian product shortcomings

“In conclusion, the implementation of ePS has encountered planning and communication challenges. However, despite these challenges, ePS has achieved significant milestones, including deployment to a production environment, obtaining an authorization to operate (ATO), onboarding a limited user base and awarding two contracts. These accomplishments give the appearance that ePS is on track to be a successful effort in the long term. However, it is essential to acknowledge the potential risks that come with the lack of critical planning,” the authors of the report said. “Without foundational planning documents such as the business process model, capabilities requirement document and business enterprise architecture, there’s a chance that the system may not fully align with the overall business requirements. This misalignment could lead to operational inefficiencies, user dissatisfaction and ultimately, an unsuccessful implementation.”

The latest iteration of ePS is partly a spin-off of CON-IT, the platform the Air Force has been implementing for Defense contract writing since 2018. It is a low-code system built on the Appian platform, meant to standardize contract writing, reduce costs and create efficiencies on both ends of the contract.

The Navy — along with the Army, the Air Force and the Fourth Estate agencies — is trying move off of the Standard Procurement System, developed in the 1990s. Like any technology system built 30 years ago, it has run its course and DoD is turning SPS off by Sept. 30, 2026, according to a 2020 memo from Kim Herring, the then-acting principal director of the Office of Defense Contract and Pricing.

The technical assessment, written by a team of NAVWAR experts led by Edmund Kuqo, whose expertise includes cloud, cybersecurity, DevSecOps, requirements and being an agile coach, found the Navy’s approach may be overly complex, based on the decision to use a “system-of-systems” approach, which added layers of engineering complexity without providing additional benefits.

“Further analysis should be performed to determine the true ‘total cost of ownership’ of including CON-IT and utilizing a system-of-systems approach and determine if the benefits outweigh the cost,” the authors wrote.

Kuqo declined to comment for the story, referring all questions to the NAWWAR press shop.

In the now-retracted document, the authors also detailed potential problems with the Appian platform, including questions about its maturity and approach to software licensing.

Appian declined to comment on the specific findings of the report, but highlighted the Navy’s progress so far.

“The Navy ePS program is a success, and the success is driven by Appian and CON-IT, which was also successful at the Air Force and is succeeding at the Army,” Cindy Cheng, Appian’s senior director of communications, said in an email to Federal News Network.

The authors of the assessment, which included at least one member who listed their expertise to include the Appian platform, found “from previous Appian experiences within the assessment team, load balancing/high availability, error handling and process isolation (threading) features are either non-existent or poorly implemented despite the official Appian documentation stating differently.”

An industry source who is familiar with the Appian platform and requested anonymity to speak about the report, said NAVWAR’s analysis was poorly done because it used outdated information, and the command was right to retract it.

“The Navy ePS program is on track, on budget and hitting all of its milestones, and since the report came out, they’ve onboarded yet another tranche of users and migrated $1.6 billion in contracts on. The system has executed over $22.6 million in new contract actions in the system, and have every expectation of hitting their goal of having 1,500 users onboarded by the end of 2024, so any suggestion that the Navy ePS program is actually struggling doesn’t seem to be supported by what we’ve seen and heard,” said the source. “That doesn’t mean that there aren’t critics who would prefer the program going a different direction. But in terms of actual data of any actual problems with the project, we haven’t seen any such.”

The source said Appian didn’t get a chance to review or comment on the technical assessment.

An ‘extremely challenged’ program

But other experts said even if you discount the concerns about Appian, the report’s findings show ePS is far from being on solid footing.

Reid Jackson, the CEO of Unison, which provides electronic procurement system software to several agencies, including the Navy’s Office of Naval Research, Navy Installations Command and the Marine Corps Community Service Organization, said it’s difficult to reach any other conclusion than ePS is an “extremely challenged program.”

“The Navy says it is missing several elements that they would routinely see in a major enterprise information system, regardless of the approach chosen. They’re missing elements like the system architecture, a business process model, external interface map they have chosen,” Jackson said in an interview with Federal News Network. “So their words about this program, when you strip off the polite language around it, I think has left the technical assessment team deeply concerned about the ultimate success of this program.”

Jackson, who highlighted that Unison didn’t participate in the report and didn’t know NAVWAR was even working on the report until it became more public, emphasized the report carries more weight because it was done by the Navy’s own systems people looking at a program that is being run by a functional organization.

The assessment team added that it does not believe selecting CON-IT was a bad decision, but the Navy should be made aware “that utilizing external dependencies can come with hidden costs and in many cases to drift from the initial code baseline due to differences in requirements between different DoD components.”

Disagreement over analysis of alternatives

Jackson said the lack of competition around ePS is another factor causing the Navy trouble.

“When I look at this report, I can’t help but feel it doesn’t have to be this way. The competitive process that is almost universally followed, and certainly advocated for by procurement policy folks, offers benefits and can avoid this problem of selecting a solution and then later finding out it doesn’t meet your needs. It’s not mature. It’s not ready for the DoD enterprise,” he said. “In the technical assessment, they also say, by the way, you don’t have requirements, you don’t have interfaces and you don’t have a business enterprise architecture. All of those are things you would have completed in order to communicate to industry as part of a competition and as part of explaining what you are trying to achieve. This is not my report. I read this report, and I think it just doesn’t have to be this way. This is specifically what competition avoids: late surprises.”

PEO-MLB’s Rodriguez pushed back against the idea that there wasn’t competition.

She said the Navy completed an analysis of alternatives in 2013 that recommended holding a competition to select a commercial product that would be extended via business process modeling capability and data integration to fit capability gaps.

“As part of the seven-month feasibility study in 2021, we assessed the feasibility of the technical baseline of CON-IT for the core contracting module (CCM), as is typical in a procurement process. Other factors considered in the study were cost, speed to deliver, and ability to meet the Navy’s complex contract writing requirements,” Rodriquez said. “Feasibility studies, prototyping and technical assessments cannot identify every potential issue with a system because system requirements evolve based on regulation and policy changes, especially when delivering solutions that need to meet the needs for 10 unique [Navy Department contracting organizations]. However, we are confident in the selection of CON-IT as the baseline platform for the CCM and that our agile software development process that includes end user and stakeholder engagement and feedback will continue to improve the CCM and address any potential challenges that arise throughout the process. We will ensure that any issues with CON-IT are addressed.”

Navy’s 7-month feasibility study

At the same time, the technical assessment team also expressed concerns about the Navy’s process to analyze all the possible options.

“While the selection of CON-IT in hindsight is not necessarily a bad decision, the action of selecting CON-IT without performing an AoA is high risk,” the report stated.

PEO-MLB’s Naval Applications and Business Services (NABS) portfolio, which is leading the latest modernization effort, did the feasibility study after several previous modernization attempts. The Navy’s goal is to modernize and consolidate more than 245 DON legacy contract writing systems (CWS) and other ancillary procurement systems. These procurement systems write and manage most DON contracts under 10 heads of contracting activity (HCA).

“DON ePS will provide standardized, seamless, end-to-end contract management (i.e., sourcing) of services, supplies and construction to the estimated more than 16,000 contracting personnel (6,420 critical users) within the DON HCAs,” said Rodriguez. “ePS will be designed to execute 100% of the DON’s contracting mission throughout the procure-to-pay process (P2P).”

The move to CON-IT comes after the Navy’s previous attempts to modernize its systems fell flat. In August 2019, it awarded CGI a 10-year, $222.9 million contract to build the system, only to cancel it two years later.

After the Navy’s struggles with CGI and seeing the Air Force’s success, leaders decided to move over to CON-IT. The Navy launched its minimum viable product last September.

“ePS went live on Sept. 29, 2023 with a minimum viable capability release (MVCR) providing automated contract writing and the ability process requirements from inception to award, while incorporating flexibility to interface with other electronic systems,” Rodriguez said. “The ePS MVCR included the ePS core contracting module (ePS-CCM), data integration layer (ePS-DI) and Navy procurement data standard (PDS) pre-validation service (ePS-PRE). MVCR functionality included basic Federal Acquisition Regulation (FAR) contracting, such as firm-fixed price (FFP) contracts, purchase orders, task or delivery orders, and indefinite delivery vehicles under the Simplified Acquisition Threshold (SAT) and using the Simplified Acquisition Procedures (SAP) for making purchases of supplies or services. Today, more than 100 contract actions worth more than $1.5 billion in contracts have been successfully executed using the ePS CCM.”

Navy requests $15.6M more in funding for 2025

Between September and July, PEO-MLB has made an additional 20 software updates and currently supports 293 users and has executed more than 100 contract actions worth more than $1.5 billion in contracts.

The Navy asked for $22 million, a $15.6 million increase, for fiscal 2025.

“Increase in funding for ePS to leverage existing technologies and capabilities to gain efficiencies. Increase supports additional sustainment costs, which includes licenses for functionality releases and user accession. The increase will support release of capabilities that aggregate to support the Navy’s end-to-end contract writing system,” the service wrote in its budget justification.

The service received $5.9 million in 2023 and requested $6.5 million in 2024.

Despite the progress on ePS, the Navy asked NAVWAR to conduct a technical assessment — which, industry experts say, is usually done in the pre-award phase, not six months into development.

Unison’s Jackson said PEO-MLB’s decision to do a technical assessment at this stage of the program is odd at best; at worst, it demonstrates there are real concerns with the program.

He said agencies usually do these types of technical assessments at the beginning stage of an acquisition strategy.

“It’s typically called out in the request for proposals that evaluates past performance, technical approach, and pricing volumes. Those are the three most common evaluation criteria,” he said. “The RFP usually explicitly says this is how we’re going to score it, and the technical assessment is routine. It is one of the reasons you compete major enterprise systems, so that the technical assessment can be done during the competition, and you uncover these foundational challenges before you make an award. As opposed to Navy, who is now, apparently 18 months into their solution, doing a technical assessment and uncovering profound structural problems with the low code approach for building a major enterprise system.”

As an example, Jackson said the Marine Corps Community Service Organization contract writing system effort put potential providers through the rigorous technical assessment as part of the competition.

Functional perspective of system development

Elliot Branch, a former deputy assistant secretary of the Navy for acquisition and procurement, who spent more than 30 years working for the Navy before retiring in 2019, disagreed with the idea that a technical assessment is done during the pre-award phase.

Branch, who is not a technologist, said the software space seems even more difficult to assess ahead of time.

“First of all, we’re really dealing with functionality and a roadmap to get to that functionality, unless you have an absolutely existing product. So with a software build, the key is always going to be implementation, and you cannot judge paper implementation with an assessment at that level of detail,” he said. “You actually have to have some kind of working product so you can go in and look at the integrity the architecture, the way it’s architected, the cybersecurity and other factors. This technical assessment really is more analogous to technical evaluation and operational evaluation in a weapons system.”

No matter the debate over the technical assessment, Branch said developing a system like this is always more complicated than first imagined.

“People really don’t understand the space from a functional perspective. Here’s the overarching issue: As I see it, you really have to make a trade between uniformity in contracting and the freedom to contract,” he said. “We think about contracting as a matter of clauses, and it’s really a great deal more than clauses. What these contracts embody, if you will, is a certain business logic between the government entity that’s contracting for supplies or services and the industry that they’re buying from. So if you want to get a standardized solution across a department, let alone the entire DoD, what you have to understand is that you’re going to give up some freedom of contracting, you’re going to give up some ability to express in the contract those business rules that that particular industry segment and the government have developed over years to get standardization.”

Branch, who retired before the Navy awarded the deal to CGI and well before the effort started to implement CON-IT, said he believes the Navy’s logic probably was the Air Force’s system presents the best opportunity to let each of the contracting activities, each of which has a very different set of buying missions with very different industry segments, preserve those business practices.

“I can see from a technologist perspective the concerns about the variety of the solutions under that one umbrella would create challenges in terms of building an application suite that had integrity,” he said.

Army, DLA also moving to new platform

The Navy isn’t alone in its struggles to move off of the DoD Standard Procurement System. The Army also failed with an earlier attempt to work with CGI, spending tens of millions of dollars before canceling the contract in 2021.

The Army seems to have righted its ship. In its most recent update from December, the Program Executive Office Enterprise Information Systems (PEO EIS) and the Office of the Deputy Assistant Secretary of the Army for Procurement (ODASA(P)) reported that the service has put 23 contract awards through the system, with total obligations of $41 million, and it had planned to train over 500 users as of January.

The Defense Logistics Agency also is implementing CON-IT. In a 2023 presentation, DLA said it was implementing an incremental pilot with more than 400 users to test the feasibility of the platform. The pilot successfully evaluated interfaces with five external systems like SAM.gov and the clause logic service, an authoritative, rules driven engine that determines the FAR and DFARS clauses for solicitations and contracts.

DLA too had some challenges with Appian’s platform, including “changes to support Procurement Integrated Enterprise Environment (PIEE) single sign-on (SSO) took longer than expected,” according to the 2023 presentation.

The Navy ePS alarm bells aren’t just being rung by the NAVWAR team. Lawmakers and the Government Accountability Office also highlighted concerns about the program.

In the fiscal 2023 Defense authorization bill, lawmakers specifically called out both the Army’s and the Navy’s efforts. While congressional concerns were focused on the old approach under CGI, which the Navy cancelled, some of the concerns remain.

“A commonality between these programs has been that their oversight and management has been run by procurement leadership teams, instead of information technology professionals, with the results clearly demonstrating the effects,” House lawmakers wrote in a report accompanying the bill. “The committee is concerned that in re-evaluating these programs, program managers have not considered technology-forward approaches, industry best practices, proven commercial-off-the-shelf solutions in use across the federal government, or innovative procurement processes, such as prize competitions. While the committee is encouraged that the services’ chief information officers are taking a larger role in technology-centric program acquisition, the committee believes more needs to be done to reach the aspirational end state.”

Trajectory of ePS unchanged?

GAO reported in June 2023 that the Navy CIO rated the program as “high risk,” and a Navy EPS official reported that the planned changes to the program’s baseline were related to greater technical complexity in development than anticipated.

Even with the Navy’s decision to retract the technical assessment, Unison’s Jackson said he hopes it will help change the trajectory of the ePS.

“What I hope will come from it is other organizations will adhere to the acquisition principles of competition, so that they do their own technical assessment as part of their competitions,” he said. “Then they will not find themselves 18 months or more into a solution and find that it’s not a mature enterprise solution and it may not be suitable for their enterprise.”

All of this information — from the warnings from Capitol Hill and GAO to the technical assessment team’s report — highlight the Navy’s potential and real challenges to modernizing its electronic procurement system.

“Modernizing the Navy’s procurement system is very complex. The majority of these solutions are outdated and out of compliance with security and audit requirements. Due to the number of applications and systems, we do not have standard business processes; therefore, developing a consolidated solution is a significant change to the procurement and contract writing community,” Rodriguez said. “Our previous attempt to modernize the DON’s contract writing system reinforced the need to perform periodic assessments. We need to identify issues early to self-assess and self-correct throughout the process. That is why we requested the March 2024 technical assessment. We also learned the implementation process and the solution itself must be flexible enough to meet the requirements of our stakeholders. The early involvement and continued engagement of our internal customers, end users and stakeholders in the development process is invaluable in ensuring the solution meets the DON’s requirements. We are leveraging the knowledge and work from those efforts to inform our strategy and ensure we avoid repeating the same lessons learned.”

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.