This story has been updated to add a quote from a USDS spokesperson on Aug. 17 at 1:25 p.m.

Let me know if you’ve heard this one before: An agency hires “experts” to develop an application, spends tens of millions of dollars and the effort falls flat.

This easily could be the story to focus on with Small Business Administration’s Certify.SBA.gov project.

A recent agency inspector general report found the agency brought in U.S. Digital Service experts, spent upwards of $30 million over the last five years to develop the platform only for most of the effort to go to scrap and forcing SBA to basically start over again.

Instead this is a story about perseverance. It’s a story about lessons learned that every agency should keep in mind. And this is a story that offers an inside view into why federal projects do fail and how simple steps could change the direction of any IT project.

A quick background to start: SBA kicked off an 11-year, $45 million project to modernize how small businesses apply to be a part of socioeconomic programs like the 8(a) and women-owned small business initiatives. The agency tried this before in 2008 and spent $3.5 million before giving up six years later.

On this second try, SBA brought in USDS experts to run the program, oversee the development and implementation of the software and ensure success. While on the surface the partnership seemed to find success, the IG report and interviews with experts show just how the program went off the rails.

Here are four lessons agencies should heed from SBA’s experience.

Lesson No. 1: Get the requirements right from the outset or else the rest of the effort suffers

This seems simple, but for whatever reason SBA, like so many agencies, struggled to know what they were trying to accomplish. There has been, and continues to be, a lot of discussion about outcomes versus outputs when it comes to system development. But for an assortment of reasons this simple concept remains elusive to many in the technology sector.

“USDS didn’t think about the companies who would come back to SBA down the road to use the other services like loans or other assistance. They looked at it from myopic perspective of getting certified,” said a former SBA official, who requested anonymity in order to talk about this controversial program. “They didn’t think it would be good for SBA to know their customer journey, whether the business started off getting a certification, then went to the Mentor-Protégé program and then they got 7(a) loans. USDS didn’t ever understand that perspective.”

A government official with knowledge of SBA said the inability to define requirements has been a common problem at the agency for years.

Read more: Reporter’s Notebook

“Every one of those [past] efforts had problems and the only thing that is consistent is incomplete requirements. That makes it impossible to build a final solution and have it right,” said the official, who also requested anonymity because they didn’t get permission to speak to the press. “SBA finally shrunk the requirements down to just women-owned small businesses and it got that one thing right. But even these requirements changed through the launch, and that’s a program office issue.”

The official added the program office also dropped the ball because it didn’t give USDS a complete set of requirements from the beginning.

Steve Cooper, a former CIO at the Department of Homeland Security and the Commerce Department and who now is consulting at SBA, said the lack of an independent verification and validation from a third party also contributed to the challenges.

“The tough part of this was there was not direct involvement of the small businesses who have to go through the certification process, and USDS and the program office attempted to use proxies for customers and that apparently failed,” he said.

It was clear that SBA didn’t get the right decision makers in room that know the processes to guide the project and make sure the users are involved from the beginning.

Lesson No. 2: Mission modernization projects need to involve the CIO’s office

Multiple sources say from the start USDS didn’t, or more specifically wouldn’t, work with the CIO’s office and, in fact, made it clear that the team only answered to the Office of Management and Budget.

“They weren’t cooperating. They had a holier than thou attitude, were dismissive and demeaning. They were not inclusionary at all,” said the former SBA official. “We tried to get them to be part of the journey, but they were dismissive of any inclusionary efforts.”

The government official added only when the Certify.SBA.gov program got into trouble did USDS bring in the CIO’s office.

“It was a wrong approach and there was no adult supervision on top of it,” the official said. “Finally, through the governance process where the CIO and CFO offices conducted reviews last summer did the agency say enough as they didn’t see an end to the costs.”

“USDS is committed to delivering better government services to the American people, and appreciated the opportunity to collaborate with SBA on work to improve the customer experience for small businesses,” said a USDS spokesperson in an email to Federal News Network.

Read more: Agency Oversight news

A SBA spokeswoman declined to comment beyond what the agency wrote in response to the IG report. The agency didn’t comment on the USDS role or its efforts in the IG report. The agency, however, pointed out that Certify.SBA.gov has made the process better and the agency has seen a 65 percent increase in new participant applications since Certify’s 8(a) application
went into production.

“SBA is pleased that the report recognizes the fact that Certify has aided in
small business participation due to the simplified process of submitting applications,” the agency stated in the report.

Cooper said anytime a vendor or third-party like USDS decided not to or fails to collaborate with the CIO’s office, the program is in more danger of problems because it’s being siloed within the mission office.

“As a former federal CIO, I would argue that oversight should’ve been done by [the] CIO’s office but in this case, it wasn’t done by anybody,” he said.

Lesson No. 3: Buy before build; custom code only as a last resort

This continues to be a lesson too many agencies have to learn. SBA’s IG said the agency’s Business Technology Investment Council (BTIC) found in August 2019 that the Certify platform was “unsustainable on a long-term basis due to the cost of maintaining and updating the platform’s 35 mostly open-source software items and services. Additionally, program officials reported that Certify’s current design had unaddressed security vulnerabilities and was difficult to understand and improve. Furthermore, program officials found latent defects and data migration errors.”

Again, multiple sources say this was because USDS decided to custom-code the platform instead of using a commercial off-the-shelf application.

The former SBA official called the decision “astounding,” especially given USDS prided themselves on bringing in commercial best practices, and nowhere does it say custom coding is the first option.

“When USDS came onboard they did not look at other agency IT systems and data structure properly. For example the data structure in the Small Business Innovation Research (SBIR) program SBIR.gov platform has similar data structure to what would need to be collected for Certify.SBA.gov, and we asked them to stop recreating the wheel and synchronize with working agency IT systems and then customize only where they need to,” the official said. “Part of the problem was Certify was being sold as an end-all solution and not a tool in the tool kit. This is one of the tools SBA is doing to solve pain points from case management and workflow perspectives.”

Read more: Technology news

Another example of the build not buy decision by USDS came with the identity access and authentication application. The former official said the developers chose not to use the Login.gov service from the General Services Administration and instead developed their own approach, which ended up not working well.

“Anytime you build a system that complex where everything is custom coded, the tail to maintain it would be very expensive and difficult to protect from cyber perspective,” said the government official. “They should’ve looked for a software-as-a-service instead of custom coding.”

In September, SBA decided to move away from the custom coded platform and awarded a $3.5 million contract to move to a Microsoft Dynamics 365-based platform as part of SBA’s new enterprise customer relation management system initiative. SBA also implemented the Login.gov service from GSA earlier this summer as part of this and other development efforts.

Lesson No. 4: Mission goals remain so know when to change direction

After, as the former SBA official said, the agency “kicked out” USDS, and the CIO’s office took over the development, the move away from the custom code and use of acommercial platform let Certify find more success.

SBA launched “version 2” of the program in July for women-owned small businesses and had plans to continue development.

The former SBA official said the agency threw out upwards of 80% of the work from USDS and spent less than $10 million to get version 2 up and running in less than nine months.

“Now SBA has an architecture and its data flows on the back end are more robust and have more fidelity so now they just have to focus on the front end,” the former official said. “It’s really just a case management system because 75% of the fields are the same no matter what you apply for so you just needed a baseline set of capabilities you could standardize, and then you could customize the front end from there for the different programs. The platform is in a much better position now and going in the direction it needs to go.”

Source say current CIO Keith Bluestein decided in recent weeks to pause the work on Certify and decide on how to move forward with the entire effort.

Cooper said moving Certify forward means reviewing the current approach and deciding whether a low-code or no-code platform makes sense for future iterations.

“The OCIO recognized that they don’t want to do custom development or as little as they can. In addition, what you have is a significant portion of the system is workflow automation. Any of these platforms can automate workflow automation without any custom coding,” he said. “As a former federal CIO, what the CIO is now doing is taking what they’ve learned in the federal enterprise over the last several years and applying industry best practices to this Certify optimization effort. In listening to SBA OCIO team, there is clearly an understanding that earlier efforts did not use best practices and moving forward and paying attention to [the] IG report, they are moving in the right direction.”

Cooper added that this experience for SBA should be shared with other agencies. He said while failed projects are wasteful and frustrating, sharing the mistakes and missteps will help others ensure they don’t step in the same potholes.

“The reason why this keeps on happening is agencies still operate within own agency boundaries and don’t consciously take the time to share what they’ve learned to help avoid other mistakes,” he said. “The federal government writ large blames people and when you do that, the learning process doesn’t happen easily. People are scared to come forward so others can learn from their experiences, especially if you think sharing will mess up your career or get you fired.”

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.