The use of agile or dev/ops for IT software development is no longer just in the pilot stage at the Homeland Security Department. So much so that the Citizen and Immigration Services put out a major solicitation in April calling for the widespread use of agile development processes across a broad range of enterprise support services.
CIS issued a request for quotes (RFQ) under the EAGLE II contract April 23 called the Support for Platform Engineering and DevOps Integration (SPEDI). It’s one of the largest and broadest support services contract to call for the use of agile development processes. CIS says in its acquisition forecast that SPEDI is worth more than $100 million.
Luke McCormack, the DHS chief information officer, said CIS is one of the more mature organizations using agile. The reason why CIS is out in front is one many agencies can understand.
“When you strain a grape you get the best wine, so sometimes when you push down on your top line, the innovation comes out. Right, the MacGyver comes out in all of us and you start to break through some barriers because you are forced to do that in some cases,” McCormack said Thursday during the AFCEA Bethesda, Md. chapter’s Law Enforcement IT Day. “I think that is a good thing because what that does is invite and entice innovation, and I think that is important. We see that now and all of us are beginning to recognize that and take advantage of that, which I think is very important to deliver on these real time needs that continue to emerge.”
In many ways, CIS’s move to agile came from failure and opportunity.
This big RFP builds on that innovation that emerged over the last two or more years because of budget pressures and changing technology.
CIS in the RFQ calls for the vendor to support the migration of its application development methodology from traditional waterfall to agile framework.
CIS says these frameworks “require cloud services that are virtualized, scalable and cost effective and utilizing dev/ops practice which emphasizes collaborative teaming between all disciplines involved in the cradle-to-grave software lifecycle,” according to the RFQ obtained by Federal News Radio.
Among the functional areas the RFQ calls for are:
Enterprise platform and cloud integration services
Enterprise project management services
Enterprise infrastructure implementation
Information security support services
Enterprise operations and maintenance
CIS says it’s in the process of “adopting agile practices for not only software development but all components of the value stream of delivering services.” CIS says “the core to implementing this is the concept of continuous delivery in which automation is used to ensure the ability to recreate an environment at all times from a configuration management database. This includes both the initial provisioning and ongoing management of the environment, and the same automation processes need to work for both development/test and production in order to reduce errors and insure consistency across all environments.”
CIS just extended the due date for this RFP to June 16.
SAIC holds the current contract and CIS extended that in January through June 27 at a rate of $26 million. CIS likely will have to extended it again as the award will not be made by the end of June when the current contract extension expires.
CIS’s move to continuous delivery is something that is spreading across nearly all of DHS.
Understand how Healthcare.gov was fixed
McCormack said DHS saw the change that was coming after the well-documented problems with IT projects whether it was the FBI’s Sentinel case management program or Healthcare.gov.
“If you haven’t spent the time to understand what happened in the health care situation, right healthcare dot gov, and more importantly what fixed it, it’s worth paying attention to that and understanding that,” he said. “If you weren’t paying attention to the fantastic delivery that one of my esteemed colleagues put together on Sentinel, which is the case management system at the FBI, then you really should because what you are seeing across our community is we are moving out of these rescue situations and into more of a real-time delivery mode.”
Every DHS component has faced those rescue operations. Whether it was Customs and Border Protection and the Immigration and Customs Enforcement TECS program or CIS’s transformation effort or TSA’s technology infrastructure program, DHS, like every agency, has faced the fact that the waterfall approach to projects no longer works.
McCormack said the change in how technology is used and delivered, such as cloud and mobility, as well as how back office infrastructure IT have become commodities have driven DHS toward agile.
“We are taking tools and techniques that we’ve learned and discovered, some of us early adopters and some of evolving to, and are being able to deliver services in what used to take years into months, and literally delivering services from months to weeks, and in some cases days,” he said. “When I’m talking about services, I’m not talking about firing up a website. I’m talking about delivering way up the value chain in regards to business applications that all of these communities need in order to do their roles.”
Agencies today are learning more and more how to assess the benefits and value of these services to deliver in near real-time, McCormack said.
Creating confidence in IT delivery
The idea of agile development isn’t new. In the late 1990s, the trend was called modular contracting. But McCormack and others say what’s different now is a combination of factors, including technology, budget cuts, and maybe most significantly, real proof that this approach works well.
The FBI’s problems with Sentinel were well-documented, and it wasn’t until the bureau moved to an agile framework that it got Sentinel on the right track. And then, the rest of the government took notice.
Jeffrey Johnson was the FBI’s chief technology officer until he went on a year- long detail to the Justice Department recently. Johnson led Sentinel’s move to agile development.
“Now that Sentinel is in regular monthly releases to production, the conversation has fundamentally shifted. We are not talking about how to get all of my requirements, all of my wishes as a stakeholder, everything that I could possibly want or think I might want for the next five years in this requirement so I will not allow for the release until it’s all there and it’s all ready because I will not get anything else,” he said. “What we are saying is, ‘hey, I’m going to give you something this month, is that good enough? It makes it better. Great. Ok, then we will talk next month and if we can make it better, we will make it better again.’ Well now the stakeholder is much more willing to accept something that is simply making their lives a little bit better, enhances the mission just a little bit because they know there is an opportunity next month to do the same thing again.”
Johnson said the use of iterative development under agile reduces the risk all the way around. The FBI also is getting capabilities into the field more quickly, and this approach is creating a more collaborative environment between IT and mission owners.
One of the biggest challenges with agile is the federal contracting process is not set up to take on these types of approaches.
CIS, for example, created its own contracting vehicle where multiple teams are working on complementary projects that are focused on the end result. But CIS has said it took nearly two years to come up with that model. McCormack said DHS is working to address this issue.
“We still are sort of cooking some of that. We’ve learned a lot recently,” he said. “We recognize we need to structure these contracts so we can do things like fail fast. We recognize that as we start to look at this continuous delivery model there is a different kind of relationship with the operational aspects of delivery system in a continuous delivery model. So that’s an area that is being examined right now to look at the best way to strengthen that capability in this ongoing model.”
McCormack said he recently met with the operational side of DHS and they expressed concerns about the change.
“You will have a bigger role than you ever had before because we are relying on you to give us that end-to-end visibility,” said McCormack in describing a recent discussion with the operational folks.
McCormack said the mission owners become more important because of how quickly the development happens, if they are happy with the capability or not, the application developers know about it right away.