The U.S. Citizenship and Immigration Services has gotten the reputation of being ahead of other agencies when it comes to IT development using agile processes.
USCIS is adding to its lead by testing out the next phase of agile development. Mark Schwartz, the USCIS chief information officer, said he’s implementing something called impact mapping. USCIS recently tested out this concept on its case verification system under the E-Verify program.
“In this impact mapping exercise, they start with a goal of , say increasing the 70 cases a day a verifier can do to a higher number so they can scale. They draw a sort of a mind map of different ways they can accomplish that goal,” Schwartz said at the Aug. 18 ATARC Federal DevOps Summit in Washington. “So you have an empowered team and they map out, with that as a goal, here are the behaviors that would need to change in the agency and those who have those behaviors. Then they map out for each of those behavior changes, what IT capabilities could be offered that might accomplish those behavior changes. Then they take each of those capabilities, think about it and say ‘if we were to provide this capability, we think it would increase the number of cases a day by X amount, whatever the X is.’ They really don’t know. It’s just a hypothesis. They think if they do ‘this’ it will increase it by this much and if they do this other thing, it will increase it by ‘this’ much. This is all at the level of capabilities. There are no requirements here.”
The development team submits their ideas to the governance board, which is made up of business owners, IT executives and other interdisciplinary leaders for approval and funding. Once the team gets the go-ahead, Schwartz said the agile sprint begins.
“The team starts working. They start developing software. On the team, they have programmers who sit down with some users of the system and they work together. As the programmers finish each of the pieces of functionality, they type ‘commit’ and when they do that, the work they have done gets automatically tested through a whole series of automated tests, and automatically gets deployed into the cloud, infrastructure provisioned immediately,” he said. “So in the first hour of the developers working, they are already deploying to a production environment in the cloud and it miraculously appeared. As they do that, people can start using it. The users start playing around with it. They give feedback. The programmers make changes and every time they make an adjustment and hit the commit button, automatic tests are run.”
Schwartz said the automated tests include regression tests to make sure the new code doesn’t break anything in the existing system. He said USCIS also automatically runs security tests and Section 508 compliance tests as well as code quality scans to validate the adherence to standards, complexity and measures and code coverage.
“It then automatically deploys the change to production as many times a day as the developers feel like it’s appropriate to deploy,” he said.
The key to impact mapping is two-fold: The sprints are short and focus on user-driven outcomes, not requirements, and the governance oversight comes from the users and executives across multiple disciplines.
Schwartz said because changes are made on the fly based on user experience, if something isn’t working, the developers can change or stop, and if something is working, it gets deployed to the community quickly so it brings in the idea of continuous improvements.
Strangler approach to IT modernization
The test of impact mapping for one part of E-Verify produced results immediately. On a program where USCIS spent the previous four years just developing and documenting requirements, through impact mapping and using dev/ops methodology, the agency delivered its first update to the system in six weeks.
Schwartz said to find the money to modernize these systems, he continues to use the “strangler” approach. This is when USCIS takes a little piece of a system and breaks it off, redevelops it and makes it interoperable with the legacy system. Then Schwartz’s staff break off other pieces of the legacy system, modernizes them and slowly moves the entire system to the cloud or new infrastructure and retires the legacy hardware and software.
Schwartz said he believes impact mapping could be the future of governing dev/ops projects.
“The reason why it works really well is it’s empowering. It gives oversight a lot of control over the projects. It’s extremely lean and efficient. And it also works because dev/ops lets us see results immediately. The cycle time from when we try out a hypothesis to when we get the information as to whether the hypothesis works is tiny,” he said. “We can take these multi-billion dollar programs that take four years to write requirements for and instead start delivering value in six weeks, and use that value delivery as a way to control the program.”
While USCIS is ready to move to impact mapping because of its maturation in implementing agile development, other agencies are just at the beginning stage of using agile.
NRC pilots agile on two projects
At the Nuclear Regulatory Commission, Cris Brown, NRC’s master data management program manager, said her agency almost was forced into moving to an agile development process after a project struggled.
“We received a report that two major pieces were going to be delayed for four and seven months. We couldn’t live with that delay, so I actually reached out to our friends at 18F,” she said. “We actually did two dev/ops pilots and we were very successful in moving projects forward. As a result of that, we realized that the dev/ops needed to be a part of the culture and we are working to codify the use of dev/ops through an internal project management tool that we have so we can continually have that reference. It hasn’t been easy. It’s a huge culture change that we have just started.”
Brown said the goal of codifying dev/ops through the internal project management tool is to make it easier for others in NRC to repeat the process.
Then there is the Environmental Protection Agency. Ann Dunkin, the CIO, said her agency’s move to dev/ops is focused on letting others, such as 18F with its cloud.gov platform, do what she called “the heavy lifting on the development side.” This lets her staff focus on the business side to ensure continuous improvements.
EPA is working on a new contract for agile services. It held an industry day earlier this month, and a final solicitation could be out as early as September.
Like so many technology evolutions in the past, the challenge with agile is culture change — people and processes.
Duncan said agencies were fooled into thinking the old “waterfall” approach was low risk. But because there is really no way anyone knows what they want in the first place and then it rarely gets built exactly that way, it turns out waterfall was much more high risk.
She said agile requires a new way of thinking.
“There is no solution without risk, but we and a lot of people believe that it reduces risk by constantly interacting. You’re constantly interacting with your vendor or development team, depending on how you are doing the work. You are constantly able to change who is doing the work for you,” Dunkin said. “We are talking about, in our agile vehicle, buying in relatively short groups of sprints so we might buy three months worth of work, for example. If that vendor is not performing for me, I’m changing them out and using another vendor. Because I’m the product manager internally, I don’t lose anything by swapping out my vendors. So having that control and that daily visibility of what’s going on — my EPA employees are in the meeting every day in the morning to understand going on every day. So that level of visibility reduces risks tremendously. It doesn’t eliminate it completely, but I think you are taking on differently kinds of risks than you did before.”
Federal IT workforce needs new skills
This also means the workforce has to have the right skills. Dunkin said on one project, EPA brought in 18F. Then after the initial sprint, it brought in a new vendor. But because EPA acted as its own systems integrator, Dunkin said it understood the goals, the development plan and has been a part of every conversation since the beginning and they haven’t missed a beat.
The agile effort also must include all stakeholders from the business folks to the security to acquisition to human resources.
NRC’s Brown said the goal is to empower the staff running the program and ensure there is plenty of communication across all parts of the team and agency. This means there are a lot of the soft skills that the interdisciplinary team needs starting with communication and collaboration.
Schwartz added a good developer isn’t someone who just develops, but someone who understands and can do security or accessibility or other types of testing.