The U.S. Agency for International Development collects data from employees, non-profits and contractors in 85 countries around the world.
And until now, USAID didn’t have a way to easily share, process and understand information from a global perspective. That means it’s not easy for agency employees working on malaria, for example, in Africa to communicate with colleagues working on a similar issue in Cambodia.
It also means that researchers and analysts back in Washington struggle to connect the dots on a global perspective and marshal resources to deal with crises.
This big data challenge is about to get a little easier. In the coming months, USAID will launch a development data library to more easily capture, share and visualize all of this information coming from around the globe.
Insight by Carahsoft: Explore use cases for edge computing and approaches for taking advantage of it by downloading this exclusive e-book.
“We’re making sure as an agency we invest in bringing that data back home and making it available to the public as a whole, making it available to our own staff so that they have the data at their fingertips to conduct daily operations,” said Brandon Pustejovsky, USAID’s chief data officer, in an interview with Federal News Radio. “The idea here is instead of just sharing on a point-to-point basis, which creates more siloes of information, let’s share this more globally to make sure that not only USAID can benefit from that in support of its mission, but even the countries in which we work can better understand what’s going on in their own environments, and be able to capture that to improve their way of life, to improve their democracy and the freedoms they may be striving for.”
Pustejovsky said currently data is shared through USB sticks or email, or between the mission area and the contractor, but not on an agencywide basis.
Additionally, USAID employees, contractors and grantees work in remote locations where heat, cold, dust or the lack of electricity may be major challenges to the sharing of data.
Pustejovsky said the goal is to capture data in a central place on a regular basis. USAID even modified all of its contract and grantee awards starting in 2014 requiring vendors and non-profits to send the data to the library on a regular basis.
He said the fact that USAID relies on so many outside experts and organizations, the ability to collect, share and analyze data is much more important as the amount and quality of information have increased over the last decade.
“We need the collective brain trust of the countries, the individuals and the civil society organizations that experience these challenges on a day-to-day basis. But we have to make sure our mission is also motivated and advanced with evidence, and as the agency that is the lead agency on this particular mission, that evidence is contained in the data,” Pustejovsky said. “The best way we can truly demonstrate the characteristics of partnership is to share the data. What is true partnership without sharing?”
Pustejovsky said USAID is taking an iterative approach to developing the data library over the last year. He said the agency has been doing some user testing in late 2016 and it will broaden the feedback effort into 2017.
“We want to make sure we cater to those audiences in a way that makes life easy for everyone, but also makes the information and data we provide is rigorous for research purposes,” he said. “We’re reaching the end of the development cycle right now, and we are starting to move into a more rigorous look at internal workflow and trying to automate the way we provide approvals for releasing data to make things easier on our staff. We also want to make sure the interface for the public user really makes sense for them. We think we are on a glide path to launching this in early-to-mid 2017 timeframe.”
In the meantime, USAID created a basic portal, listing existing data sets and an easier way to upload information, as a stop gap measure until the library is ready.
Pustejovsky said the data library is part of a broader strategy to do data differently. He said in the next three-to-five years, he expects USAID to transform its processes and approaches to data management.
“The challenge we have as an agency that has been around over 50 years is that data has been managed in different ways, and in some cases for specific sectors, the jury may still be out on how you gather data in very nuanced ways in specific disciplines, and we understand that. But as we begin to move forward now what we are saying now is in terms of gathering data at the field level on a day-to-day basis to meet our goals, objectives and to make sure we have accurate measurements against how we are doing business, we do need to get that in the door in a standardized way,” he said. “We have done some pilots of similar systems, a dozen or so around the world, to start looking at that standardization both in terms of the data and the business processes. We’ve seen improvements and efficiencies. It’s much better to have our foreign service staff, for example, go from mission A to mission B and know they can use the same system and deal with the data in the same way rather than have them learn an entirely new system all over again when moving from one place to another.”
Pustejovsky said USAID already has seen benefits from this big data effort, including improved data quality.
He said during crises from the earthquake in Katmandu, Nepal to the response to the Ebola outbreak in West Africa, the sharing of data helped the responders do their jobs better.
“We sent a data scientist with the team responding to West Africa to make sure we could harness that data, a lot of which was coming in through mobile applications that gave us some sense of where are the symptoms of the disease manifesting themselves? What do medical records look like for certain cases that may have presented themselves? Are there medication stock outs that need to be addressed? What are the capabilities of various clinics? All that information was coming in via mobile phones and coming in from multiple sources,” Pustejovsky said. “By mapping the mobile phone coverage and obtaining the data based on that understanding of the coverage gave us a much clearer picture in a more real-time way of how the disease or outbreak was manifesting itself in that area. Those are the kinds of things we want to start mainstreaming into our broader operations by taking them from the cutting edge side and then moving that into the way we do business as an agency.”