Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

We take them for granted now – robotic vehicles operating in emergency or disaster situations. Reliance on robots didn’t just happen. Among other things, it took the establishment of accurate and repeatable testing methodologies, to know for sure if a robot could traverse a rocky slope or a smoky building. For his work at developing advanced robotics testing, Adam Jacoff is a finalist in this year’s Service to America Medals program. He’s the robotics research engineer in the intelligent systems division at the National Institute of Standards and Technology, and he spoke to Federal Drive with Tom Temin.

Interview transcript:

Tom Temin: Mr. Jacoff, good to have you on.

Adam Jacoff: Hi, how are you?

Tom Temin: Alright, so tell us about these robots. These are the kinds of wheeled vehicles that might sniff around a suspected bomb, or tell us the range of what they normally do nowadays.

Adam Jacoff: Well, those are the types of robots you would have seen maybe on the news, if you’re thinking about bomb squads approaching improvised explosive devices and other things trying to disarm them. But you need to think a little bit broader because robotics is generally any smart systems that are either ground based or underwater. Or actually, you’re seeing a lot of drones flying around the airspace these days. Those drones are very smart, robotic systems in and of themselves, some of the smartest I get to work with, and they have their own application space for emergency response and others.

Tom Temin: Alright. And manufacturers sell these things, I guess, to government agencies at the federal state and local level. They’re not tested when they’re sold, or how does the testing that you develop come into the whole system here?

Adam Jacoff: Yeah. So when we started this, this was literally, almost 20 years ago, there was really no measurement science going on for how to evaluate such systems. And there were no standards involved, no standard test methods that would help people quantify the performance. So we stepped into that void, we just started filling it with at first ground robot tests, terrains and dexterity tasks and laser mapping kind of tasks, in an effort to help the manufacturers and to inspire the researchers toward answering the needs of emergency responders. They may have been buying equipment, but they really didn’t know exactly what that equipment was going to do. And invariably, that early equipment was not ready for primetime, they were just not capable of doing the things that emergency responders needed. And so the standard test methods became sort of a tangible language that all these very disparate communities could use. I mean, you can literally watch a robot go across a physical terrain to a statistically significant level of repetition so that you have some confidence that really does work before purchasing. And then ultimately, people started using the test to specify their purchases. So you need a robot that does, X, Y, and Z — well not every robot does all three well, and you need to start revising your expectations based on what the commercial market can provide. But in the meantime, although the commercial market wasn’t really working in its most efficient self, that was because emergency responders didn’t know how to ask pointed questions. So our test methods really enabled them to understand what the state of the science really looked like, what the good robotics could do if they had it. So they could ask for it directly, and even demand it from the manufacturers.

Tom Temin: So it sounds like you were able to establish testing methodologies that the industry basically said, fine we’ll adopt these, you didn’t have too much pushback from industry or maybe you did in the beginning?

Adam Jacoff: No, good question. It has been interesting. So I was a design and build robotics engineer at NIST for years before getting into this idea of developing test methods. Once removed from the design, itself, but our approach was really holistic in that we were not trying to hurt anybody with these tests, we were just trying to measure, measure, measure, and to do so reliably and reproducibly. So everyone can measure themselves, actually. These test methods are very, very simple to look at. They’re very, very easy to build and low cost. And that’s kind of the whole idea. And when you put a suite of tests together to try to quantify the entirety of the system, whoever designed that robot might have had abject failure first time in I mean, we saw a lot of abject failure first time in, but they’re all engineers. And they all said, Oh, that’s what you mean by advanced mobility. Okay, you know, advanced mobility was in the name of their robot, except that robot ended up on its back as soon as it met something akin to a rubble pile like terrain, or even less. So they went away with their lessons, maybe with their tail between your legs and their lessons, and they came back stronger, more reliable, and with new thinking on the idea of what does it mean to be really mobile in really unstructured environments.

        Read more: Technology

Tom Temin: We’re speaking with Adam Jacoff. He’s a robotics research engineer in the intelligence Systems Division at NIST and he’s also a finalist in this year’s Service to America Medals Program. And describe some of the setups that you had to do to test robots. You mentioned, say gravelly hill or rocks in the way. at NIST headquarters. There’s lots of open land that just set up test beds. I mean, how did you do this?

Adam Jacoff: So everyone’s first thought at this game always lands on operational scenario. So something that is true and real, and physical and representative of the mission they have in mind. But we stepped one further back to the notion of real baseline capabilities, evaluation. So like, literally the abstracted version of whatever operational scenario you have in mind. So let’s just take bomb squads, for example, right, bomb squad, sir, were early adopters of our tests and use them widely. And that’s pretty much why all this has happened. Because worldwide, there’s a problem. And people are in harm’s way. And they ought not be because the state of that technology is such that they can do what they need to do remotely, as long as the robots are holding up there and of that technology burden. So with bomb squads, you might have a package size device. And we generated some test methods that are very simple to replicate. Everyone builds it out of wood, and PVC pipes, and buckets. So if you look at our website, it’s at robot test methods, all one word at NIST.gov, you’ll see what looks like you know, a big box store kind of purchase list and very simple instructions. But what was inventive about it was the idea that it gets at the nature of the problem, which is if it’s mobility, you see terrains made out of lumber put together in such a way, or you see directed inspection tasks made out of omnidirectional pipes. And that grew into omnidirectional buckets for aerial systems and drones. And you’re going to see very soon because these things are kind of getting swept across the country and internationally. These are the basic tests to establish your remote pilot proficiency to know that you’re safe operating in the national airspace. So the basic tests forces you to do certain moves in sequence, but it really rewards you when you get there because you know, you’re in the right spot, because you’re looking at a target in a recessed bucket. And you know, all you need to know about where you’re supposed to be. And then you just need to move on to the next step and the next stop. And when you add enough of these moves together in any type of robot, round area, or aquatic, you are starting to measure yourself quantitatively. And that’s the whole idea. So they’re really abstract tests, but they’re the basics. They’re the calisthenics that you would do before scrimmaging. On your sports team. The scrimmage is operational training. But there’s something you do as a team before that, which are the perfunctory strengthening muscle memory kind of tests that every athlete knows.

Tom Temin: You must have developed quite a data set that could be reusable for training these types of systems.

Adam Jacoff: Yeah, so when we’re going through these evaluations, you know, they’re very public. So all the manufacturers involved, I founded a robot competition in like 2000/2001, with a Japanese colleague that has annual every year, all these test methods set up for anyone who wants to try their robots, and it’s an international competition. So the data starts flowing from repeated trials. And you know, now these competitions turn out like 400 trials in three, four days, because it’s just massively concurrent. So that data ends up informing procurement decision. Now, sometimes it’s an operator proficiency evaluation, if you already own your robot, but if it’s a brand new robot, and no one is good at operating it, we go to the manufacturer themselves to set the bar at the 100th level of proficiency. So that 100th percentile is any operator that makes that robot dance. And then you and I can both get on that same robot in that same test method and measure ourselves against that expert, operator or pilot.

Tom Temin: Sounds like a lot of fun this work?

Adam Jacoff: Yeah, if you stood on the sidelines and looked in, you’d say that guy is having a good time. It is not easy work. It is long and grueling at times. But ultimately when you see the payoff, it has its own inspiring effect. Like the competition’s the test method validation exercises, the interactions with the emergency responders who truly need the help. I mean, these people are other heroes. And they’re just need the State of the Science working for them to do their jobs a little safer, keep them out of harm’s way, while they’re trying to work for us to protect us to rescue us. Bringing all those groups together around a tangible physical test apparatus and getting to use that test apparatus to sort of guide and focus research. is extremely gratifying and so it has its own benefits every day.

Tom Temin: Adam Jacoff is robotics engineer in the intelligence Systems Division at the National Institute of Standards and Technology. He’s a finalist in this year’s Service to America Medals program. Thanks so much for joining me.

        Want to stay up to date with the latest federal news and information from all your devices? Download the revamped Federal News Network app

Adam Jacoff: A pleasure, thanks for having me.

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.