Using 3D simulations to train first responders

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

Commercial pilots have long used simulators, a form of augmented reality, to make their mistakes where no one gets hurt. Now, under the auspices of the National Institute of Standards and Technology, a coalition of organizations is posing a grant challenge to develop augmented reality for first responders. For details, Federal Drive with Tom Temin turned to Scott Ledgerwood, UI-UX lead in the public safety communications research division, and Alison Kahn, an electronics engineer in the advanced communications research group.

Interview transcript:

Tom Temin: And let’s begin with what it is you’re actually doing here. First of all, what are you trying to get at with this program?

Scott Ledgerwood: This challenge is really about enabling public safety to have better access to IoT information through a more usable interface. And the challenge is actually split up into two types of contest, one focused on the augmented reality community, testing them with creating heads up displays and holograms for first responders and incident commanders — and the other targeting data sciencetist and researchers in the IoT space.

Tom Temin: Got it. And Allison, what are the Internet of Things, IoT types of data that would need to be synthesized for first responders who usually use their eyes and their stethoscopes and their fire extinguishers?

Alison Kahn: That’s a great question. So what we’re looking at one of the primary data types that first responders are looking for is location. And right now, although we have GPS for outdoor location, indoor location is still very early days. So that is one area that we’re looking at. We’re also looking at the emergence of smart cities and smart buildings and the sensors associated with those to give first responders additional situational awareness that they currently don’t get with just their eyes and their teammates and their dispatch.

Tom Temin: And so what are the technical challenges to doing this?

Alison Kahn: Well, right now IoT data is very proprietary. So we’re not seeing a lot of integrated streams outside of a single company who develops multiple sensor streams. So we’re looking to this competition and the IoT contestants to create a sensor data landscape for the first responders to act upon. So they’re going to be developing sensor data streams that depict an emergency scenario.

Tom Temin: And then these could be at some point viewed on like those viewers people were when they’re playing games only what the first responders would see would be perhaps the way down the corner to where the emergency is.

Scott Ledgerwood: Yeah, so the main way of interacting with these heads up displays and holograms for the first responder applications is going to be through an augmented reality headset. The augmented reality devices up until late have been pretty limited as far as displays and filtering. You go. So it was really kind of hard to see how this technology would play a role for public safety. However, we’ve seen some really great enhancements to the hardware over recent years. And it really looks like industry is pushing R&D in this space. So I think there’s gonna be a high likelihood that AR will be used as part of the public safety operations in the future.

        Read more: Technology

Tom Temin: And it looks like these headsets in the displays can be incorporated into, say, firefighters helmets or whatever it is other types of first responders might have on their heads?

Scott Ledgerwood: Potentially. Yeah, I think that’s the hope, right. So in this challenge, we’re really trying to drive that initial content, and those initial applications to get that in front of the first responder community, and allow them to see the benefits and the potential impact of a technology like this. But I think long term, we really need to have a more user centered design approach for developing these different types of headsets and heads up displays, make sure that they’re integrated into the personal protective equipment and that they’re conducive to their their operations and then their environment.

Tom Temin: And who are you seeking to answer the challenge? What types of outfits?

Scott Ledgerwood: Yeah, so the nice thing about the project Challenge format is that we can tap pretty much anybody in the world. So we’ll be looking at industry and academia to play a role. But the prize challenge format also allows kind of mom and pop shops or university students, individuals to all participate in this type of challenge.

Tom Temin: And tell us about the programmatic side of this. Who are your partners in this challenge? Because it does have a dot com, I guess it’s called chariotchallenges.com, who’s involved with you?

Scott Ledgerwood: Yeah, so for this challenge, we have a number of co-sponsors that kind of have come on board and really help enable us to execute this challenge. So on the AR side of the house, we’ve teamed up with Magicaly and they’ll be coming in and providing some expertise and education to our developer community, and also be lending out some of the headsets for use in the end of the development and the final challenge aspect. We also have FirstNet built with AT&T and first responder network authority onboard helping highlight aspects of their network and again, providing feedback and education to our contestant base. MSA safety company will be coming in and providing some more perspective on developing first responder are developing technologies for the first responder, community and Blue Force Development on the IoT side of the house will come in and share some expertise about how to interact with IoT data.

Tom Temin: And Allison, what will you be getting in as responses pieces of software or will there be prototype, say integrated systems? What do you expect them to submit for the prizes?

        Want to stay up to date with the latest federal news and information from all your devices? Download the revamped Federal News Network app

Alison Kahn: The final submission will actually be a piece of equipment, a transmitter that actually transmits the sensor data. But we’re also asking contestants to help us develop the sensor data. How do they see it being portrayed in future smart city scenarios? The frequency, the accuracy, thedata types, we’re giving the IoT developers free rein to kind of imagine the emergency sensor data, how they see it in the future. So they will be developing the data and then utilizing small hardware to transmit that sensor data. And hopefully that will eventually in the final be a component that the AR developers can utilize to display mission critical tasks for the first responders.

Tom Temin: Let me just ask one basic question here, is the assumption that locales that would be visited by first responders already have sensor networks in place, but that the data that they produce is not being utilized to the most efficient or efficacious way?

Alison Kahn: That is our assumption here. I mean, I think we’re just starting to see IoT Data really evolved out of, you know, small home based systems and into larger scale buildings and citywide implementations. And so we’re kind of utilizing the chariot challenge to display kind of a future state where cities are well equipped with sensors, buildings are well equipped with sensors and first responders have access to that information but how are they going to view it.

Tom Temin: Who decides who gets what and what will the winners receive?

Scott Ledgerwood: Yeah, so on the judging side, we actually bring in some active first responder colleagues, and communication and technology specialists in the UI, UX and IoT space, and they’ll be participating witnessed and doing the review of all the different kinds of concepts that are submitted during this phase.

Tom Temin: And so the initial round then we’ll get some sort of have cash grants to continue working on their projects.

Scott Ledgerwood: Exactly. So the various phases we provide some funding to essentially provide resources to continue the development on this challenge. And then at the end of the event and the final phase, we’ll have a live competition where the IoT data in various augmented reality solutions will be used to conduct various tasks across four different scenarios and active shooter qualifier, a flood and a mass transit incident. And the winners of that final phase will walk away with $25,000 for first place down to $7500 for fifth place.

Tom Temin: Okay, so it sounds like it’s going to take you into next year at this point.

Scott Ledgerwood: We’re hoping for the live challenge to take place in early November at a large scale first responder training center and we’ll be able to simulate some of these large scale scenarios and actually allow the first responders to actually go through these various enviorments and conduct tests like they wouldn’t in the real world.

Tom Temin: Well, let’s hope we can have large gatherings by November-December timeframe. Scott Ledgerwood is the user interface-user experience (UI-UX) lead fror the Public Safety Communications Research Division at NIST. Thanks so much for joining me.

Scott Ledgerwood: Thank you, Tom. It’s been a pleasure.

Tom Temin: And Alison Kahn is an electronics engineer in the advanced communications research group at NIST. Thank you also.

Alison Kahn: Thank you again Tom.

Read more here.

Copyright © 2024 Federal News Network. All rights reserved. This website is not intended for users located within the European Economic Area.