Technology

In-person exclusive with the Test & Evaluation director at the Transportation Security Laboratory in Atlantic City

Luggage and passenger screening is a complicated applied science. An idea has to be verified before it can be built into prototype equipment for testing and eve...

Tom Temin@tteminWFED

February 15, 2023 3:36 pm

9 min read

Luggage and passenger screening is a complicated applied science. An idea has to be verified before it can be built into prototype equipment for testing and eventual production. For instance, in an airport, determining whether the basic technology can reliably detect what it is supposed to, such as a dangerous chemical. That is the function of the Independent Test and Evaluation Division of the Transportation Security Laboratory or TSL. The lab is operated by the Science and Technology Directorate of the Homeland Security Department. In the second of this week’s series of four interviews on location at TSL in Atlantic City, the Federal Drive with Tom Temin spoke with the Test and Evaluation Director Lee Spanier.

Interview transcript:

Lee Spanier
My division performs the final exam, that supports the decision-making for acquisition and deployment of TSA. And systems come in for certification testing and we conduct that test.

Tom Temin
I guess, there is a gap, sometimes, between what sounds like a really great scientific idea, for purposes of passenger and cargo and luggage screening. But it doesn’t always translate into the scale and reliability that’s needed in operation. And so it sounds like, you determine, your group determines, yes, this is good to go. TSA, you can buy it and rely on it.

        Join us Apr. 25 at 1 p.m. EST for Federal News Network's StateRAMP Exchange where we'll explore how the StateRAMP program provides cyber assurances as state agencies continue their IT modernization journeys. | Register today!

Lee Spanier
Well, let me clarify, I don’t do the what’s called, the operational suitability assessment. I do the conformity assessment for detection and only detection. So that’s the primary objective of the system, it’s a screening system. So we measure its detection capability. Not any of the other attributes, such as reliability, maintainability, whether the UPS works.

Tom Temin
Got it. All right. So give us some examples, of things that need to be detected and how you find out that if they can be reliably detected.

Lee Spanier
Well, just as a matter of practice, TSA defines the detection standard. That detection standard defines, exactly, what they expect the system to detect. And what we do is, we take that detection standard and convert it into something called, the challenge matrix, that lists out each of the scenarios. Whether the target is in the bag or on the body or in a bottle or in any other circumstances. And those scenarios, are presented back to TSA, they then authenticate that as an accurate portrayal, of their detection standard. And once they’ve given us, the go ahead, we turn that around and make that into a full test. And so a typical test could have thousands of those scenarios.

Tom Temin
Give us an example.

Lee Spanier
Well, an object concealed from view on your body, perhaps, on your chest, on your back, on your ankle. Those would be examples of a target concealed on your body.

Tom Temin
Right. So you need to verify then, that is detectable and can be detected.

Lee Spanier
Yes, or in bags. That could be the object is concealed in a ski boot.

Tom Temin
And does the mission extend to knowing, what it is that is there? Or just that something is there, that needs further screening?

        Read more: Technology

Lee Spanier
Well, it depends on the system and the [Concept of Operations (CONOPS)]. So I can’t give you all the details, because I don’t know exactly how TSA does it. But I can tell you that, for an AIT system, the passenger screening.

Tom Temin
That’s the Advanced Imaging Technology.

Lee Spanier
Thank you very much. It uses that as an anomaly. It is not looking to sort threat items from benign items, it is looking for items that are anomalous to your body. And so it requires a pat down for resolution.

Tom Temin
Well, what is left to determine? Because it seems like for years, they’ve been able to tell if there’s something in your pocket, or in your breast pocket, or in your back pocket, or in your shoe. So what are some of the challenges you’re looking at?

Lee Spanier
Well, the challenge is that the TSA, remember, I’m just downstream, I do the quality assurance piece of this. The developers, are the manufacturers and they are responding to, both our headquarters, science and technology and TSA is sponsorship of their systems. The requirements that I can talk about that are changing, are seeking to maintain that detection capability, but lower the false alarm rate. The false alarm rate, is the rate, in which resolution is required for by hand pat down. So every time there’s an alarm, that requires a pat down touch rate and effect. Ok, so right now, one of the challenges, is to reduce that touch rate.

Tom Temin
And that has been reduced, significantly, too hasn’t it? Over the last several years.

Lee Spanier
It has been shrinking, of the latest technology, which is being trialed deployed. It has a ,very significant, reduction in that false alarm rate.

        Want to stay up to date with the latest federal news and information from all your devices? Download the revamped Federal News Network app

Tom Temin
Right. And what are some of the technologies that can reduce it? If something’s in your pocket, it’s in your pocket. What would cause a false positive, say of, something not in your pocket, but showing up in your pocket?

Lee Spanier
The technology that is changing, as far as I understand, for these baseline systems, which are in millimeter wave, are based upon advanced image processing and machine learning. So many of those techniques are being applied and with great effect. The same can be said for the computer tomography systems, whose reduction in false alarm.

Tom Temin
And so your job is to say, yes, this really is the case, because we’ve independently tested that system, that someone else has said, yes, this reduces false positives. Because you don’t develop the false positive detection system or the avoiding false positive. You just, independently, verify that it works?

Lee Spanier
Yeah. So the vendor will come in and claim that they’ve met the standards and specifications that TSA has defined. And we will evaluate those claims. All right. And just to be clear, I defined target scenario. So that’s one set of tests for detection. But the other half of the coin, is false alarm. And we measure the rate, when alarms are occurring, when in the stream of commerce or a simulated stream of commerce, where there are or should not be any targets detected.

Tom Temin
Got it. And so what form does your work take? And what kind of equipment gear and skills are needed, to be able to test these things?

Lee Spanier
Well, it depends on the test service lines, I’ve got seven different test service lines. So in general, most of what we do is, create targets, concealed targets. We’ll acquire bags and put in the proper clutter mix, in those bags and store those bags for use in testing. For passenger screening, we will acquire temporary hire individuals, we’ll train them up, there’s a series of qualifications that we obtained for them, then they are on call for testing. So those are the examples, of the types of material and personnel that we use. There’s hundreds of mock passengers that we use for our tests.

Tom Temin
Yeah, so you have to develop the test methodologies then, too, right?

Lee Spanier
Yeah, of course.

Tom Temin
So that’s using the equipment that’s being considered. Don’t you need that piece of equipment, to be able to test that?

Lee Spanier
Of course. So let me back up. So when TSA issues a solicitation, the very first step is the vendor, the manufacturer will submit a certification data package, that provides evidence of compliance. And there’s some prerequisite things, certainly, electrical safety, we don’t evaluate. So we require on third party. There third party certifications, as a prerequisite to us. We’ll do an audit of that certification data package. And if it looks like they have sufficient evidence, then we’ll welcome them in, they’ll install one or more of those units, in our laboratory. We’ll conduct a Test Readiness Review. We’ll get all the preparations together and execute a test and it can be as little as, one to two weeks. Or, in some cases, it can take six to eight weeks to conduct test.

Tom Temin
And you send them away disappointed, sometimes? Does this is the stuff, generally, work when it gets to you?

Lee Spanier
It’s interesting. I would say there are very few manufacturers, that passed on their first trial. All right, required a couple of trials. I think, there might have been one vendor, that passed on its first attempt.

Tom Temin
So you’re like, the Fruit of the Loom guy. It doesn’t reduce the false positives. So Lee Spanier says it reduces false positives.

Lee Spanier
Well, I’m not the authority, I just conduct a test. And just to be clear, I don’t issue the certification letter. I execute the test and make the recommendations to TSA, they issue the letter got it.

Tom Temin
All right. And just the people that are in your group, what are the types of skills or education requirements? What types of people do this work? I don’t mean the test subjects, but the people designing the tests.

Lee Spanier
That’s an important question that you asked, because they’re physicists, they’re engineers and they’re chemists. Because of the range of test equipment that we’re evaluating. So for example, chemists would be involved, in the Raman scattering spectroscopy, for bottle liquid scanners. Trace chemists are also involved, in the explosive trace detectors, which are IMS, ion mobility spectrometers. Physicists and engineers are involved in, CT scanners, X-ray based computed tomography and millimeter wave AIT scanners.

Tom Temin
So a lot of technologies come to bear here.

Lee Spanier
Yes, certainly.

Tom Temin
What else do we need to know?

Lee Spanier
We do other things for other customers. One of them is, we do lots of special excursion tests, things that pop up. Individuals that say, hey, how come I’m getting patted down all the time? Why is it? Why am I alarming? I don’t have any targets on me. And they help us investigate that. We lead [Institute of Electrical and Electronic Engineers (IEEE)], image quality standards committees. So about half of these, are image quality systems or imaging systems. And we’ve come up with techniques, to monitor the quality control of those images. We also, work very closely with other nations that want to do the same type of certification processes. So the European Union and other countries in Eastern Asia, would like to stand up their own certification processes. So we do something, which TSA refers to as, international harmonization. Harmonizing standards and harmonizing test methods.