Stephanie Fussell is one of the leading minds at the intersection of science, technology and aircraft, working for the past decade to better understand and provide guidance on how extended reality (XR) can reshape workforce training, aviation and defense in a safe and effective manner—with humans at the center of leading the charge.
She’s contributed to the FAA’s first formally published guidance on using XR in qualified commercial flight simulation training devices, as part of a broader framework for integrating virtual, augmented and mixed reality into commercial pilot training.
Fussell, a scientist at Aptima, has spent years researching how extended reality can safely improve aviation training. It led to her becoming a technical advisor to the International Civil Aviation Organization Personnel Training and Licensing Panel (PTLP) as part of the U.S. delegation, and a member of the Federal Aviation Administration’s (FAA’s) Air Carrier Training Aviation Rulemaking Committee Flight Simulation Training Device (FSTD) Working Group.
Her research focuses on usability, user experience, transfer of training, and human factors that influence how immersive technologies are adopted in tune with more effective, adaptable and accessible training methods that don’t compromise quality.
She’s been an Aptima for about one year and is in their training, learning and readiness division. Aptima is a tech-enabled services and products company that primarily works with the Department of Defense and other military and government entities to create products or services, as well as long-term upkeep and sustainment.
“A lot of our products and services are focused on the use of compound AI and tools to create and AI to create tools, especially in my division, that aid training and learning, and it’s called readiness,” Fussell told Military.com. “That way, the war fighter has great sustainment over time.”
Her primary role involves working with the Air Force Research Lab’s Gaming Research Integration for Learning Laboratory, or GRILL, where they use commercial and government off-the-shelf products to create test beds and different types of virtual environments that can then be used in training environments or for research purposes.
“A lot of what we do is either on a computer, or in a virtual or mixed-reality environment,” she said. “Part of what I do with GRILL is help design and develop these environments, and then help test them and evaluate them for human factors and usability and objective training performance measures.”
Virtual Training Versus a Real-Life Cockpit
Fussell’s doctoral degree is in aviation, with a focus on immersive simulation technology for flight training purposes in order to better design immersive sim techs for more efficient and effective training.
She said she got mostly into virtual reality, but then it morphed into extended reality technologies more generally to understand the training benefits. That translated into understanding how to design an environment where a flight student could gain proficiencies in the extended reality environment as opposed to a real-world cockpit.
“That takes a lot more time and a lot more resources, obviously, to get up to the aircraft,” she said. “So, we started to explore how virtual reality and other technologies could be used instead of a physical environment. That led me to really focusing my research in that space.”
She worked at Kent State University for a few years as an educator and researcher in the College of Aeronautics and Engineering, where she could test such technologies on students and then write scientific journal articles on thefindings.
That led to guidance issued through the PTLP, and eventually morphed into the current FAA guidance being evaluated and trained around XR.
Differences Between VR and XR
Most people are likely familiar with virtual reality, or VR, as opposed to XR.
Fussell explained that XR is more of the umbrella term for such immersive technologies.
“VR is more of when you or when you’re in a headset on your head, you’re interacting with and seeing virtual elements,” she said. “Whereas [in] mixed reality, there is some physical, real-world element that you can either see or interact with. There’s different levels of overlay.”
That might mean seeing virtual screens and interacting with physical knobs and switches in a cockpit, or a joystick. Within augmented reality, she said, it’s often much more about virtual overlay onto the real world.
A benefit of XR technology is that you can supplement training, she said, and provide more of it. With VR one can purchase a lot of technologies for quicker access and training—or connect devices to apps that can be linked to whatever learning management system the training provider has.
In a similar vein, Fussell said that there’s better access to the training environment via whatever XR technology is utilized.
You can train more efficiently. And by efficiently, I mean you can access it when you need to. You can access it more often. You can access it perhaps without an instructor around.
The difference is time and energy involved, she added. Whereas a very large simulation device would require booked time, it would also necessitate help for setting up. It’s also more difficult to change a virtual environment.
If the tech is all in a headset or in an application, it can provide trainees with more tools quicker, more efficiently, and then they can practice over and over without the resource or typically required resources.
FAA Guidance Takes ‘Literal Years’ to Go Into Effect
Fussell said that the FAA’s drafting involves committees and consultation—and, depending on the type of documentation, it then has to go out to the public for review when people can give feedback which is later incorporated.
“It can take literal years for actual like regulations to be pushed through,” she said.
The recently published memo took a few years to draft, involving 1 1/2 to 2 years of research, information gathering, and actually composing the draft in consultation with training providers and original equipment manufacturers and evaluators.
“There is a need to ensure that there’s no extra burden on the lowest of operators,” Fussell said about the FAA.
Supporting XR in regulations and in guidance goes beyond the traditional sim, allowing a more critical look at requirements while still ensuring a high level of fidelity is reached and a high training equivalency is reached.
“With the guidance that we just put out, some of the parameters are what you would see on the commercial side,” she said. “If the research indicates that certain frame rates are better for the user because they will not induce cyber sickness, then that’s pretty much good enough. We don’t need to go too far down that rabbit hole.
“On the military side, and I will say that for the FAA and most general and commercial aviation sides, they’re not looking into hours and hours of continuous use with an XR device. Maybe it’s smaller chunks of time, 15 to 40 minutes.”
On that military side, Aptima and GRILL are working on an initiative that’s been in the works, ensuring that a trainee can wear them for hours at a time to do high complex maneuvers. The military can determine it’s a strong baseline or foundation, good enough for commercial aviation.
“But now we need to establish: Is that minimum the same as the fighter pilot maneuvers that need to be done—which are obviously more complex from a cognitive load, but also in a sim require more complex manipulations of the environment than, say, ‘I’m in a jumbo jet and I need to land?’ Obviously, that is complex,” she said. “It is a high cognitive load, but it’s still different.
Where XR and Training Goes From Here
So, what does the future hold?
Fussell said that work will delve into how to create evaluation training programs to ensure that equivalent levels of safety are met. That means that students or trainees aren’t incurring negative training and, therefore, a negative transfer of training to the aircraft.
Recommendations are accelerating, though she said there’s “probably a good couple of years’ worth of back and forth” still required. Another thing happening is that different parts of the FAA are looking at where to incorporate XR into the regulations and how to change the requirements for pilots in their training programs.
“A lot of the upcoming guidance will probably be more XR focused because that’s what we’ve been able to get information on, and then the rest of it will be a little bit more vague, so that growth can occur,” she said.
