Semaxone keeps an eye (and an ear) on the stress levels of pilots

In flight, an airplane pilot must be able to make the right decisions quickly. This cool-headedness is difficult to maintain in an often stressful environment… Semaxone, a start-up present at VivaTech 2022 with IMT, aims to monitor the cognitive state of pilots by studying their cerebral blood flow and analyzing their voice.

Airline and fighter pilots often have to manage complex or critical situations, sometimes in an emergency. This is made more difficult by the extreme conditions in which they work.

Pilots’ mental workload

In flight, in addition to using an artificial oxygen supply due to altitude, pilots are subjected to intense vertical accelerations of up to 10 g, i.e. ten times the gravitational pull at the Earth’s surface. In this situation, the person feels like their weight has been multiplied by ten. The acceleration also causes variations in blood distribution within the body. When the brain is affected, it becomes much more difficult to make decisions or stay conscious. Above 4 or 5 g, the pilot can experience loss of vision or even pass out.

In addition, in the case of an incident, the plane’s devices can also generate stress by emitting multiple alarms at the same time. The pilot must process this mass of information and understand where the problem comes from while maintaining the aircraft in flight. This can have a negative impact on the individual’s cognitive abilities. “The human factor and the ability to maintain one’s cognitive resources, in particular in reaction to a failure or an abnormal situation, are central to flight safety,” says Guilhem Belda, founder of the start-up Semaxone.

Blood analyses

The start-up, incubated at IMT Mines Alès, develops algorithms to detect variations in the physical or cognitive state of pilots. “Our aim is not to provide absolute measurements of stress or fatigue,” says Guilhem Belda. “Instead, we study changes in the individual’s cognitive capacity to respond to different situations in flight, an ability which itself is influence by stress, fatigue, information overload, etc.” The ultimate aim is to help pilots better manage flights (see insert at the end of the article). In collaboration with researchers at IMT Mines Alès, the EuroMov DHM research center and the Avignon Computer Science Laboratory (LIA), Semaxone’s analyzes focus on two main areas.

Firstly, the company develops sensors to study changes in blood flow, and therefore oxygen concentration, in the brain, as well as the activity of certain regions in of the brain. These measurements make it possible to observe how the brain reacts according to the amount of oxygen available. The start-up uses near-infrared spectroscopy, a technique that involves sending infrared light into the skull to calculate the oxygen level in the brain. “This technology is not new,” says the engineer. “The innovation lies in its use in extreme conditions.”

Voices in flight

Secondly, Semaxone focuses on pilots’ voices in flight. This is a much more complex field of study than it may seem. “Speaking requires the mobilization of multiple cognitive processes and a large number of muscles,” says the start-up’s founder. “Physiological variations affect both these aspects.” The idea is to “listen” to the pilot to analyze their emotions, hesitations, breathing and variations in their voice. These elements can all indicate a deterioration in cognitive status.

However, collecting data in flight is a real challenge. “The conditions are nothing like those in the laboratory,” says Guilhem Belda. “It is firstly due to the physical constraints on the sensors, which must be compact and light, but also to the noise recorded during the measurements. How can we tell whether the variations observed in the pilot’s voice are due to stress or a simple change of position?” The start-up therefore concentrates a large part of its efforts on isolating the noise and increasing the reliability of physiological measurements in extreme environments.

Building AI from scratch

Lastly, brain activity and voice data are analyzed by artificial intelligence. The aim is to extract a “library” of physiological markers from this data flow and identify the warning signs of the deterioration of a pilot’s cognitive state from this vast quantity of information. “Our idea is not only to sound the alert if there is a problem, but to anticipate a possible deterioration based on physiological indicators,” explains Guilhem Belda.

More precisely, Semaxone uses machine learning algorithms that require a learning phase before they can be used in real-life conditions. However, there are currently no

exploitable datasets on the cognitive responses of pilots in flight. The start-up therefore has to build one first! “With our partners, we develop highly controlled experiments in which we reproduce the phenomena that interest us in the laboratory,” explains the company’s founder. “We also carry out tests in real-life situations in a simulator or during training flights. For example, we studied the influence of acceleration during an aerobatic exercise and presented pilots with various incidents in an A320 aircraft simulator.”

The company uses these tests to carry out its analyses, the results of which are complemented by a survey among the pilots. The survey responses, information collected during the flight and performance indicators (quality of maneuvers, landing, etc.), are used to identify and define the observed phenomena and the recorded reactions. It is a long process, but one that is necessary before any practical application can be launched.

For aviation… and beyond?

What applications are concerned? Semaxone’s analyzes can be used in two ways. Either by the pilots themselves during debriefings or initial and in-service training, or by aircraft manufacturers, who can improve the ergonomics of their aircraft or design new functionalities, such as warning systems, using the information collected by the start-up.

Could the company then look to new horizons? “We have already thought about adapting our solution to sectors such as nuclear energy and surgery,” says Guilhem Belda. “Generally speaking, we think we can bring added value to all activities requiring the use of cognitive skills in extreme situations.”

Bastien Contreras