In-cockpit pilot-fatigue detection on a Raspberry Pi.

This project focuses on developing an edge device for in-cockpit pilot-fatigue detection using a Raspberry Pi. The system aims to monitor physiological signals to identify fatigue before it impairs performance.

• Vision: Utilizes a real-time face pipeline to compute Eye Aspect Ratio (EAR) for blink and microsleep detection, and tracks yawning frequency. Aviation-informed thresholds are used to flag microsleep events.
• Biometrics: Integrates heart rate and heart-rate variability (RMSSD) to generate a stress index, escalating the fatigue assessment based on sustained anomalies.
• Environment: Incorporates cabin conditions like altitude, recognizing that hypoxia can compound fatigue risk.
• Identity: Employs face embeddings for pilot enrollment to establish individual baseline comparisons.

The fused indicators are published continuously over TLS MQTT, a protocol chosen for its efficiency and suitability for intermittent cockpit connectivity. The system prioritizes edge computing to minimize bandwidth, latency, and privacy concerns, ensuring detection capabilities even without a stable network link. The core challenge was achieving real-time computer vision within the Raspberry Pi's resource constraints.

The project involved leading the edge team, which included building the necessary circuitry, guiding the device architecture, and sourcing the hardware. The Python/OpenCV pipeline was developed by teammates, while the ground station's operator frontend and pilot-enrolment API were also part of the lead's contributions.

The complete system, comprising edge devices and a cloud ground station, was a final-year capstone project that achieved a score of 97%. This project highlighted the critical role of hardware limitations in shaping software design.