AI-Powered Airport Ground Crew Operations Monitoring System

Project Overview:

This project is designed to enhance airport logistics, safety monitoring, and operational efficiency using a combination of computer vision, artificial intelligence, and automation. By leveraging Google Gemini AI and OpenAI’s API, the system captures real-time video footage from airport tarmacs, detects ground crew activities, and generates structured reports on operations. It focuses on airport ground handling services, machinery movement, and worker coordination to ensure safety compliance and efficiency.

Key Features & Components:

1. Real-Time Video Processing

• The system captures video footage from a live webcam or pre-recorded video (airport.mp4).
• Frames are processed continuously to detect activities occurring in designated Areas of Interest (ROI).
• OpenCV is used for video capture, frame resizing, and annotation overlays.

2. Region of Interest (ROI) Detection

• Uses a predefined polygonal area to crop specific zones where airport operations occur.
• Ensures that only relevant areas are analyzed, reducing noise and improving AI efficiency.
• ROI is highlighted visually using colored polylines for easy monitoring.

3. AI-Powered Activity Recognition with Google Gemini AI

• Extracts Base64-encoded images from the video frames.
• Sends cropped images to Google Gemini AI via OpenAI API for activity recognition.
• AI returns structured insights in a table format, identifying key operations happening in the selected area.
• Example activities detected:
  • Aircraft refueling
  • Baggage loading/unloading
  • Pushback operations
  • Ground crew coordination

4. Asynchronous Processing for Efficiency

• Uses Python threading to send images to Gemini AI without blocking video processing.
• Ensures smooth and non-disruptive monitoring.
• Processes new frames every 5 seconds, reducing unnecessary API calls.

5. Automated Report Generation & Storage

• AI responses are saved into structured text files for documentation.
• File format: “Airport_Ground_Crew_Operations_{Area}.txt”
• Ensures a historical record of activities, useful for auditing, compliance, and analytics.

6. Debugging & Interactive Monitoring

• Implements mouse event tracking to provide real-time feedback on frame coordinates.
• Allows users to visually inspect the monitored area on-screen.
• Includes keyboard shortcuts (Press ‘Q’ to exit the program).

Technical Workflow:

1. Image Cropping & AI Processing

   • Extract region of interest (ROI) using polygon-based cropping.
   • Convert the cropped image into Base64 format for AI processing.
   • Send the image to Google Gemini AI via OpenAI API.

2. AI Response Handling & Report Generation

   • Retrieve structured activity details from AI-generated responses.
   • Save insights into a text file for analysis and auditing.

3. Real-Time Monitoring & Interaction

   • Display annotated video feed with polylines and labels.
   • Handle mouse events for debugging.
   • Process new frames asynchronously every 5 seconds.

Applications & Use Cases:

🔹 Airport Safety & Compliance Monitoring

• Detects safety violations, unauthorized personnel, or hazardous activities.
• Helps regulatory authorities ensure ground crew follow protocols.

🔹 Operational Efficiency Analysis

• Tracks delays, worker movement, and vehicle coordination on the tarmac.
• Helps airport managers optimize workflows based on real-time insights.

🔹 Automated Documentation & Report Generation

• Stores structured reports for post-event review, audits, and compliance verification.
• Provides a historical record of operations, useful for data-driven decision-making.

🔹 AI-Driven Decision Support for Ground Handling Operations

• Enables predictive analytics for airport logistics.
• Assists in improving turnaround time for flights by streamlining processes.

Technologies Used:

Future Enhancements:

✅ Multi-Area Detection: Expand monitoring to multiple regions in the airport simultaneously.
✅ Live Streaming Support: Integrate RTSP/IP camera feeds for real-time surveillance.
✅ Advanced AI Analysis: Use YOLOv8 for object detection alongside Gemini AI for richer insights.
✅ Automated Alerts: Generate real-time notifications for critical safety violations.
✅ Dashboard Integration: Visualize AI insights in an interactive dashboard for better accessibility.

Conclusion:

This AI-powered Airport Ground Crew Operations Monitoring system combines real-time video analysis, AI-based activity detection, and automated documentation to enhance airport efficiency, safety, and compliance. By using Google Gemini AI with OpenAI’s API, the project provides structured, insightful reports on ground operations, helping airports optimize workflows and ensure regulatory compliance.