AI-Based Retail Human Activity Recognition System

Project Description:

This project showcases a real-time retail environment monitoring system using YOLO with instance segmentation and tracking. It is designed to detect and classify human activities—such as walking, touching, and watching products—from surveillance footage in shopping aisles. The system intelligently overlays bounding boxes, red masks, and bold class labels to provide an interpretable visualization of actions being performed by each person in the scene.

By leveraging a custom-trained YOLO model, this solution is well-suited for applications in loss prevention, customer behavior analytics, and intelligent surveillance.

Core Objectives:

• 🎯 Detect multiple human subjects in busy indoor environments (e.g., retail stores).

• 🎯 Classify specific actions like “walking,” “touching,” and “watching.”

• 🎯 Visualize detected actions with bounding boxes and segment masks in real time.

• 🎯 Enable behavioral tracking to understand customer-product interactions.

• 🎯 Build a framework for anomaly detection and retail analytics.

Key Features:

✅ Activity-Specific Detection: Recognizes multiple concurrent human activities with accuracy.

✅ Instance Segmentation: Applies red masks over each individual to enhance visual distinction.

✅ Tracking with IDs: Uses YOLOv8’s built-in tracking for associating actions with consistent identities across frames.

✅ Bold Action Labels: Provides uppercase activity labels (e.g., “WALKING”, “TOUCHING–WATCHING”) for quick interpretation.

✅ Optimized Inference Loop: Skips unnecessary frames to maintain smooth real-time performance.

✅ Scalable Code Structure: Easily extendable to new environments or activity types (e.g., theft detection, loitering).

Technologies Used:

• 🧠 YOLO (Ultralytics) – For object detection, segmentation, and tracking.

• 🧠 OpenCV – For video I/O, image manipulation, and mouse interactions.

• 🧠 cvzone – For enhanced drawing (e.g., labeled rectangles).

• 🧠 Python – Main language for logic, processing, and orchestration.

Use Cases:

• 🏬 Retail Analytics – Track how customers engage with products.

• 🛡️ Loss Prevention – Detect suspicious behavior like frequent touching or prolonged product watching.

• 🧍‍♂️ Customer Flow Mapping – Understand movement patterns in-store.

• 🎥 Smart Surveillance – Automatically monitor behavior in shopping malls or warehouses.

• 📈 Business Intelligence – Generate data for optimizing shelf layout and product placement.

Possible Enhancements:

• 💾 Event Logging System – Save frame-wise detections with timecodes for audit trails.

• 📊 Behavioral Heatmaps – Show zones where touching or watching is most frequent.

• 🔔 Anomaly Alert System – Alert if an object is touched without being picked or placed back.

• 🌐 Web-Based Dashboard – Live feed display with activity summaries.

• ⚡ Hardware Acceleration – Integrate ONNX/TensorRT for edge device deployment.

Conclusion:

This intelligent retail behavior detection system accurately identifies and visually annotates human activities in real time using YOLO advanced features. By combining instance segmentation, robust tracking, and action classification, it provides retailers and security teams with a powerful tool for automated surveillance, customer engagement analysis, and safety compliance. The clear UI with colored masks and labels ensures both human readability and machine-actionable insights.