Pedestrian–Vehicle Interaction Modeling and Risk Assessment

This repository contains three modeling pipelines for pedestrian trajectory prediction and crossing-intention prediction:

1. Baseline LSTM (JAAD)
2. JAAD-only Transformer
3. SUMO-pretrained → JAAD-finetuned Transformer (Single-Agent + Multi-Agent)

---

Project Overview

We model pedestrian behavior in urban traffic scenes using:

• Trajectory prediction (future pedestrian path)
• Intention prediction (Crossing vs Non-Crossing)

Core idea: learn motion + interaction patterns from real-world JAAD, and improve generalization with synthetic SUMO simulations (then fine-tune on JAAD).

---

Dataset: JAAD - Joint Attention in Autonomous Driving

JAAD provides real-world driving videos with pedestrian bounding-box annotations and behavior labels (e.g., crossing/non-crossing).

Example behavior timeline (JAAD)

---

Baseline Model: LSTM (JAAD)

Pipeline Diagram

Summary

• Input: observed pedestrian motion sequence (relative displacements; normalized)
• Backbone: LSTM encoder
• Heads:
  • Trajectory head (future relative displacements)
  • Intention head (crossing vs non-crossing)
• Evaluation:
  • Trajectory: ADE / FDE
  • Intention: Accuracy, Precision/Recall/F1, Balanced Accuracy, AUC

---

JAAD-only Transformer

Pipeline Diagram

Summary

• Input embedding: motion features (REL + NORM) + optional context features
• Encoder: Transformer encoder (multi-head self-attention)
• Heads:
  • CLS token → Intention head
  • Last observed token → Trajectory head
• Loss: Multi-task learning (Trajectory regression + Intention classification)

---

Synthetic Data using SUMO (Simulation of Urban Mobility)

We generate synthetic pedestrian–vehicle interaction sequences using SUMO to cover controlled, diverse traffic conditions.

Scenario Maps

S1: Euclid & University Blvd

S2: Euclid & Speedway

S3: Main Gate Square

Synthetic Data Generation (High-level)

• Run SUMO simulation with configured routes, signal phases, pedestrian flows, and vehicle types
• Export FCD / trajectory logs
• Convert to windowed sequences (T_obs, T_pred) for:
  • Single-Agent (one pedestrian track)
  • Multi-Agent (pedestrian + nearby agents, depending on your implementation)
• Apply the same feature format used by JAAD models (REL + NORM) for compatibility

---

SUMO → JAAD Transfer: Pretrain + Finetune (Single-Agent & Multi-Agent)

Unified Training Diagram

Summary

• Pretraining: Train Transformer on SUMO synthetic windows (learn motion/interaction priors)
• Finetuning: Adapt the pretrained model on JAAD windows
• Tracks supported:
  • Single-Agent Transformer
  • Multi-Agent Transformer
• Outputs (both tracks):
  • Trajectory prediction (ADE/FDE)
  • Intention prediction (Acc/Precision/Recall/F1, Balanced Acc, AUC)

---