Autonomous Vehicles: Detecting Pedestrians Using YOLOV5

Using Al to identify and track pedestrians in real-time is helping autonomous vehicles become smarter and safer than ever before. Here’s an outline of an indigenous project developed to do just that using YOLOV5, a deep learning object detector.

- By: Abhishek Raj Permani

On a cool evening in Bangalore, as I was coming out of my office, I saw an electric shuttle come to a smooth halt as a child dashed across a crosswalk inside our tech park. No driver. No delay. Just action. That’s when I realised that autonomous vehicles aren’t just futuristic concepts anymore—they’re real and learning how to navigate the very same unpredictable streets that we humans do every day when driving our vehicles.

Behind this seamless stop was an intelligent system trained to spot, track, and respond to every human movement. So let’s talk about what brought about this vision system’s intelligence to life—a system that combines YOLOv5, one of the fastest and best deep learning object detectors, with SORT, a lightweight yet powerful tracking algorithm. Together, these allow machines to detect, recognise and follow pedestrians in each frame as they manoeuvre using the sensor systems they are equipped with.

This isn’t just a lab demo or a theoretical model that we are talking about here. This is a real-world implementation that I have seen and helped to develop. It’s been built with excellently engineered deployment in mind—starting from the way it selects the visual clutter to the way it assigns unique IDs to every person who appears in its view.

And, most importantly, this intelligent AI system prioritises human safety.

The challenge of human movement

While vehicles follow lanes and signals, pedestrians generally don’t. They hesitate sometimes, may reverse their course, may get distracted by phones, or emerge from occluded spaces like other parked vehicles or some tree lines. Tracking such unpredictable movement requires more than just detection—it needs some kind of memory.