Cornell researchers have developed a system that anticipates what a driver is going to do a few seconds before it happens.

Some cars are already equipped with safety systems that monitor a car’s movement and provide a warning after the driver has acted.

By observing the driver’s body language and what is happening outside of the car, the new system determines the probability that the driver will turn, change lanes or continue straight ahead.

“There are many systems now that monitor what’s going on outside the car,” explained Ashutosh Saxena, assistant professor of computer science. “Internal monitoring of the driver will be the next leap forward.”

Saxena and graduate student Ashesh Jain will describe their system in a workshop on “Model Learning for Human-Robot Communication” at the 2015 Robotic Science and Systems conference July 16 in Rome.

Combining driver anticipation with radar or cameras to locate other vehicles, the car’s safety system could warn the driver when the anticipated action could be dangerous. The warning might be a light, a sound or even a vibration. “If there’s danger on the left, the left side of the steering wheel or the seat could vibrate,” Jain suggested.

Drawing on street maps and GPS information, the system also might give an “illegal turn” message if the driver was planning to turn the wrong way on a one-way street.

The system still needs refinement, the researchers noted. Six percent of the time, they found, face tracking was confused by shadows of passing trees and other lighting variations. The system also can be misled by drivers interacting with passengers. In some situations, such as turning from a turn-only lane, drivers don’t always give the same head cues. Sometimes they rely on short-term memory of traffic conditions and don’t turn their heads to check. It may come down to tracking eye movements, the researchers said.

Also contributing to the research are graduate student Hema S. Koppula and Stanford University graduate students Bharad Raghavan and Shane Soh. Saxena’s work is supported by the U.S. Army Research Office, the Office of Naval Research and the National Science Foundation.