Driving environment assessment using fusion of in- and out-of-vehicle vision systems

Because the overall driving environment consists of a complex combination of the traffic Environment, Vehicle, and Driver (EVD), Advanced Driver Assistance Systems (ADAS) must consider not only events from each component of the EVD but also the interactions between them. Although previous researchers focused on the fusion of the states from the EVD (EVD states), they estimated and fused the simple EVD states for a single function system such as the lane change intent analysis. To overcome the current limitations, first, this paper defines the EVD states as driver’s gazing region, time to lane crossing, and time to collision. These states are estimated by enhanced detection and tracking methods from in- and out-of-vehicle vision systems. Second, it proposes a long-term prediction method of the EVD states using a time delayed neural network to fuse these states and a fuzzy inference system to assess the driving situation. When tested with real driving data, our system reduced false environment assessments and provided accurate lane departure, vehicle collision, and visual inattention warning signals.