Driving a see-through car?
If you have ever driven, you will almost certainly have been gripped at one time or another by a sudden feeling of apprehension as another vehicle—a car, a motorcycle or even a bicycle—suddenly looms up alongside yours. You ask yourself, “How come I didn’t see it in the side mirror?”
Mirrors are vital safety kit for checking what is going on behind and alongside a vehicle—areas not in the driver’s direct line of sight. Yet they cannot completely eliminate blind spots. The world’s drivers definitely need an in-vehicle electronic display, linked to cameras mounted on the vehicle, showing everything that is happening around the vehicle. It’s reassuring to know that Toshiba is developing such a solution.
Visual product technology now assists safer driving
“Various pieces of the automotive jigsaw will become electronic as we transition to fully autonomous vehicles,” says Rintaro Akiyama*, who is in Toshiba’s Storage & Electronic Devices Solutions Company. “When it comes to enabling this transition, Toshiba, as a leading electronics manufacturer, should be able to contribute a lot.”
Rintaro Akiyama, Technical Marketing Promotion Dept., Technical Marketing Div., Storage & Electronic Devices Solutions Company, Toshiba Corporation (position as of June 2017)
Rather than conceiving of the electronic mirror as a standalone semiconductor product, Toshiba is developing an integrated solution based on various types of ICs and memory chips. On the assumption that it is a nascent market with tremendous growth potential, Toshiba is working to apply image synthesis technology in the automotive space.
“The project began with the idea of taking the technology for integrating images from multiple sources which was originally developed with visual products in mind, and using it for automotive applications. We have developed technology that enables a driver, in effect, to see through the vehicle. It eliminates blind spots by creating a composite image of side and rear views. Combining the side and rear views was technically tricky because of the different camera angles, but we managed to substantially improve the composite image quality,” says Katsuhiko Araki*, who is in Toshiba’s Corporate Solutions Development Center.
The current technology works on a development platform designed for checking the functional performance. Going forward, plans call for the migration of the technology to a platform for practical automotive applications. No doubt we will have to iron out glitches and refine the monitoring system in the run-up to commercialization, but we expect it to eventually do everything that conventional vehicle mirrors do and much, much more, delivering major gains in driving safety and convenience.
Likely evolution of electronic mirrors
This next-generation cockpit system is an example of an advanced driver assistance system (ADAS). What issues will have to be resolved before electronic mirror-based ADAS goes mainstream?
“We have a lot to do. A marked improvement in the performance of composite video image processing is needed and, of course, we must ensure the technology complies with safety standards. An issue common to all camera-based monitoring systems is the need for real-time responsiveness to changing environmental conditions. For example, the degree of illumination outside the vehicle varies greatly depending on the time of the day or night, whether you are basking in bright sunshine, or lost in fog, or suddenly enter a tunnel. We are examining the possibility of using high dynamic range (HDR) cameras or technologies for enhancing image clarity to get better results,” comments Araki.
Katsuhiko Araki, Audio & Visual Technology Dept., Corporate Solutions Development Center, Toshiba Corporation (position as of June 2017)
Araki also points out how combining the technology with image recognition functions could improve safety by alerting the driver to other vehicles, people or obstacles.
“In future you would want a range of flexible interior displays inside the vehicle, such as a curved display along the side pillars. Using technology to project camera views inside the cabin, it should be possible to eliminate every blind spot,” says Araki.
But what of the prediction that electronic mirrors will become unnecessary once fully autonomous vehicles hit the road, since there would no longer be a human at the wheel?
“Still, we expect cameras to remain in the car and the need for the technology in displaying visual information just as good as mirrors will be required. In terms of image resolution, frame rates and viewing angles, there would also still be a need for more advanced technology to make the video images look more natural,” says Akiyama.
The debut of electronic mirrors will likely be the opening shot of a revolution in the automotive sector.