Image Recognition System
Advanced vehicles equipped with camera vision based advanced driver assistance systems (ADAS) increase rapidly. Such ADAS use one or multiple cameras (monocular or stereo cameras) for automotive to detect and recognize vehicles, pedestrians, traffic signs and so on arround own car, and state of driver and passengers in own car by image recognition techonologies.
Toshiba's image recognition processor LSI Visconti™ Series integrates RISC type media processing engines "MPE" and image processing accelerators suitable for various image processing in the above ADAS. The MPE has 3 grain parallelism architectures, multi-core for task level parallelism, VLIW (Very Long Instruction Word) for instruction level parallelism and SIMD (Single Instruction stream, Multiple Data stream) for data level parallelism, to gain high image data processing performance. The accelerators help real-time image processing under low power consumption. The Visconti™ Series enable various image recognition applications shown below for the above ADAS.
Vision-Based Advanced Driver Assistance Systems(Image Video)
Examples of Image Recognition Applications for ADAS
Toshiba's high-performance Heterogenous multi-core architecture image recognition processor LSIs "Visconti™" Series will help to enable various applications in the ADAS that detects and recognizes in real-time traffic lines, vehicles, pedestrians, traffic signs, traffic signals and so on around a own car, and state of driver and passengers by using multiple cameras.
- Lane Departure Warning (LDW)
- A LDW system warns a driver in highway if the own car may depart from the current travel lane without lane change signal by detecting traffic lanes from image data of a front or rear camera. The Visconti™ Series will help to realize such system.
- Forward Collision Warning (FCW) and Backward Collision Warning (BCW)
- A FCW system detects a vehicle in front of the own car by using image data from a stereo camera or a monocular camera with milliwave rader at the front of the own car and warns a driver of a potential collision risk, based on the distance between the two vehicles and the own vehicle's speed.
A BCW system detects a vehicle in rear of the own car by using image data from a rear monocular camera.
- Traffic Sign Recognition (TSR)
- A TSR system recognizes traffic signs using image data from a front-mounted monocular camera and displays the recognized information of the recognized traffic sign on a dispaly panel. This system may be used to compare the recognized speed limit with the vehicle speed read via CAN and then it notifies a driver. TMPV7504XBG and TMPV7506XBG will help to realize such system.
- Night Vision (NV) and Foward Pedestrian Detection (FPD)
- These systems detect pedestrians with front-mounted camera (far- or near-infrared camera in cace of the NV system) and highlights detected pedestrians on the display panel and alert a driver. Toshiba image recognition processor Visconti™ Series realizes the NV system to detect pedestrians at night with using infrared camera(s). TMPV7506XBG will help to realize the NV and FPD systems to detect pedestrians at daytime and/or night time.
- Parking Assistance with Bird's Eye View
- This system captures images around a vehicle from four fish-eye or wide angle lens cameras and converts them to bird's-eye view point image and displays synthesized image from the above 4 images and a top view image of the own vehicle to assist when parking. TMPV7506XBG has four channels of color camera inputs, an image processing acclerator for converting image data and a video output interface with synthesis of image data. The TMPV7506XBG will enable easily to such system.
- Driver State Monitoring (DSM)
- A DSM system tracks the driver's face direction and notifies if the driver is not paying attention to the ahead.
- Obstacle Detection and Lane Change Assistance (LCA)
- An obstacle (vehicles) detection system detects obstacles such as vehicles around own car for lane change assistance and forward collision warning. It uses three monocular cameras (one at the front and one each at the rear-right and rear-left).
A LCA system detects rear vehicles to assist to change lane by using 2 image data from rear-left and rear-right cameras.
- Left/Right-turn Awareness
- A left/right-turn awareness system detects pedestrians, bicycles and motorcycles using cameras mounted on either the left or right side of the own vehicle and warns the driver of the danger of hitting them. TMPV7506XBG will help to realize such system.
- Backover Prevention (BOP)
- A backover prevention system detects pedestrians with rear-mounted camera(s) and highlights the detected pedestrians on the display panel and alerts the driver when a vehicle goes back. TMPV7506XBG will help to realize such system.
Examples of Advanced Driver Assistance Systems
- ACCS: Adaptive Cruise Control System
- LKAS: Lane Keep Assist System
- PCSS: Pre-Crash Safety System
Recommended Products
| Part Number | Product Name | Main Features | Package |
|---|---|---|---|
| TMPV7506XBG (Visconti™2) |
Image recognition processor | One Toshiba original 32-bit RISC CPU MeP:Media embedded Processor, Four Toshiba original 32-bit RISC media processing engines (MPEs), Video input I/Fs (Up to 4 inputs, Up to 1.3Mpixel size, Color/Grayscale cameras), Video output I/F (Up to WVGA size, RGB888), Six image processing accelerators, DDR2 SDRAM controller, NOR Flash/SRAM controller, External MCU interface, PCI express, etc. | 516-pin PBGA |
| TMPV7504XBG (Visconti™2) |
One Toshiba original 32-bit RISC CPU MeP:Media embedded Processor, Four Toshiba original 32-bit RISC media processing engines (MPEs), Video input I/Fs (Up to 2 inputs, Up to 1.3Mpixel size, Color/Grayscale cameras), Video output I/F (Up to WVGA size, RGB888), Five image processing accelerators, DDR2 SDRAM controller, NOR Flash/SRAM controller, External MCU interface, etc. | ||
| T5BG3XBG (Visconti™) |
Three Toshiba original 32-bit RISC media processing engines (MPEs), Video input I/Fs (Up to 3 inputs, Up to VGA size, Grayscale cameras), Video output I/F (Up to VGA size, RGB888), Affine transformation accelerator, SDRAM controller, NOR Flash/SRAM controller, External MCU interface, etc. | 456-pin PBGA | |
| TX4964FG | Automotive Display Controller | Toshiba 64-bit RISC core (TX49/L4); graphics display controller with per-pixel alpha-blending; 2D graphic accelerator; communication modules (2-channel full-CAN controller, SIO/UART, I2C); system control (24-bit RTOS system timer, watchdog timer, 3-channel 16-bit timer) | 176-pin LQFP |
| TX4966XBG | 64-bit TX49/L4 Core, 2-channel graphics display controller (DGC) with per-pixel alpha-blending, 2D/3D graphic accelerator, 2-channel camera/video frame grabber, APIX, Communication modules (4-channel full CAN, ESEI, SIO/UART, I2C, I2S), System control (24-bit RTOS system timer, watchdog timer, PWM, 8-channel 16-bit timer) | 456-pin PBGA |
Product Introduction
* System and product names mentioned herein may be trademarks or registered trademarks of respective companies or organizations.