Often times in robotics vision applications require high bandwidth, long-range support, data integrity, and camera reliability. MIPI CSI-2, USB 3.0, and GigE are some of the common camera interfaces in these applications. Ethernet and CAN are often used in cars but they aren’t fast enough to handle large amounts of high-resolution video data in modern vehicles.
GSML cameras solves the problem
Gigabit Multimedia Serial Link™ (GMSL) cameras use the GMSL and GMSL2 technology — a SerDes technique — to carry high speed video, bidirectional control data, and power over a single coaxial cable.
What is GMSL?
GMSL is a high-speed serial interface mainly used in automotive video applications like infotainment and advanced driver assistance systems (ADAS). It is utilized in robotic applications or in retail space as security cameras. The GMSL interface includes families of serializers and deserializers with various interfaces such as HDMI, CSI-2, DSI, asymmetric DSI, eDP, oLDI, and single/dual/quad GMSL1/GMSL2 at the input or output. GMSL employs a serializer on the transmitter side to convert data to a serial stream and a deserializer on the receiving side to convert the serial stream back to parallel data for processing. It can transfer video at speeds up to 6 Gb per second.
Leopard Imaging GMSL (Gigabit Multimedia Serial Link) cameras use the MIPI CSI-2 interface. These cameras are connect to high-performance computing platforms, NVIDIA AGX Orin, using a GMSL-to-MIPI CSI-2 bridge. This setup allows for high-speed data transmission suitable for applications requiring robust and reliable video input, such as autonomous vehicles, robotics, and other embedded vision systems.
Here’s a brief overview of how these components connect:
- GMSL Cameras: The cameras use GMSL technology to transmit video data over long distances with high bandwidth.
- Serializer: The GMSL camera includes a serializer that converts the parallel data from the camera into a serial data stream for transmission.
- Deserializer: At the receiving end, a deserializer converts the serial data stream back into parallel data.
- MIPI CSI-2 Interface: The deserialized data is then output through a MIPI CSI-2 interface, which connects to the NVIDIA AGX Orin or other compatible platforms.
This configuration ensures high-speed and reliable video data transfer from the GMSL cameras to the processing unit.
SerDes — Serialize and Deserializer
The NVIDIA Jetson Virtual Channel with GMSL (Gigabit Multimedia Serial Link) Camera Framework supports virtual channels which allow for multiple camera streams to be handled over a single physical connection, utilizing the capabilities of the MIPI CSI-2 interface as shown below.
for 2 virtual channels
IMX390 is a sony camera sensor integrated with Maxim IMX9295 serializer using i2c.
for 4 virtual channels
Virtual Channels: These allow multiple camera streams to be multiplexed over a single physical MIPI CSI-2 interface. Each stream is identified by a unique virtual channel ID, enabling efficient use of bandwidth and reducing the number of physical connections needed.
Jetson Platform: NVIDIA’s Jetson platform is designed for AI and edge computing applications. It supports various camera interfaces, including MIPI CSI-2, which is used in conjunction with GMSL cameras for high-performance vision applications.
Deserializer: A deserializer converts the serialized camera data back into parallel data that the Jetson platform can process. The MAX9296 deserializer, for example, can handle multiple virtual channels and route the data appropriately.
Camera Synchronization: Proper synchronization of multiple camera streams is essential for applications requiring simultaneous data from multiple sensors, such as 3D vision and stereo imaging.
Driver Configuration:
Virtual Channels: The Jetson software stack is configured to recognize and manage multiple virtual channels. Each virtual channel corresponds to a different camera stream.
Processing: The driver configuration ensures that each camera stream is correctly identified, synchronized, and processed. This involves setting up the virtual channels within the driver so that the data from each camera can be used by the system.
Output: The processed and synchronized camera streams are then used for various computer vision and AI applications. This might include tasks like object detection, navigation, and other autonomous functions.
components and data flow in the NVIDIA Jetson Virtual Channel with GMSL Camera Framework
Application Use: The synchronized and processed camera streams are used in various computer vision and AI applications, such as autonomous navigation and robotics.
Deserializer IC (Maxim MAX9296):
Architecture of the MAX9296 IC
IC architecture
Input Interface
GMSL Deserialization: The IC receives serialized data from GMSL serializers over coaxial or STP cables. It converts this serial data back into parallel data streams.
Control Channel: Manages bidirectional control and communication between the serializer and deserializer.
Deserializer
Deserialization Engine: Converts the incoming serialized GMSL data into parallel MIPI CSI-2 data.
Virtual Channel Management: Assigns and manages virtual channels for different data streams, enabling the multiplexing of multiple camera feeds over a single MIPI CSI-2 interface.
Output Interface
MIPI CSI-2 Output: The deserialized and decoded data is output through a MIPI CSI-2 interface, ready to be processed by a downstream device, such as an NVIDIA Jetson module.
Sync & timing
Frame Sync: Provides synchronization signals to ensure that multiple cameras are aligned in time, which is essential for applications like 3D vision or surround-view systems.
Clock Management: Manages clock signals for proper data timing and integrity.
Power management
Voltage Regulators: Integrates power management to ensure stable operation across various input voltages.
Low Power Mode: Supports low power operation modes to reduce energy consumption when the system is idle or in a standby state.
GMSL camera interface
GMSL camera interface meets the growing demand of fast data rates, high bandwidth, data integrity and better EMI/EMC performances. We will look at the similarities and differences between GMSL and MIPI interfaces along the following parameters:
- Distance of transmission
- EMI/EMC performance
- Video duplication
- Automatic Repeat Request (ARQ) feature
- Backward compatibility
- Virtual channel support
- Compatible platforms
Distance transmission:
Generally MIPI® CSI-2 camera connection is 30 cm (max) from camera to process. This is very limiting where distance between camera and the processor is large like in big truck or in retail space. That is where the GMSL SerDes technology can be more helpful enabling a long distance transmission up to a distance of 15m. These cameras can be placed 15 meters away from the host processor through the 50Ω coaxial cable or 100Ω STP to operate at full speed, and still support a high frame rate and less latency.
EMI/EMC performance
Another source of interferecne is the EMI. The programmable output with spread spectrum capability is built into each GMSL serializer and deserializer chip to make camera applications safer. This feature improves the electromagnetic interference (EMI) performance of the connection without needing an external spread-spectrum clock.
Automatic Repeat Request (ARQ) feature
ARQ is an error control and packet recovery method employed in GMSL to ensure reliable data transmission. This method allows the transmitting side to automatically re-transmit packets when it doesn’t acknowledge the packet from the receiver to stay error-free. In GMSL2, ARQ works in combination with cyclic redundancy check (CRC) to detect whether the packets are received or not. This allows high robustness of critical control functions in the system.
Virtual channel support
4 GMSL cameras connected to GMSL deserializer connected to MIPI-CSI 2 interface of Nvidia AGX Orin
Using virtual channels, the SerDes architecture allows for the capture of multiple camera feeds. Virtual channels are supported by the MIPI CSI-2 and CSI-3 specifications. A GMSL deserializer (from LeopardImaging), such as the MAX9296, can decode up to 16 virtual channel IDs when using dual 4-lane MIPI CSI-2. This deserializer merges all incoming video streams and outputs them through CSI-2, with each stream identified by virtual channel packets. It’s important to achieve frame sync locked status in this process.
Cost
The setup is expensive. GMSL deserializer is about $500 from Leopard sensing, there are other companies as well, ConnectTect’s GMSL deserializer is also in the same range.
Other low cost interfaces : MIPI/USB/giga bit ethername
Applications:
Autonomous mobile robots (AMRs) are used for various automated tasks in sectors like warehousing, logistics, healthcare etc. Cameras in AMR and robotics systems requires multiple cameras for navigation, object detection, and object identification. When selecting cameras for robotic applications, fast frame rates and low exposure times are crucial factors. GMSL cameras offer an optimal solution due to their high frame rates and low exposure time capabilities. Additionally, GMSL cameras can transmit data up to 15 meters from the host, making them ideal for remote robotics applications.
