Elevating Embedded Vision Systems Using NVIDIA Orin and FPD-Link

VLS3-AR0144-CB

Embedded vision has been fast evolving in the last 5 to 6 years. With the world becoming smarter with the advent of AI, the associated systems are also growing in capability and prominence. Embedded camera technology is one such domain that has leveraged artificial intelligence to find its way into newer applications.

But in embedded vision, it’s not just AI that has led to a monumental shift in how images and videos are captured. The advancements in processing and interface technologies have significantly led to more and more embedded systems manufacturers adopting vision into their products.

NVIDIA has been at the forefront of the innovations happening in processing technology with multiple series of processors capable of powering low-end embedded vision systems to the most data-hungry ones. NVIDIA Jetson Orin, the latest processor series from NVIDIA, is a great example of the huge strides NVIDIA is making in this direction.

In addition, new-age interfaces like FPD-Link from Texas Instruments have made it possible to fully utilize the processing power offered by processor families like NVIDIA Jetson. This combined with state-of-the-art embedded cameras has helped modern systems like robots, drones, mining vehicles, smart traffic devices, etc., perceive and capture images of their environment for performing vision-related tasks.

Today, let’s dive into the exciting world of embedded vision, and how FPD-Link cameras powered by NVIDIA Jetson Orin are paving the way for a new vision-enabled transformation across the industrial, retail, commercial, and medical sectors.

What is FPD-Link?

FPD-Link is a SerDes (Serializer/Deserializer) interface offered by Texas Instruments that is suitable for high-resolution data transfer in automotive and autonomous systems like robots, autonomous tractors, ADAS (Advanced Driver Assistance Systems), etc. It supports uncompressed video with the ability to transfer power and control data over a single cable – giving product designers the flexibility to build a compact and optimized system. It also ensures low latency, which is extremely helpful in autonomous navigation systems where real-time decision is key.

FPD-Link III, the 3^rd generation of FPD-Link technology, boasts a transfer rate of 4.16 Gbps, which meets the needs of even the most sophisticated AI-based vision systems. It is compatible with the MIPI CSI-2 standard and has an operating temperature range of -40 to 105 ⁰C. It should also be noted that FPD-Link III serializers are available with a variety of input interfaces, including LVDS and parallel RGB, making it useful for interfacing with other image sensors as well.

Some of the other features of FPD-Link III include:

Support for multi-camera synchronization.
Supports single-ended coaxial or Shielded Twisted Pair(STP) cable.
A low power consumption of about 0.28W.
Comes with a Power Over Coax (PoC) transceiver.

With these key features, FPD-Link III has become the interface of choice for product engineers building new-age embedded vision systems like smart traffic devices, surround view systems, in-vehicle surveillance systems, etc.

What is NVIDIA Jetson Orin?

NVIDIA Jetson Orin is the latest series of processors from edge computing giant NVIDIA, which they call the ‘world’s most powerful AI computer for energy-efficient autonomous machines.’

According to NVIDIA,

“NVIDIA® Jetson Orin™ modules give you up to 275 trillion operations per second (TOPS) and 8X the performance of the last generation for multiple concurrent AI inference pipelines, plus high-speed interface support for multiple sensors.”

NVIDIA offers seven different modules under three series including Jetson AGX Orin, Jetson Orin NX, and Jetson Orin Nano. While Jetson AGX Orin varies from 200 TOPS to 275 TOPS in AI performance, Jetson Orin NX has two variants with 70 and 100 TOPS. Jetson Orin Nano on the other hand offers 20 and 40 TOPS of AI power with the two modules it offers.

NVIDIA positions the Orin series for next-gen robotics applications. However, it is perfectly suited for similar applications such as other autonomous vehicles (drones, tractors, automated forklifts), smart traffic devices, smart street lighting systems, and sports analytics devices.

Why the Combo of FPD-Link and NVIDIA Orin can be Powerful?

Both NVIDIA Orin and FPD-Link are leading names in their respective categories. Using them together in the same vision system with the right cameras can offer several advantages to product developers – in terms of performance and convenience – such as:

The ability to cater to the complex size and shape of various embedded vision systems.
Enabling long-distance transmission of high-resolution images and videos.
Access to SDKs that help reduce time to market by many months, especially while building robots.
Industrial-grade design for the processor, interface, and cable.

Catering to Complex Sizes and Shapes

As you know, embedded vision systems come in all forms and shapes. Take robots for example. A telepresence robot is different from a delivery robot. A patrol robot is not the same as a lawn mower.

Many robots today come in large sizes extending to a few meters in height. There are large robotic arms that operate in factories that are even longer than 4 to 5 meters.

Given this reality, robots need to have a camera interface that can transmit data over long distances. In addition, the processing platform must be compatible with the interface. The NVIDIA Jetson Orin-FPD-Link combo is a perfect solution for a scenario like this.

Long-distance Transmission of High-Resolution Images and Videos

Using a high-performance processor like NVIDIA Jetson Orin that can support multiple concurrent AI inference pipelines easily meets the processing needs of even the most modern robotic and autonomous systems. FPD-Link on the other hand, facilitates the transmission of image data from the camera to the processor over long distances (even beyond 10 meters).

For example, consider a large mining truck with cameras installed on it to eliminate blind spots. The vehicle will have to use multiple cameras (often more than 4 or 5) to cover the entire surroundings. This would naturally mean that some of the cameras will have to be placed far from the processing unit. Also, if the quality of the video feed must be high, the processing power of the system has to be high accordingly.

Access to NVIDIA SDKs

NVIDIA Jetson Orin shares the same System-on-Chip (SoC) architecture as that of the other Jetson modules. NVIDIA offers an extensive set of software accelerators that help accelerate product development and time to market for embedded vision system manufacturers.

Some of these software accelerators include:

CUDA toolkit
DeepStream SDK
Isaac ROS compatibility

By leveraging these, product developers can start their development activities from day 1 without having to wait around for other components to be built. This is extremely helpful for businesses that intend to hit the market quickly with their products.

Industrial-grade Design for the Interface and Cable

NVIDIA Jetson Orin and FPD-Link systems can be built to withstand and survive adverse industrial conditions such as high temperatures, shocks, and vibrations. All of TechNexion’s FPD-Link cameras, for example, come with a FAKRA connector – ensuring sturdiness and reliability in any kind of automotive or industrial device.

Camera Applications of NVIDIA Jetson Orin and FPD-Link

Robotics is probably where the fastest revolution is happening when it comes to using the combo of NVIDIA Jetson Orin and FPD-Link cameras. From warehouse robots to personal robot assistants and tele-existence robots, Orin and FPD-Link are seeing increased adoption.

Here are a few more applications where we think this trend is going to be more prominent in the coming years:

Large trucks and mining vehicles: Their large size makes them a perfect case for Jetson Orin and FPD-Link.
Smart traffic and parking lot surveillance systems: These devices mostly have cameras fitted a few meters away from the processing unit. And if they involve heavy AI processing, a high-performance processor like NVIDIA Jetson Orin with FPD-Link is a well-suited solution.

Sports analytics systems: Sports analytics systems installed for automated broadcasting purposes often have cameras mounted on a pole with the camera at the top and the processor at the bottom.
Autonomous shopping systems: Automatic checkout systems in retail stores have heavy processing needs. In some cases, cameras and processors are far apart too. Jetson Orin’s unmatched AI performance combined with FPD-Link’s high-bandwidth transmission can be a game-changer here.

It is important to note here that not all scenarios will require NVIDIA Jetson Orin and FPD-Link. Every use case is different. For instance, in certain designs, cameras in smart traffic devices are placed close to the processor, in which case an interface like MIPI might suffice. In essence, you need to choose the solution that fits your application the best.

TechNexion: Enabling Next-gen Robotic Development

TechNexion helps its customers leverage the power of FPD-Link and NVIDIA Jetson Orin by offering a full suite of solutions including camera modules, FPD-Link coaxial cables, Orin-based embedded box computers, and FPD-Link III display panels.

This enables product developers to save significant amounts of time by utilizing ready-to-use components rather than having to build everything from scratch.

TechNexion also supports customers throughout the product development journey. We also can manufacture our products at volume with a state-of-the-art manufacturing facility capable of manufacturing complex PCBs by using Package on Package (PoP) technology (more about that here).

When it comes to camera modules, we offer a wide variety in terms of shutter type (both rolling and global shutter), resolution (up to 13MP), chromaticity (monochrome and color), light sensitivity (visible and Near IR-enhanced cameras), dynamic range, etc. Please watch the video to learn more about our embedded vision solutions:

TechNexion is backed by a great partner ecosystem that helps us support our customers across multiple geographies and product lines. To learn more about our partner ecosystem, please visit the partners page.