The role of embedded vision in building an autonomous world

The world is getting smarter with technological advancements occurring at a rapid pace. The wide adoption of AI has accelerated technology-led transformations in various sectors such as healthcare, education, industrial automation, and logistics. Another such AI-enabled revolution which positively impacts multiple industries is autonomous vehicles.

Autonomous vehicles are intelligent devices that can move from one place to another without the use of external guidance markers, performing tasks such as path planning, obstacle detection, and object recognition. In many cases, they use embedded vision cameras to capture images and videos on the go and analyze them in real time to take necessary action.

In this article, we explore how embedded vision is playing a critical role in building the autonomous vehicles of tomorrow. You will learn about different types of autonomous vehicles in use today and what makes embedded cameras so useful in these applications.

Embedded vision and its rise in popularity

Embedded vision refers to an integrated system consisting of one or more cameras and a host processor, making it possible to combine on-device image processing. In the last decade or so, product developers have started to turn to embedded vision instead of machine vision cameras due to its advantages such as:

Compactness: Since embedded vision systems do not use a dedicated external processing system, the overall form factor can be more compact.
Lower cost: Embedded vision systems tend to be less expensive as an external host is not required.
Edge AI integration: Edge AI is the process of running AI and ML models in the embedded system, instead of an external processor or the cloud.

While embedded vision hasn’t yet captured a major portion of the vision market, it promises huge growth potential, especially with the popularity of compact autonomous systems.

For further reading: Embedded Vision vs. Machine Vision – Everything You Need to Know

Embedded vision and its rise in popularity

Compactness: Since embedded vision systems do not use a dedicated external processing system, the overall form factor can be more compact.
Lower cost: Embedded vision systems tend to be less expensive as an external host is not required.
Edge AI integration: Edge AI is the process of running AI and ML models in the embedded system, instead of an external processor or the cloud.

While embedded vision hasn’t yet captured a major portion of the vision market, it promises huge growth potential, especially with the popularity of compact autonomous systems.

For further reading: Embedded Vision vs. Machine Vision – Everything You Need to Know

Different types of autonomous vehicles

There are many (more than 40) types of autonomous vehicles. We will go over systems where embedded cameras are widely used as the adoption rate is higher.

Autonomous Mobile Robots (AMRs)

Autonomous Mobile Robots, or AMRs, are self-driving robots that can move from one place to another without any human supervision or dedicated external guidance markers. AMRs can also automatically perform tasks like measuring the distance (depth) to and between objects, identifying objects, and handling materials.

Depending on where they are used, AMRs can be classified into:

Warehouse automation robots, such as material handling and picking robots.
Telepresence robots – These are used for remote communication.
Agricultural robots – Robots used for various agricultural tasks such as harvesting robots, automated weeders, and autonomous tractors.
Hospitality and cleaning robots, such as those used in restaurants to move used utensils and dishes from the dining room to the back.
Delivery robots, which are used for automated last-mile delivery.

An autonomous mobile robot (VCI-AR0144-SL)

Not every robot must be autonomous. Some use guided navigation. Autonomous navigation is made possible with AI and embedded cameras, combined with dedicated external markers, such as QR codes on the floor. These markers allow the robot to determine its location in the local space and ensure that it is on the right path. Embedded cameras capture the necessary images and videos, the robot’s processing system processes this data to make navigation decisions.

Drones

Higher-end drones may also navigate autonomously or with guidance. Autonomous navigation in most commercial drones makes use of satellite location systems (e.g. GPS, GNSS). However, for object detection and collision avoidance tasks, embedded vision is used. Using multiple cameras to capture the surroundings, an edge processor performs AI-based analyses to determine the nature of obstacles and distance to objects and can be fed into the flight management system to adjust its flight path in real time to avoid a collision, or to follow a particular object of interest.

A drone

Self-driving cars

Autonomous cars have already started hitting our roads. However, safety concerns and compliance challenges have prevented manufacturers from making cars completely autonomous. Pioneers such as Tesla have taken a data-driven approach to building autonomous cars, where their sophisticated ML model is trained with highly curated data from known-good driving, captured over many years from cameras integrated into their customer’s vehicles. The AI model is further refined based on camera data captured from semi-autonomous (Tesla refers to this as FSD – augmented) vehicles.

Autonomous tractors

One of the biggest challenges faced in the agricultural sector is labor shortage. Normal mechanical tractors, which helped scale agricultural activities, still require human supervision, and driving them around requires many man-hours of work.

This is where autonomous tractors brought about a huge impact. Through autonomous navigation, and using embedded vision, autonomous tractors can operate without human supervision, accomplishing tasks such as plowing fields, spreading fertilizer, and dispensing pesticides.

A single autonomous tractor can have as many as 40+ cameras, depending on the complexity of its functions and its requirements. Autonomous navigation in tractors is made possible in one of three ways:

Global satellite positioning systems such as GPS, which allows the vehicle to determine its location with accuracy.
3D depth cameras such as LiDAR or stereo, allow the vehicle to detect objects and distances to them, as well as ground contours, and furrow tracking.
Combination of 3D and 2D cameras. In this case, data from 2D cameras can be used for object detection but can more so be used to augment navigation.

Automated forklifts

Forklifts are used to lift and move heavy objects in industrial settings. Cameras make automated forklifts possible by allowing them to:

Detect and avoid obstacles by using a combination of 3D and 2D vision.
Measure the depth to objects for lifting and moving them automatically.

With the help of automated forklifts equipped with embedded cameras, warehouse owners can do much more with a significantly leaner workforce, leading to long term cost benefits and reduced safety hazards.

An automated forklift

What makes embedded cameras autonomous-ready?

Autonomy in smart devices is not limited to navigation. There could be other functions in which embedded cameras help these devices gain autonomy.

For example, a sorting robot must automatically sort objects based on certain characteristics, separating them as they move on the conveyor belt, and place them into the right container. This level of autonomy is only possible with embedded cameras and sophisticated AI algorithms.

The following features and characteristics are what make embedded cameras suitable for use in autonomous robots and vehicles:

Higher resolution: Modern camera modules offer a higher resolution without increasing the sensor size, providing higher image quality without making the system bulky. This is critical as more cameras are integrated into a given equipment.
High sensitivity: Back side illumination, and enhanced pixel well depth have allowed sensors to capture more light per pixel and increase sensitivity.
3D depth sensing: Methods such as stereo, time of flight, and structure light have increased precision when it comes to depth measurement.
Increased dynamic range: Using a combination of sensors and ISPs (Image Signal Processors) that deliver high-quality HDR (high dynamic range), operating in outdoor environments is possible for robots and unmanned vehicles.

In addition, some of the other camera features that are enabling automated navigation are:

Global shutter
High frame rate
High speed interface (such as FPD-Link III/IV and GMSL2/3)
Flexible lens mounts
Industrial-grade camera components (an IP68 enclosure for example).

With improvements in camera technology and the ever-increasing interest in autonomous vehicles, the demand for embedded cameras is only going to rise in the coming years.

TechNexion – camera solutions for AI-enabled autonomous vehicles

TechNexion designs and manufactures embedded vision cameras for new-age vision systems. Of the many target applications, autonomous vehicles like robots and drones have been one of our key focus areas. With features such as global shutter, high sensitivity, high resolution, and high dynamic range, our cameras are suited for any autonomous equipment. Visit the embedded vision cameras page to view our complete portfolio of camera solutions.